Debugging 3ds Max plug-ins using Visual Studio

Overview

Developers may use the Visual Studio IDE to debug 3ds Max plug-ins. This provides source code level debugging capabilities while execution is taking place inside the context of the plug-in. One can do things such as set breakpoints (specific lines in the source code where execution stops temporarily so the developer can examine variables, review the call stack, etc.) and establish variables to 'watch'. These variables appear in a separate window and display their current contents as they change.

Setting Up for Debugging

Developers must use the plug-in project's Hybrid configuration for debugging. The Release build does not contain any source level information and thus cannot be used. The Debug configuration is only available for registered members of the Autodesk Developer Network using a debug build of 3ds Max. For more information see Debug Builds of 3ds Max.

You can set the Hybrid configuration as the current one by choosing Hybrid from the Configuration drop down list on the standard toolbar. Then choose Build > Build from the main menu. This will be create a DLL and associated files appropriate for use while debugging.

Next, VC++ needs to know where the 3ds Max executable is. You can set this through the IDE by opening the project property pages dialog. Then choose the Configuration > Debugging folder. In the Command field, enter in the full pathname to 3DSMAX.EXE. In the 'Working directory' field enter the full path to the 3ds Max install.

Setting Up Public Symbols

Autodesk provides public debugging symbols for 3ds Max. These symbols contain useful debugging information, and allow plug-in developers to see their function calls in the context of the calls 3ds Max makes. For more information on symbols, see Introduction to Symbols on MSDN.

To set up Autodesk public symbols in Visual Studio 2012:

1. Go to Tools Options Debugging Symbols
2. Add http://symbols.autodesk.com to the list of symbol file locations, and make sure it is selected (checked)
3. As you start debugging, Visual Studio will pull the symbols corresponding to the modules being loaded into the debugger from the symbol server into the local symbol cache.

For other debuggers, set the _NT_SYMBOL_PATH environment variable as follows:

set _NT_SYMBOL_PATH =
srv*path_to_local_symbol_cache*http://symbols.autodesk.com/symbols

Executing 3ds Max and Your Plug-in

There are several ways to cause execution to halt at a specific spot in your code. One is to set a breakpoint at the spot. This is done by placing the cursor on the line where you want execution to stop and pressing F9. Then, to run the debugger simply press F5. This will launch 3ds Max and get things going. 3ds Max will run and load your plug-in as usual upon startup. Begin execution of your plug-in as you normally would inside 3ds Max (for example, if your plug-in is a modifier apply it to an object). When execution of your plug-in's code reaches the breakpoint, control will return to VC++ and you may use its tool to examine the state of things (see the section below for more details).

A second way to reach a certain line in your code is to use 'Run to Cursor' option. This is chosen by placing the cursor on the desired line of your source file and selecting 'Build/Start Debug/Run to Cursor' (or by pressing Ctrl+F10). This will begin execution and stop when the point in your source code where the cursor sits.

Once execution is stopped, you can begin again by pressing F5, or pressing F11 to step to the next line of source or into the next function or method. Note that if you attempt to step into 3ds Max's code (not your own) you'll wind up with a window showing the disassembly of the source (since no debugging information is available for 3ds Max's internal code.) If this happens you can simply close this window and set another breakpoint inside your own code.

Examining Variables and the Call Stack

A Variables Windows is available to examine variables within the program's current context. This window also has a dropdown list for the call stack that shows the hierarchy of functions that are pending completion. This is handy in the case of a program crash since it can often be used to show the sequence of calls made up to the point of the crash.

Also available is a Watch Window. This will display the contents of any variable and updates automatically as the variable changes.

For additional details on these and other debugging tools see the VC++ IDE online help.