This topic provides a series of commonly-encountered problems and their solutions when writing 3ds Max plug-ins.
Problem 1: Your DLL compiles and links fine, but when you run 3ds Max it complains that LibVersion() is not implemented. You know that you implemented this, so what is going on?
Solution: Did you remember to create a .DEF file with this listed, and if so did you remember to include the file in your project?
Problem 2: When you link InitCommonControls comes up unresolved.
Solution: Did you remember to link in COMCTL32.LIB (which is not included by default by Visual C++ in the list of libraries to link)?
Problem 3: You have a dialog proc to process a button in your user interface. The code is in place to process the button but it never gets called.
Solution: You may have a macro redefinition for one of the buttons in another header file in your project. The compiler takes the first definition and uses it instead of your definition. Therefore yours never gets processed. The compiler will flag this condition as a warning but not as an error.
Problem 4: When you de-allocate memory the system crashes. You have checked that the item being de-allocated exists and is OK to delete. What is happening?
Solution: In Visual C++ the debug and release C runtime libraries use different heap management. This means that allocating an object in debug mode and de-allocating it in release mode (and vice versa) can cause a crash. You need to change your settings for your DEBUG configuration. Choose Build/Settings... from the pulldown menus. From the Settings For: section choose your Debug configuration. From the C/C++ tab, under the Categories: drop down list choose Code Generation. From the Use runtimelibrary: drop down list choose Multithreaded DLL instead of Debug Multithreaded DLL.
Problem 5: You have a plug-in which when compiled under the hybrid settings works fine. When you go to make it in release mode, it works fine except for any time it is used and 3ds Max is exited you get a "Runtime error. R6017 unexpected multithread lock error".
Solution: The project settings for the Release configuration in the code generation section for run-time library are set to DLL instead of Multithreaded DLL. Changing to Multithreaded DLL corrects the problem.
Problem 6: You have created a plug-in that runs fine on certain machines but on other machines it simply won't load.
Solution: You probably have a missing system DLL. The easiest way to see what DLLs your plug-in requires is to use the DUMPBIN.EXE program (which comes with Visual C++ and is usually found in the bin folder) and use the /IMPORTS switch. This will tell you all of the DLLs that your DLL depends upon. Make sure the machines that need to run the plug-in have the proper DLLs.
For example: dumpbin importsutility.dlu
Problem 7: You have a plug-in whose interface won't show up in the command panel yet everything seems fine code wise. What's wrong?
Solution: You may have forgotten to enter the class names for the custom controls in the VC++ dialog editor. See the topic Custom User Interface Controls -- especially the section How to a Create a Rollup Page using the Custom Controls for a description of the values to be used in the Class field of the Custom Control Properties dialog.
Problem 7: AppWizard doesn't work as expected in Vista.
Solution: You may not be running Visual Studio with administrator privileges. When the wizard tries to save its options (e.g. 3ds Max SDK path) it may get a permission denied error because it is trying to write to the Program Files directory, and it doesn't have the appropriate level of privilege.