Tutorial 6 - Attaching the VSCode debugger to the 3ds Max process

It is possible with VSCode to debug C++ and Python at the same time.

Here is an overview of how this works:

Adding the proper extensions (if not already installed in VS Code)

Adding debug configurations

        {
            "name": "(Windows) Attach",
            "type": "cppvsdbg",
            "request": "attach",
            "processId": "${command:pickProcess}"
        },

The Python configuration for remote attach is also needed, something like:

        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "localhost",
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "${workspaceFolder}"
                }
            ]
        }

Starting the app and enabling python debugging

For the Python part of the debugger we still need to enable it manually in our python host by typing this on the listener:

import sys
import os
import debugpy

def startup():
    sysexec = sys.executable
    (base, file) = os.path.split(sys.executable)
    if file.lower() == "3dsmax.exe":
        sys.executable = os.path.join(base, "python", "python.exe")
        host = "localhost"
        port = 5678
        debugpy.listen((host, port))
        print(f"-- now ready to receive debugging connections from vscode on (${host}, ${port})")
        sys.executable = sysexec

startup()

For 3dsmax.exe, if we want to attach to the process with the debugger, we need to launch the process manually first, by doing something like 3dsmax.exe -python3.

Attaching to the running application

The two configurations are independently started in VSCode: multiple debuggers can run at the same time and they operate pretty much independently (e.g. when a python breakpoint is hit the C++ part will appear as running).

To attach to the process, first choose the "(Windows) Attach (3dsmax)" item in the Run and Debug dropdown and then press the green triangle to start it.

To attach to python, first choose the "Python: Remote Attach (3dsmax)" item in the Run and Debug dropdown and then press the green triangle to start it.

Debugging

From the moment the configurations are attached to the running application it becomes possible to select them to debug one or the other aspect of the running application.

At the same moment the application may appear stopped on a python breakpoint and running (not stopped) in C++.

Usefulness

This dual mode debugging can be especially useful to debug C++ code that invokes the python interpreter (or that deals with it in some way). It is a bit more complicated than only debugging python, and the python part is the same.