Custom debug engines in Visual Studio 2022?

Question

What's the currently accepted way to create a custom debug engine for Visual Studio 2022 or 2026?

I've found a tutorial for "Building a Debug Engine Using ATL COM", but it's clearly for an older version of Visual Studio (perhaps 2013). VS 2022 does not have the ATL wizards described in the tutorial, and the code generated by the non-ATL wizards is not complete.

I could install VS 2015 and go through all the steps, but I'm worried I'll end up with something that doesn't work with VS 2022.

(I'm doing this because I've made a custom CPU with a custom C-like programming language. I'm using VS to write code and run the compiler. I'd like to use it for debugging too. I prefer the VS UI to VSCode.)

Thank you.

Answer 1

To create a custom debug engine (DE) for Visual Studio 2022, you should follow the steps outlined in the Visual Studio SDK documentation. The process involves:

1. Registering the DE with Visual Studio: This includes setting up the necessary registry entries so that Visual Studio can recognize your debug engine.
2. Enabling a program to be debugged: You need to ensure that your DE can attach to or launch the program you want to debug.
3. Implementing execution control and state evaluation: This involves coding the logic to control program execution and evaluate the state of the program being debugged.
4. Sending events: Your DE should be able to send events back to Visual Studio to update the UI and inform it about the debugging state.
5. Setting up termination and detaching: Finally, you need to handle the cleanup when debugging sessions end.

The documentation also provides a roadmap for extending the debugger, which includes samples and reference information that can help you customize the debugger further. You can find specific guidance on creating a custom debug engine and the necessary components in the Visual Studio SDK documentation.

For your specific case of debugging a custom CPU with a custom programming language, ensure that your DE is tailored to support the unique features and requirements of your environment.

It's also worth noting that while older tutorials may provide some insights, the best approach would be to refer to the latest documentation for Visual Studio 2022 to ensure compatibility and leverage the latest features.

---

References:

• Create a custom debug engine
• Roadmap for extending the debugger

Answer 2

Thanks for reaching out!

 

Here are the steps and considerations for creating a custom debug engine:

1. Understand the Basics: A debug engine (DE) interacts with the operating system or interpreter to handle tasks like execution control, breakpoints, and expression evaluation. You need to determine which runtime your custom language will target—be it CLR for managed code or Win32 for native applications. If your language targets one of these, Visual Studio provides the necessary debug engines, and you may only need to implement an expression evaluator.
2. Creating a Custom DE: Here’s a high-level overview of the steps to create a custom debug engine:
   • Register the DE with Visual Studio.
     • Enable debugging capabilities for your language.
       • Implement execution control and state evaluation functions.
         • Manage debugging events through the DE interfaces.
           • Set up termination and detaching mechanisms for the debug session. For more detailed steps, you can check out this guide on creating a custom debug engine.
3. Debugging the DE: You can debug your custom debug engine by:

• Opening two instances of Visual Studio: one for your DE project and another for your main project that launches the DE.
• Attaching the debugger from your DE project to the process you want to debug. For detailed instructions, take a look at the debugging custom debug engines guide. Let me know if you need any further help with this. I will be happy to assist. If you find this helpful, Kindly mark the provided solution as "Accept Answer", so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

Answer 3

I've discovered two things about the "Tutorial: Building a Debug Engine Using ATL COM":

First, to add ATL Simple Objects in VS 2022, you don't do "Add Class...". You do "Add", "New Item", and select ATL on the left. Then ATL Simple Object appears as an option on the right.

Secondly, the tutorial has you create multiple ATL Simple Objects. I found that I started getting errors when trying to add them. I'm guessing, but I'm not sure, that it's because , the tutorial has you remove their interfaces from the TextInterpreter.idl file. So instead, I went through the whole tutorial and created all the ATL Simple Objects at the beginning. That worked.