Debug a bot with inspection middleware

APPLIES TO: SDK v4

This article describes how to debug a bot using inspection middleware. This feature allows the Bot Framework Emulator to debug traffic into and out of the bot, and to see the current state of the bot. You can use a trace message to send data to the Emulator and then inspect the state of your bot in any given turn of the conversation.

We use an EchoBot built locally using the Bot Framework v4 in the Create a bot quickstart to show how to debug and inspect the bot's message state. You can also Debug a bot using IDE or Debug with the Bot Framework Emulator, but to debug state you need to add inspection middleware to your bot. The inspection bot samples are available for C#, JavaScript, Java, and Python.

Note

The Bot Framework JavaScript, C#, and Python SDKs will continue to be supported, however, the Java SDK is being retired with final long-term support ending in November 2023. Only critical security and bug fixes within this repository will be undertaken.

Existing bots built with the Java SDK will continue to function.

For new bot building, consider using Power Virtual Agents and read about choosing the right chatbot solution.

For more information, see The future of bot building.

Prerequisites

• Knowledge of bot Middleware and Managing state
• Knowledge of how to Debug an SDK-first bot and Test and debug with the Emulator
• An install of the Bot Framework Emulator
• An install ngrok (if you want to debug a bot configured in Azure to use other channels)
• A copy of the inspection bot sample for C#, JavaScript, Java, or Python

Update your Emulator to the latest version

Before using bot inspection middleware to debug your bot, update your Emulator to version 4.5 or later. Check the latest version for updates.

To check the version of your Emulator, select Help, then About in the menu. You'll see the current version of your Emulator.

Update your bot code

C#

The inspection state and inspection middleware are configured in the Startup.cs file and then used by the adapter.

Startup.cs

});

services.AddSingleton<ConversationState>();

// Create the Bot Framework Authentication to be used with the Bot Adapter.

AdapterWithInspection.cs

{
    public class AdapterWithInspection : CloudAdapter
    {
        public AdapterWithInspection(BotFrameworkAuthentication auth, IConfiguration configuration, InspectionState inspectionState, UserState userState, ConversationState conversationState, ILogger<IBotFrameworkHttpAdapter> logger)
            : base(auth, logger)
        {
            // Inspection needs credentials because it will be sending the Activities and User and Conversation State to the emulator
            var credentials = new MicrosoftAppCredentials(configuration["MicrosoftAppId"], configuration["MicrosoftAppPassword"]);

            Use(new InspectionMiddleware(inspectionState, userState, conversationState, credentials));

            OnTurnError = async (turnContext, exception) =>
            {
                // Log any leaked exception from the application.
                logger.LogError(exception, $"[OnTurnError] unhandled error : {exception.Message}");

                // Send a message to the user
                await turnContext.SendActivityAsync("The bot encountered an error or bug.");
                await turnContext.SendActivityAsync("To continue to run this bot, please fix the bot source code.");

                // Send a trace activity, which will be displayed in the Bot Framework Emulator
                await turnContext.TraceActivityAsync("OnTurnError Trace", exception.Message, "https://www.botframework.com/schemas/error", "TurnError");
            };
        }

Update the bot class in the EchoBot.cs file.

EchoBot.cs

protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
    var conversationStateProp = _conversationState.CreateProperty<CustomState>("customState");
    var convProp = await conversationStateProp.GetAsync(turnContext, () => new CustomState { Value = 0 }, cancellationToken);

    var userStateProp = _userState.CreateProperty<CustomState>("customState");
    var userProp = await userStateProp.GetAsync(turnContext, () => new CustomState { Value = 0 }, cancellationToken);

    await turnContext.SendActivityAsync(MessageFactory.Text($"Echo: {turnContext.Activity.Text} conversation state: {convProp.Value} user state: {userProp.Value}"), cancellationToken);

    convProp.Value++;
    userProp.Value++;
}

JavaScript

Before updating your bot's code, update its packages to the latest versions by executing the following command in your terminal:

npm install --save botbuilder@latest

Then you need to update the code of your JavaScript bot as follows. You can read more here: Update your bot's code or refer to the Inspection middleware sample on GitHub.

index.js

Set up the inspection state and add the inspection middleware to the adapter in the index.js file.

// Import required bot services.
// See https://aka.ms/bot-services to learn more about the different parts of a bot.
const {
    CloudAdapter,
    ConversationState,
    InspectionMiddleware,
    InspectionState,
    MemoryStorage,
    UserState,
    ConfigurationBotFrameworkAuthentication
} = require('botbuilder');

const { MicrosoftAppCredentials } = require('botframework-connector');

const botFrameworkAuthentication = new ConfigurationBotFrameworkAuthentication(process.env);

// Create adapter.
// See https://aka.ms/about-bot-adapter to learn more about how bots work.
const adapter = new CloudAdapter(botFrameworkAuthentication);

// Create the Storage provider and the various types of BotState.
const memoryStorage = new MemoryStorage();
const inspectionState = new InspectionState(memoryStorage);
const userState = new UserState(memoryStorage);
const conversationState = new ConversationState(memoryStorage);

// Create and add the InspectionMiddleware to the adapter.
adapter.use(new InspectionMiddleware(inspectionState, userState, conversationState, new MicrosoftAppCredentials(process.env.MicrosoftAppId, process.env.MicrosoftAppPassword)));

// Catch-all for errors.
adapter.onTurnError = async (context, error) => {
    // This check writes out errors to console log .vs. app insights.
    // NOTE: In production environment, you should consider logging this to Azure
    //       application insights. See https://aka.ms/bottelemetry for telemetry
    //       configuration instructions.
    console.error(`\n [onTurnError] unhandled error: ${ error }`);

bot.js

Update the bot class in the bot.js file.

this.onMessage(async (context, next) => {
    // get the state objects
    const testConversationState = await this.conversationStateAccessor.get(context, { count: 0 });
    const testUserState = await this.userStateAccessor.get(context, { count: 0 });

    // print the current state to the reply to show we are incrementing it
    await context.sendActivity(`You said '${ context.activity.text }' conversation-state: ${ testConversationState.count } user-state: ${ testUserState.count }`);

    // do the actual increment
    testConversationState.count++;
    testUserState.count++;

    // By calling next() you ensure that the next BotHandler is run.
    await next();
});

Java

Set up the inspection state and add the inspection middleware to the adapter in the Application.java file. The inspection state is set by providing a new Spring @Bean to supply the BotFrameworkHttpAdapter that is set to be @Primary so it will override the default BotFrameworkHttpAdapter provided by the BotDependencyConfiguration base class. See the code update below or refer to the Inspection middleware sample on GitHub.

Application.java

/**
 * Create an adapter with InspectionMiddleware.
 *
 * <p>
 * NOTE: This is marked as @Primary to override the default Bean.
 * </p>
 *
 * @param configuration     The configuration.
 *                          {@link BotDependencyConfiguration#getConfiguration()}
 * @param inspectionState   The InspectionState.
 *                          {@link BotDependencyConfiguration#getInspectionState(Storage)}
 * @param userState         The UserState.
 *                          {@link BotDependencyConfiguration#getUserState(Storage)}
 * @param conversationState The ConversationState.
 *                          {@link BotDependencyConfiguration#getConversationState(Storage)}
 * @return An AdapterWithInspection object.
 */
@Bean
@Primary
public BotFrameworkHttpAdapter getInspectionBotFrameworkHttpAdapter(
    Configuration configuration,
    InspectionState inspectionState,
    UserState userState,
    ConversationState conversationState
) {
    return new AdapterWithInspection(
        configuration,
        inspectionState,
        userState,
        conversationState
    );
}

AdapterWithInspection is implemented as part of the com.microsoft.bot.integration package and can be reviewed from the Java SDK source code.

Update the bot class in the EchoBot.java file.

EchoBot.java

@Override
protected CompletableFuture<Void> onMessageActivity(TurnContext turnContext) {
    // Get state data from ConversationState.
    StatePropertyAccessor<CustomState> dataAccessor =
        conversationState.createProperty("customState");
    CompletableFuture<CustomState> convStateFuture =
        dataAccessor.get(turnContext, CustomState::new);

    // Get profile from UserState.
    StatePropertyAccessor<CustomState> profileAccessor =
        userState.createProperty("customState");
    CompletableFuture<CustomState> userStateFuture =
        profileAccessor.get(turnContext, CustomState::new);

    return convStateFuture.thenCombine(userStateFuture, (convProp, userProp) -> {
        convProp.setValue(convProp.getValue() + 1);
        userProp.setValue(userProp.getValue() + 1);

        return turnContext.sendActivity(
            MessageFactory.text(
                String.format(
                    "Echo: %s conversation state %d user state %d",
                    turnContext.getActivity().getText(), convProp.getValue(),
                    userProp.getValue()
                )
            )
        );
    }).thenApply(resourceResponse -> null);
}

Python

Before updating your bot's code, install the necessary PyPI packages by running the following commands in a terminal:

pip install aiohttp
pip install botbuilder-core>=4.7.0

Set up the inspection state in the app.py file by adding a middleware to the adapter.

app.py

from botbuilder.core import (
    ConversationState,
    MemoryStorage,
    TurnContext,
    UserState,
)
from botbuilder.core.inspection import InspectionMiddleware, InspectionState
from botbuilder.core.integration import aiohttp_error_middleware
from botbuilder.integration.aiohttp import CloudAdapter, ConfigurationBotFrameworkAuthentication
from botbuilder.schema import Activity, ActivityTypes
from botframework.connector.auth import MicrosoftAppCredentials

# Create adapter.
# See https://aka.ms/about-bot-adapter to learn more about how bots work.
ADAPTER = CloudAdapter(ConfigurationBotFrameworkAuthentication(CONFIG))

# Catch-all for errors.
USER_STATE = UserState(MEMORY)
CONVERSATION_STATE = ConversationState(MEMORY)

# Create InspectionMiddleware
INSPECTION_MIDDLEWARE = InspectionMiddleware(
    inspection_state=InspectionState(MEMORY),
    user_state=USER_STATE,
    conversation_state=CONVERSATION_STATE,
    credentials=MicrosoftAppCredentials(
        app_id=CONFIG.APP_ID, password=CONFIG.APP_PASSWORD
    ),
)
ADAPTER.use(INSPECTION_MIDDLEWARE)

# Create Bot
BOT = EchoBot(CONVERSATION_STATE, USER_STATE)

Update the bot class in the echo_bot.py file.

bots/echo_bot.py

async def on_message_activity(self, turn_context: TurnContext):
    # Get the state properties from the turn context.
    user_data = await self.user_state_accessor.get(turn_context, CustomState)
    conversation_data = await self.conversation_state_accessor.get(
        turn_context, CustomState
    )

    await turn_context.send_activity(
        MessageFactory.text(
            f"Echo: {turn_context.activity.text}, "
            f"conversation state: {conversation_data.value}, "
            f"user state: {user_data.value}"
        )
    )

    user_data.value = user_data.value + 1
    conversation_data.value = conversation_data.value + 1

Test your bot locally

After updating the code, you can run your bot locally and test the debugging feature using two Emulators: one to send and receive messages, and the other to inspect the state of messages in debugging mode. To test your bot locally:

1. Go to your bot's directory in a terminal and execute the following command to run your bot locally:

   C#

   dotnet run
   

   JavaScript

   npm start
   

   Java

   mvn package
   java -jar .\target\bot-inspection-sample.jar 
   

   Python

   python app.py
   

2. Open your Emulator. Select Open Bot. Fill in the Bot URL with http://localhost:3978/api/messages and the MicrosoftAppId and MicrosoftAppPassword values. If you have a JavaScript bot, you can find these values in your bot's .env file. If you have a C# bot, you can find these values in the appsettings.json file. For a Java bot you can find these values in the application.properties file. Select Connect.

3. Now open another Emulator window. This second Emulator window will work as a debugger. Follow the instructions as described in the previous step. Check Open in debug mode and then select Connect.

4. At this point you'll see a command with a unique identifier (/INSPECT attach <identifier>) in your debugging Emulator. Copy the whole command with the identifier from the debugging Emulator and paste it into the chat box of the first Emulator.

   Note

   A unique identifier is generated every time when the Emulator is launched in debug mode after you add the inspection middleware in your bot's code.

5. Now you can send messages in the chat box of your first Emulator and inspect the messages in the debugging Emulator. To inspect the state of the messages, select Bot State in the debugging Emulator and unfold values on the right JSON window. You'll see the state of your bot in the debugging Emulator:

   

Inspect the state of a bot configured in Azure

If you want to inspect the state of your bot configured in Azure and connected to channels (like Teams) you'll need to install and run ngrok.

Run ngrok

At this point, you've updated your Emulator to the latest version and added the inspection middleware in your bot's code. The next step is to run ngrok and configure your local bot. Before running ngrok you need to run your bot locally.

To run your bot locally:

1. Go to your bot's folder in a terminal and set your npm registration to use the latest builds

2. Run your bot locally. You'll see your bot expose a port number like 3978.

3. Open another command prompt and go to your bot's project folder. Run the following command:

   ngrok http 3978
   

4. ngrok is now connected to your locally running bot. Copy the secure (HTTPS) public IP address.

   

Update your bot resource

Now that your local bot is connected to ngrok, you can configure your bot resource in Azure to use the ngrok URL.

1. Go to your bot resource in Azure. On the left menu, under Settings, select Configuration.

   1. Set the Messaging endpoint to the ngrok IP address you copied. If necessary add /api/messages after the IP address. For example, https://e58549b6.ngrok.io/api/messages.

   2. Select Enable Streaming Endpoint.

      

   3. Select Apply to save your changes.

      Tip

      If Apply isn't enabled, you can uncheck Enable Streaming Endpoint and select Apply, then check Enable Streaming Endpoint and select Apply again. You need to make sure that Enable Streaming Endpoint is checked and the configuration of the endpoint is saved.

2. Go to your bot's resource group.

   1. Select Deployment, and then select the bot resource that previously deployed successfully. Select Template from the left menu to get the MicrosoftAppId and MicrosoftAppPassword for the web app associated with your bot.

      

   2. Update your bot's configuration file (appsettings.json for C#, or .env for JavaScript) with the MicrosoftAppId and MicrosoftAppPassword.

3. Start your Emulator, select Open Bot, and enter http://localhost:3978/api/messages in the Bot URL. Fill Microsoft App ID and Microsoft App password with the same MicrosoftAppId and MicrosoftAppPassword you added to our bot's configuration file. Then select Connect.

4. Your running bot is now connected to your bot resource in Azure. To test your bot in Azure in Web Chat, go to your bot resources, select Test in Web Chat, and send messages to your bot.

Enable debugging mode

1. In your Emulator, select Debug, then Start Debugging.

2. Enter the ngrok IP address (don't forget to add /api/messages) for the Bot URL (for example, https://e58549b6.ngrok.io/api/messages).

   1. For Microsoft App ID, enter your bot's app ID.
   2. For Microsoft App password, enter your bot's app secret.
   3. Make sure Open in debug mode is checked as well.
   4. Select Connect.

3. With the debugging mode enabled, the Emulator generates a UUID. A UUID is a unique ID generated every time you start the debugging mode in your Emulator.

4. Copy and paste the UUID to the Test in Web Chat chat box for your channel's chat box. You'll see the message "Attached to session, all traffic is being replicated for inspection" in the chat box.

You can start debugging your bot by sending messages in the configured channel's chat box. Your local Emulator will automatically update the messages with all the details for debugging. To inspect your bot's state of messages, select Bot State and unfold the values in the right JSON window.

Next steps

• Learn how to Debug your bot using transcript files.
• Learn how to Debug a skill or skill consumer.