Obtaining Azure Sphere device debug output

  • Article

When developing an IoT Solution you often need to have access to debug output from devices. While this can be achieved through a debug connection to Visual Studio/Code (but can also be achieved from the command line), you may also need to view debug output for devices that are not connected to Visual Studio/Code. Such devices may be running long-term tests, or possibly even deployed to production. There are several options for getting access to debugging data:

  • Obtaining debug output from a device connected to a development PC
  • Obtaining debug output for a device not connected to a development PC
  • Logging to external storage
  • Logging to Azure

Obtaining debug output from a device connected to a development PC

Problem: How to obtain debug output when debugging using Visual Studio/Visual Code?

Options:

  • High-level application

    An Azure Sphere high-level application can use the Log_Debug API to send debug output with printf style formatting to an attached PC during debugging. This output can be viewed using Visual Studio or Visual Studio Code debug window, or from the command line.

    You may want to consider setting up debug verbosity flags in your application, and using CMake compile definitions to control how much debug output is displayed when your application runs. In your CMakeLists.txt file you can create a compile time definition:

    add_compile_definitions(DEBUG_FLAG)

    In your high-level application code you can then increase or decrease the amount of debug output displayed by your application using #ifdef, for example:

    #ifdef DEBUG_FLAG
        Log_Debug("My Message\n");
    #endif

  • Real-time capable application

    An Azure Sphere real-time capable application (running on one of the M4 cores) can write debug/log information to a dedicated M4 transmit-only UART. This requires a USB/Serial adapter such as an FTDI Friend, and a terminal emulator.

    The Azure Sphere Hello World sample demonstrates how to print to the M4 debug UART.

    There are also sample applications available from CodeThink and MediaTek:

    Debug flag compile time definitions can also be used in real-time capable (M4) applications.

  • Using inter-core communications to send state from a real-time capable application to a high-level application

    If you are building a system that combines a high-level and real-time capable application you might want to use the high-level application to log system state for both applications. Inter-core communications can be used for this — the Azure Sphere Inter-core Communication sample implements a simple interface for passing a message between a high-level and real-time capable application.

    This Azure Sphere learning module demonstrates how to use Azure Sphere and Azure RTOS, combined with an inter-core messaging model to pass custom messages between the cores.

Obtaining debug output for a device not connected to a development PC

Problem: How to log debug output when your device is not connected to a development PC?

Options:

  • Send debugging output over the network or a UART

    Obtaining debug log information when a device is connected to a development PC is fairly straightforward. However, you may also want to collect debug/log information when a device is not connected to a PC. For example, you may have a set of devices running long-term tests.

    If your devices are connected to a network, you might want to send the debug output over the network to an application that can collect and analyze the information. This Azure Sphere Gallery application demonstrates how to override the default Log_Debug behavior to send and receive that output over a UDP socket.

    Note that the mechanism used to override the default high-evel application Log_Debug behavior can also be used to send the debug log information to other places, for example, to output the data on one of the Azure Sphere UARTs. You can use this UART sample as a reference to combine with the UDPDebugLog Gallery application to log your messages to a UART.

  • Send debugging output to Azure IoT Hub/Azure IoT Central

    While debugging output can be useful for diagnosing problems as they happen, it can also be useful to store telemetry/log information from a device for post-processing.

    Setting up an instance of Azure IoT Hub or Azure IoT Central gives you an endpoint to send device telemetry data that may be used as part of your business logic, you can also send device state/log information which can be treated separately from the telemetry/business data.

    The Logging to Azure sample demonstrates how to forward log messages as IoT Hub telemetry and store them within an Azure Data Explorer cluster for advanced querying.

    • Azure IoT Hub

      Your Azure IoT Hub instance could be configured to send data to an Azure Database for storage/analysis, you may also want to filter the messages, which can be achieved through EventHub and an Azure Function.

      For Azure IoT Hub you can send data to an Azure Function which can then process the messages. The Azure IoT Hub trigger for Azure Functions article explains how to link an Azure Function to an Azure IoT Hub instance.

    • Azure IoT Central

      Azure IoT Central includes the capability to continually export your data to various endpoints, including:

      • Azure Event Hubs
      • Azure Service Bus Queue
      • Azure Service Bus topic
      • Azure Blob Storage
      • Webhook

      The WebHook is a REST API Endpoint that you create, this could be an Azure Function.

    Note that you may want to filter messages in your Azure Function based on the Device ID that's sending the data, you can obtain the Device ID using the following code in the Azure Function:

    public static async Task Run(EventData message, ILogger log)
    {
        var deviceId=message.SystemProperties["iothub-connection-device-id"];

        // Code to filter the messages goes here...
    }

Azure IoT Hub and Azure IoT Central support Device Twins, which includes desired state (set in the IoT Hub/Central application), and reported state (state from the device). You can use Azure IoT Hub/Central Device Twin to set a desired state for log data verbosity (increase/decrease logging frequency, or the richness of logging data). The Azure IoT Sample demonstrates how to handle Device Twin Desired State changes.

Logging data to storage

Azure Sphere supports up to 64KB of mutable storage for a high-level application, this can be used to persist settings, application state, or other data, application developers are responsible for serializing/deserializing data to mutable storage - The Azure Sphere Gallery includes a project that shows how to use a Key/Value pair (dictionary) to write/read state to mutable storage (up to 64KB depending on how the application manifest is configured).

You may want to write more than 64KB of Log/State data to some form of external storage, this can be useful for devices that have intermittent connectivity, or need to store/retrieve more than 64KB of data.

One option is to add external storage, perhaps using SPI flash to store the data - this project uses SPI flash to store an over-the-air update for a downstream device, and could be modified to support logging telemetry/state data from an Azure Sphere application.