Symbolicating Unmanaged Code Crashes

Article
03/14/2024

Important

Visual Studio App Center is scheduled for retirement on March 31, 2025. While you can continue to use Visual Studio App Center until it is fully retired, there are several recommended alternatives that you may consider migrating to.

Learn more about support timelines and alternatives.

App Center Diagnostics supports symbolicating unmanaged C/C++ code crashes in your application. You can symbolicate unmanaged code crashes that originate in your Android NDK code, and unmanaged code crashes formatted as Breakpad minidumps uploaded through the Upload Crashes API.

Using Breakpad with App Center

Breakpad is a library and tool suite that helps produce C and C++ stack traces. These stack traces are generated from minidump files produced once Breakpad is integrated with your project. Breakpad offers starter guides for integrating with Linux, Windows, and Mac applications.

Sending Breakpad Crash Logs to App Center

Android NDK

App Center offers an SDK integration for Android applications using the NDK to run unmanaged code. This integration will create Breakpad minidumps and automatically upload them to App Center for you.

Other Applications

Once Breakpad is integrated with your application, you can Upload Breakpad crash logs and minidumps to App Center.

Symbolicating Unmanaged Crashes

Generating Breakpad Symbols Payload

App Center Diagnostics requires symbols to generate a readable stack trace from a Breakpad minidump. The uploaded symbols archive must either contain .sym files, which are produced using the Breakpad dump_syms tool, or .so binary files.

Generate a .zip file to upload

There are two ways for App Center to retrieve the symbols necessary for symbolication. App Center can generate them from the native binaries used in your project, or you can upload the Breakpad symbols directly.

Option 1: Upload native binaries

Put all .so files from the project's obj/local/$ABI/ directory into a .zip file.

Option 2: Upload Breakpad symbols

Dump the symbols using the Breakpad toolchain as described in the Breakpad documentation under section "Get the debugging symbols".
Create a symbols.zip file with the following structure:

Note

If you're uploading your symbols from macOS, then you must clean your symbols of any extraneous folders, e.g. __MACOS gets generated and to delete this you can use zip -d <symbols.zip> __MACOSX/\*.

$ unzip -l symbols.zip
Archive:  symbols.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  07-22-13 15:07   symbols/
        0  07-22-13 15:07   symbols/libnative.so/
        0  07-22-13 15:07   symbols/libnative.so/EAC901FB6DDCCE8AED89E1A8E4A58360/
    12468  07-22-13 15:07   symbols/libnative.so/EAC901FB6DDCCE8AED89E1A8E4A58360/libnative.so.sym
        0  07-22-13 15:07   symbols/libnative.so/FDC5C9E715C4F16408C0B78F95855BF0/
    12467  07-22-13 15:07   symbols/libnative.so/FDC5C9E715C4F16408C0B78F95855BF0/libnative.so.sym
 --------                   -------
    24935                   6 files

Uploading Symbols to App Center

Symbols can be uploaded through the App Center Portal, API, or CLI.

Log into App Center and select your app.
In the left menu, navigate to the Diagnostics section and select Symbols.
In the top-right corner, click Upload symbols and upload the file.
After the symbols are indexed by App Center, crashes will be symbolicated for you.

The process for uploading symbols through the API involves a series of three API calls: one to allocate space on our backend, one to upload the file, and one to update the status of the upload. The body of the first API call should set symbol_type to Breakpad.

Trigger a POST request to the symbol_uploads API. This call allocates space on our backend for your file and returns a symbol_upload_id and an upload_url property.

curl -X POST 'https://api.appcenter.ms/v0.1/apps/{owner_name}/{app_name}/symbol_uploads' \
    -H 'accept: application/json' \
    -H 'X-API-Token: {API TOKEN}' \
    -H 'Content-Type: application/json' \
    -d '{JSON BODY}'

Using the upload_url property returned from the first step, make a PUT request with the header: "x-ms-blob-type: BlockBlob" and supply the location of your file on disk. This call uploads the file to our backend storage accounts. Learn more about PUT Blob request headers .

curl -X PUT '{upload_url}' \
    -H 'x-ms-blob-type: BlockBlob' \
    --upload-file '{path to file}'

Make a PATCH request to the symbol_uploads API using the symbol_upload_id property returned from the first step. In the body of the request, specify whether you want to set the status of the upload to committed (successfully completed) the upload process, or aborted (unsuccessfully completed).

curl -X PATCH 'https://api.appcenter.ms/v0.1/apps/{owner_name}/{app_name}/symbol_uploads/{symbol_upload_id}' \
    -H 'accept: application/json' \
    -H 'X-API-Token: {API TOKEN}' \
    -H 'Content-Type: application/json' \
    -d '{ "status": "committed" }'

Note

The symbol uploads API doesn't work for files that are larger than 256MB. Use the App Center CLI to upload these files. You can install the App Center CLI by following the instructions in our App Center CLI repo.

Ignoring symbols

When App Center doesn't have all the symbol files to fully symbolicate crash reports, the crashes are listed in the Unsymbolicated tab. The required symbols are uploaded from this page if you have access to them.

If you can't upload the symbols, you can mark them as Ignored by selecting rows in the table and clicking the Ignore versions button. This button tells App Center to process the crashes and symbolicate them as fully as possible with the symbols on file. Once they've finished processing, they'll appear in the Crashes tab partially symbolicated. New crashes that also depend on those same symbol IDs marked as ignored will bypass the Unsymbolicated tab as they come in and flow through the system.

Share via