Analyzing AppSource submission validation telemetry

APPLIES TO: Business Central 2021 release wave 1, version 18.4, and later

When you submit an app to AppSource using Partner Center, it starts an automated validation process. This technical validation process ensures the extensions in the app meet the technical requirements for going live. It goes through many of the same checks described in technical validation.

If an app's set up for it, telemetry traces are emitted to and recorded in Application Insights. The data provides details about the success or failure of different phases of the validation. For more information about setting up telemetry for an app, see Sending Extension Telemetry to Azure Application Insights.

Note

In order to start analyzing your validation results, use this troubleshooting guide Dynamics 365 Business Central Troubleshooting Guide (TSG) - AppSource Submission Results (SaaS).

Validation process overview

The validation process starts when you publish the app. The validation runs against each extension in the app, for each country/region (market) specified for the offer in Partner Center, and for each Business Central version that the submissions targets. For more information versions, see Against which releases of Business Central is your submission validated?.

Extensions are validated using the AL compiler and the AppSourceCop code analyzer. Traces are emitted at different phases during the process. Each submission is assigned a unique identifier (ID). This ID is included in each trace for a submission, allowing you to query all trace related to the submission. The general flow for the validation process is illustrated below:

1. AppSource submission validation request started
2. Validation diagnostic reported for information, warnings, or errors that occur on the submission
3. Version (X), country-region (X) validation started
4. Extension (X) validation started
5. Validation diagnostic reported for each error that occurs.
6. Extension (X) validation completed (successfully or with failures)
7. Repeat 3-5 for each extension in the submission
8. Version (X), country-region (X) validation completed (successfully or with failures)
9. Repeat 2-7 for other country-regions for the Business Central release (X)
10. Repeat 2-8 for other Business Central releases targeted by the submission
11. AppSource submission validation request completed (successfully or with failures)

To reduce the risk of failing the AppSource validation process, review the technical validation checklist before you submit an app.

AppSource submission validation request started

Occurs when an app is published from Partner Center. For more information about publishing from Partner Center, see Review and publish a Dynamics 365 offer - Validation and publishing steps.

General dimensions

Dimension	Description or value
message	AppSource submission validation request started: {validationRequestId}

Custom dimensions

Dimension	Description or value
eventId	LC0028
countryRegions	Lists the localized versions (markets) of the app that will be validated, like US or DK.
extensions	Specifies information about the extensions that are part of the submission, which will be validated.
severity	Information
versions	Lists the Business Central release versions that the app will be validated against, like 19.0 or 18.4
containsHotfix	Specifies whether the submission contains a hotfix of an AppSource extension.

Common custom dimensions

The following table explains other custom dimensions that are common to all AppSource submission validation traces.

Dimension	Description or value
validationRequestId	Specifies the unique identifier assigned to the app submission validation process. All traces for the submission validation will include this ID.
telemetrySchemaVersion	Specifies the version of the Business Central telemetry schema.

Diagnostic reported on AppSource submission validation request

Occurs during the submission validation request to report diagnostics related to the submission itself. For example, it could be that a submission has duplicate apps, or it doesn't target any Business Central release. This signal isn't necessarily an error, but can also be warning or information. This signal differs from other diagnostic signals like LC0034, which are reported during the compilation of one app for one country/region release.

General dimensions

Dimension	Description or value
message	Diagnostic reported on AppSource submission validation request: {validationRequestId}
severityLevel	1 for information, 2 for warning, 3 for error

Custom dimensions

Dimension	Description or value
eventId	LC0038
diagnosticCode	Specifies the diagnostic identifier, like AS0001 or AL0001.
diagnosticMessage	Specifies the diagnostic message.
diagnosticSeverity	Info, Warning, or Error. Matches the severityLevel.
containsHotfix	Specifies whether the submission contains a hotfix of an AppSource extension.
See common custom dimensions

Sample KQL code

This KQL code can help you get started analyzing request warnings using the LC0038 event:

traces
| where timestamp > ago(1d) // adjust as needed
| where customDimensions.eventId == 'LC0038'
| project timestamp 
, requestId = customDimensions.validationRequestId
, diagnosticCode = customDimensions.diagnosticCode
, diagnosticMessage = customDimensions.diagnosticMessage
, diagnosticSeverity = customDimensions.diagnosticSeverity
, containsHotfix = customDimensions.containsHotfix
, message

To learn more about how to setup alerting on Business Central telemetry, see Alert on Telemetry.

(Version, country-region) validation started

Occurs when the validation has started for a specific version and country/region.

General dimensions

Dimension	Description or value
message	(Version, country-region) validation started: version {version}, country/region {countryRegion}
severityLevel	1

Custom dimensions

Dimension	Description or value
eventId	LC0030
allExtensions	Specifies information about the extensions that are part of the submission and extensions that the submission extensions depend on. Select the arrow to expand the dimension to see the details about each extension.
baselineExtensions	Specifies the previous versions of the extensions in the app and the extensions they're dependent on. These extensions form the baseline for validation if publishing a newer version of an app that's already published.
extensions	Specifies the extensions that are part of the submission and will be validated. Select the arrow to expand the dimension to see the details about each extension.
severity	Information
version	Specifies the Business Central release versions that the app will be validated against, like 19.0 or 18.4.
containsHotfix	Specifies whether the submission contains a hotfix of an AppSource extension.
See common custom dimensions

Extension validation started

Occurs when the validation for a specific extension the submission has started.

General dimensions

Dimension	Description or value
message	Extension validation started: version {version}, country/region {countryRegion} for extension {extensionName} version {extensionVersion} by {extensionPublisher} ({extensionId})
severityLevel	1

Custom dimensions

Dimension	Description or value
eventId	LC0032
countryRegion	Specifies the localized version of the app that will be validated.
extensionId	Specifies the ID of the extension in the submission that will be validated.
extensionName	Specifies the name of the extension in the submission that will be validated.
extensionPublisher	Specifies the publisher of the extension in the submission that will be validated.
extensionVersion	Specifies the version of the extension in the submission that will be validated.
severity	Information
version	Specifies the Business Central release that the extension will be validated against, like 19.0 or 18.4.
hotfixValidation	Specifies whether the extension validated is a hotfix.
See common custom dimensions

Validation diagnostic reported

Occurs when an error occurs during the validation of an extension. The errors are reported by the AL compiler or by the AppSourceCop analyzer.

General dimensions

Dimension	Description or value
message	Validation diagnostic reported: version {version}, country/region {countryRegion} for extension {extensionName} version {extensionVersion} by {extensionPublisher} ({extensionId})
severityLevel	3

Custom dimensions

Dimension	Description or value
eventId	LC0034
countryRegion	Specifies the localized version of the app that was validated.
diagnosticCode	Specifies the diagnostic identifier, like AS0001 or AL0001.
diagnosticMessage	Specifies the diagnostic error message.
diagnosticSeverity	Error
diagnosticSourceLocation	Specifies the location in the source code where the diagnostic was reported. The format used is (Line, Character).
diagnosticSourcePath	Specifies the path to the file where the diagnostic was reported.
extensionId	Specifies the ID of the extension in the submission that will be validated.
extensionName	Specifies the name of the extension in the submission that was validated.
extensionPublisher	Specifies the publisher of the extension in the submission that was validated.
extensionVersion	Specifies the version of the extension in the submission that was validated.
severity	Error
version	Specifies the Business Central release that the extension was validated against, like 19.0 or 18.4.
hotfixValidation	Specifies whether the extension validated is a hotfix.
See common custom dimensions

Sample KQL code

This KQL code can help you get started analyzing request failures using the LC0034 event:

traces
| where timestamp > ago(1d) // adjust as needed
| where customDimensions.eventId == 'LC0034'
| project timestamp 
, countryRegion = customDimensions.countryRegion
, version = customDimensions.version
, extensionId = customDimensions.extensionId
, extensionName = customDimensions.extensionName
, extensionPublisher = customDimensions.extensionPublisher
, extensionVersion = customDimensions.extensionVersion
, diagnosticCode = customDimensions.diagnosticCode
, diagnosticMessage = customDimensions.diagnosticMessage
, diagnosticSeverity = customDimensions.diagnosticSeverity
, diagnosticSourceLocation = customDimensions.diagnosticSourceLocation
, diagnosticSourcePath = customDimensions.diagnosticSourcePath

To learn more about how to setup alerting on Business Central telemetry, see Alert on Telemetry.

Extension validation completed successfully

Occurs when the validation for a specific extension the submission has completed, and no errors occurred.

General dimensions

Dimension	Description or value
message	Extension validation completed successfully: version {version}, country/region {countryRegion} for extension {extensionName} version {extensionVersion} by {extensionPublisher} ({extensionId})
severityLevel	1

Custom dimensions

Dimension	Description or value
eventId	LC0033
countryRegion	Specifies the localized version of the app that was validated.
extensionId	Specifies the ID of the extension in the submission that will be validated.
extensionName	Specifies the name of the extension in the submission that was validated.
extensionPublisher	Specifies the publisher of the extension in the submission that was validated.
extensionVersion	Specifies the version of the extension in the submission that was validated.
severity	Information
version	Specifies the Business Central release that the extension was validated against, like 19.0 or 18.4.
hotfixValidation	Specifies whether the extension validated is a hotfix.
See common custom dimensions

Extension validation completed with failures

Occurs when the validation for a specific extension the submission has completed, but errors occurred.

General dimensions

Dimension	Description or value
message	Extension validation completed with failures: version {version}, country-region {countryRegion} for extension {extensionName} version {extensionVersion} by {extensionPublisher} ({extensionId})
severityLevel	3

Custom dimensions

Dimension	Description or value
eventId	LC0037
countryRegion	Specifies the localized version of the app that was validated.
extensionId	Specifies the ID of the extension in the submission that will be validated.
extensionName	Specifies the name of the extension in the submission that was validated.
extensionPublisher	Specifies the publisher of the extension in the submission that was validated.
extensionVersion	Specifies the version of the extension in the submission that was validated.
failureReason	One or more error diagnostics were reported. For more information about the diagnostics, see traces with eventId LC0034.
severity	Error
version	Specifies the Business Central release that the extension was validated against, like 19.0 or 18.4.
hotfixValidation	Specifies whether the extension validated is a hotfix.
See common custom dimensions

Sample KQL code

This KQL code can help you get started analyzing request failures using the LC0037 event:

traces
| where timestamp > ago(1d) // adjust as needed
| where customDimensions.eventId == 'LC0037'
| project timestamp 
, requestId = customDimensions.validationRequestId
, countryRegion = customDimensions.countryRegion
, version = customDimensions.version
, extensionName = customDimensions.extensionName
, failureReason = customDimensions.failureReason

To learn more about how to setup alerting on Business Central telemetry, see Alert on Telemetry.

(Version, country-region) validation completed successfully

Occurs when the validation has completed for a specific version and country/region, and no errors occurred.

General dimensions

Dimension	Description or value
message	(Version, country-region) validation completed successfully: version {version}, country/region {countryRegion}
severityLevel	1

Custom dimensions

Dimension	Description or value
eventId	LC0031
allExtensions	Specifies information about the extensions that were part of the submission and extensions that the submission extensions depend on. Select the arrow to expand the dimension to see the details about each extension.
baselineExtensions	Specifies the previous versions of the extensions in the app and the extensions they're dependent on. These extensions form the baseline for validation if publishing a newer version of an app that's already published.
countryRegion	Specifies the localized version of the app that was validated.
extensions	Specifies the extensions that were part of the submission and were validated. Select the arrow to expand the dimension to see the details about each extension.
severity	Information
version	Specifies the Business Central release that the app was validated against, like 19.0 or 18.4.
containsHotfix	Specifies whether the (Version, country-region) validation contains a hotfix of an AppSource extension.
See common custom dimensions

(Version, country-region) validation completed with failures

Occurs when the validation has completed for a specific version and country/region, and errors occurred.

General dimensions

Dimension	Description or value
message	(Version, country-region) validation completed with failures: version {version}, country-region {countryRegion}
severityLevel	3

Custom dimensions

Dimension	Description or value
eventId	LC0036
allExtensions	Specifies information about the extensions that were part of the submission and extensions that the submission extensions depend on. Select the arrow to expand the dimension to see the details about each extension.
baselineExtensions	Specifies the previous versions of the extensions in the app and the extensions they're dependent on. These extensions form the baseline for validation if publishing a newer version of an app that's already published.
countryRegion	Specifies the localized version of the app that was validated.
extensions	Specifies the extensions that were part of the submission and were validated. Select the arrow to expand the dimension to see the details about each extension.
failureReason	One or more extension validation tasks have failed.
severity	Error
version	Specifies the Business Central release that the app was validated against, like 19.0 or 18.4.
containsHotfix	Specifies whether the (Version, country-region) validation contains a hotfix of an AppSource extension.
See common custom dimensions

Sample KQL code

This KQL code can help you get started analyzing request failures using the LC0036 event:

traces
| where timestamp > ago(1d) // adjust as needed
| where customDimensions.eventId == 'LC0036'
| project timestamp 
, requestId = customDimensions.validationRequestId
, countryRegion = customDimensions.countryRegion
, version = customDimensions.version
, extensions = customDimensions.extensions
, allExtensions = customDimensions.allExtensions
, baselineExtensions = customDimensions.baselineExtensions
, failureReason = customDimensions.failureReason

To learn more about how to setup alerting on Business Central telemetry, see Alert on Telemetry.

AppSource submission validation request completed successfully

Occurs when the submission validation process has fully completed, and no errors occurred.

General dimensions

Dimension	Description or value
message	AppSource submission validation request completed successfully: {validationRequestId}
severityLevel	1

Custom dimensions

Dimension	Description or value
eventId	LC0029
countryRegions	Lists the localized versions (markets) of the app that were validated, like US or DK.
extensions	Specifies information about the extensions that were part of the submission and validated.
severity	Information
versions	Lists the Business Central release that the app was validated against, like 19.0 or 18.4.
containsHotfix	Specifies whether the submission contains a hotfix of an AppSource extension.
See common custom dimensions

AppSource submission validation request completed with failures

Occurs when the submission validation process has fully completed, but errors occurred.

General dimensions

Dimension	Description or value
message	AppSource submission validation request completed with failures: {validationRequestId}
severityLevel	3

Custom dimensions

Dimension	Description or value
eventId	LC0035
countryRegions	Lists the localized versions (markets) of the app that were validated, like US or DK.
extensions	Specifies the extensions that were part of the submission and validated.
failureReason	One more extension validation tasks have failed.
severity	Error
versions	Lists the Business Central releases that the app was validated against, like 19.0 or 18.4.
containsHotfix	Specifies whether the submission contains a hotfix of an AppSource extension.
See common custom dimensions

Sample KQL code

This KQL code can help you get started analyzing request failures using the LC0035 event:

traces
| where timestamp > ago(1d) // adjust as needed
| where customDimensions.eventId == 'LC0035'
| project timestamp 
, requestId = customDimensions.validationRequestId
, countryRegions = customDimensions.countryRegions
, versions = customDimensions.versions 
, extensions = customDimensions.extensions
, failureReason = customDimensions.failureReason

To learn more about how to setup alerting on Business Central telemetry, see Alert on Telemetry.

Related information

Telemetry overview
Sending App Telemetry to Azure Application Insights
Alert on Telemetry
Technical Validation Checklist
Technical Validation FAQ