Analyzing outgoing web service request telemetry

Article
09/29/2023

APPLIES TO: Business Central 2020 release wave 2 and later

Note

Azure Active Directory is now Microsoft Entra ID. Learn more

Outgoing web service request telemetry gathers data about outgoing web service requests sent using the AL HTTPClient module. As a partner, the data gives you insight into the execution time and failures that happen in external services that your environment and extensions depend on. Use the data to monitor environments for performance issues caused by external services, and be more proactive in preventing issues from occurring.

General dimensions

The following table explains the general dimensions included in an outgoing Web Services Call (Outgoing) trace. The table lists the dimensions that are specific to Business Central.

Dimension	Description or value
message	Web Service Called (Outgoing): {endpoint}
severityLevel	1

Custom dimensions

The following table explains the custom dimensions included in a Web Services Call trace.

Dimension	Description or value
aadTenantId	Specifies the Microsoft Entra tenant ID used for Microsoft Entra authentication. For on-premises, if you aren't using Microsoft Entra authentication, this value is common.
alAuthenticationMethod	Specifies the user authentication used by the Business Central service . Values include: Windows, UserName, NavUserPassword, AccessControlService. For more information about the authentication types, see Authentication and Credential Types.
alHttpTimeout	Specifies the timeout defined for the request. The timeout is the time to wait before a request gets canceled. The value has the format hh:mm:ss. The timeout is defined either by the NavHttpClientMaxTimeout setting on the Business Central Server instance or by a TimeOut method call in extension code. The TimeOut method call takes precedence.
alObjectId	Specifies the ID of the AL object that made the request.
alObjectName	Specifies the name of the AL object that made the request.
alObjectType	Specifies the type of the AL object that made the request.
alStackTrace	The stack trace in AL. Introduced in version 20.1.
clientType	Specifies the type of client that opened the session, such as Background or Web. For a list of the client types, see ClientType Option Type. Introduced in version 21.3.
companyName	The current company name.
component	Dynamics 365 Business Central Server
componentVersion	Specifies the version number of the component that emits telemetry (see the component dimension.)
deprecatedKeys	A comma-separated list of all the keys that have been deprecated. The keys in this list are still supported but will eventually be removed in the next major release. We recommend that update any queries that use these keys to use the new key name.
endpoint	Specifies the endpoint for the request. The endpoint is cleaned to include only the base URI.
environmentName	Specifies the name of the tenant environment. See Managing Environments.
environmentType	Specifies the environment type for the tenant, such as Production, Sandbox, Trial. See Environment Types
eventId	RT0019
extensionId	Specifies the appID of the extension that made the request.
extensionName	Specifies the name of the extension that made the request.
extensionPublisher	Specifies the publisher of the extension that made the request.
extensionVersion	Specifies the version of the extension that made the request.
httpHeaders	Specifies the http headers set in the request. Introduced in version 17.2.
httpMethod	Specifies the HTTP method used in the outgoing request. Values include: POST, GET, PUT, PATCH, or DELETE.
httpReturnCode	Deprecated in version 17.2. Use the dimension httpStatusCode instead. Specifies the http status code returned when a request has completed. This dimension further indicates whether request succeeded or not, and why. Use it to verify whether there was an issue with a request even though the request was logged as successful. The dimension displays one of the following values: 200 OK. The request succeeded. 404 Not found. The given endpoint wasn't valid.
httpStatusCode	Specifies the http status code returned when a request has completed. This dimension further indicates whether request succeeded or not, and why. Use it to verify whether there was an issue with a request even though the request was logged as successful. The dimension displays one of the following values: 200 OK. The request succeeded. 404 Not found. The given endpoint wasn't valid. In the case of a failure, the reason for the underlying issue could be network connectivity, DNS failure, server certificate validation or timeout. The Business Central Server does not know which if these it might be.
serverExecutionTime	Specifies the amount of time it took the server to complete the request, including the time to open the company. The time has the format hh:mm:ss.sssssss.
totalTime	Specifies the amount of time it took to process the request, including the time to open the company. The time has the format hh:mm:ss.sssssss.
telemetrySchemaVersion	Specifies the version of the Business Central telemetry schema.

Sample KQL code

This KQL code can help you get started analyzing outgoing web service calls.

traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'RT0019'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
, alAuthenticationMethod = customDimensions.alAuthenticationMethod
, alHttpTimeout = customDimensions.alHttpTimeout
, alObjectId = customDimensions.alObjectId
, alObjectName = customDimensions.alObjectName
, alObjectType = customDimensions.alObjectType
, alStackTrace = customDimensions.alStackTrace // This dimension was introduced in Business Central 2022 release wave 1, version 20.1.
, clientType = customDimensions.clientType // This dimension was introduced in Business Central 2022 release wave 2, version 21.4.
, endpoint = customDimensions.endpoint
, extensionId = customDimensions.extensionId
, extensionName = customDimensions.extensionName
, extensionVersion = customDimensions.extensionVersion
, httpMethod = toupper( customDimensions.httpMethod ) // httpMethod is logged as the value used in the AL code
, httpStatusCode = case( isnotempty(customDimensions.httpStatusCode), customDimensions.httpStatusCode, customDimensions.httpReturnCode ) 
, httpHeaders = customDimensions.httpHeaders       
, executionTime = customDimensions.serverExecutionTime
, executionTimeInMS = toreal(totimespan(customDimensions.serverExecutionTime))/10000 //the datatype for executionTime is timespan
, usertelemetryId = case(
  // user telemetry id was introduced in the platform in version 20.0
  toint( substring(customDimensions.componentVersion,0,2)) >= 20, user_Id
, 'N/A'
)
// these three lines httpHeadersTmp, httpHeadersJSON, and Authorization illustrate how to extract data from the httpHeaders dimension
| extend httpHeadersTmp =  tostring( httpHeaders )
| extend httpHeadersJSON = parse_json(httpHeadersTmp)
| extend Authorization = tostring( httpHeadersJSON.['Authorization'] )

Performance considerations

If you call an external web service using the HttpClient module in AL, be aware that the Business Central Server blocks the execution of AL code for the session until the call completes. For interactive sessions, this behavior means that the user sees a spinning wheel during the call.

If you have enabled telemetry for your environment or app, you can use this KQL query to analyze how much time users are delayed in the UI by calls to external services.

traces
| where customDimensions has 'RT0019'
| where customDimensions.clientType == 'WebClient'
| extend executionTimeInMs = toreal(totimespan(customDimensions.serverExecutionTime))/10000 //the datatype for executionTime is timespan
| summarize count() // how many calls
, sum(executionTimeInMs) // sum of delays for UI sessions
, avg(executionTimeInMs) // average waiting time by this app
, max(executionTimeInMs) // average waiting time by this app
by 
// which app is calling out from the UI
  extensionPublisher = tostring( customDimensions.extensionPublisher )
, extensionName = tostring( customDimensions.extensionName )
, extensionVersion = tostring( customDimensions.extensionVersion )

Troubleshoot errors

When calling external web services using the HttpClient module in AL, make sure that you handle the HTTP status code returned by the service in case of errors. For more information, see HttpHttpResponseMessage.HttpStatusCode() Method.

If you've enabled telemetry for your environment or app, you can use this KQL query to analyze errors in calls to external services.

traces
| where customDimensions has 'RT0019'
| extend httpStatusCode = toint( case( isnotempty(customDimensions.httpStatusCode), customDimensions.httpStatusCode, customDimensions.httpReturnCode ) )
| where httpStatusCode > 299
| summarize count() // how many calls fail
by httpStatusCode 
, endpoint = tostring( customDimensions.endpoint ) // which endpoint

HTTP status codes

When you try to access content on a server by using the HTTP protocol, it returns a numeric code that indicates the result of the request and semantics of the response, including whether the request was successful.

The first digit of the status code defines the class of response. The last two digits don't have any categorization role. There are five values for the first digit:

1xx (Informational): Provisional response - the request was received, continuing process.

2xx (Successful): The server successfully received and accepted the request.

3xx (Redirection): Further action needs to be taken in order to complete the request.

4xx (Client Error): The request contains an error and can't be fulfilled.

5xx (Server Error): The server failed to fulfill the request.

For more information, see HTTP Semantics

Common HTTP status error codes

When you call a web service endpoint, either a Business Central API or from AL using Httpclient datatype, you get an HTTP status code as part of the response. All HTTP status codes that start with 4 (sometimes also written 4xx) are classified as client errors and it is your responsibility to react on these errors and fix them in your code.

In the following table, we list some common 4xx HTTP status codes and suggestions to how to fix them:

HTTP status code	Short name	Description	Suggested solution(s)
400	Bad Request	This status code indicates that the server can't or won't process the request due to an error on the client side. For example, it could be a malformed request syntax, header too long, or something else.	The client code that calls the endpoint needs to fix things on their end. For an incoming call of category OData/API, consider using telemetry to find the error. You can also set up a debugger and debug the endpoint code. For an outgoing call, you need to review/debug the AL code that sends the request.
401	Access denied	The request failed because it lacks valid authentication credentials for the target resource.	For an incoming call, this is an authorization issue on the Business Central end of this, either in the OnOpenCompany trigger or a permission issue. For an outgoing call, you need to examine the AL code that sets up certificates or sets authorization header(s).
402	Payment Required	Indicates that the caller must make a payment to access the requested resource. Typically used in situations where the server requires payment before granting access to the content or service. This status code isn't returned by the Business Central server for incoming calls.	For an outgoing call, you need to examine the AL code that calls the service and setup error handling to handle this situation.
403	Forbidden	This status code is returned when there's some kind of access restriction policy implemented for the requested resource.	For an outgoing call, you need to examine the AL code that sets up certificates (see Httpclient.AddCertificate) or sets authorization header(s). Or maybe your app shouldn't call the specified endpoint at all.
404	Not found	The resource you're calling doesn't exist (anymore?)	For an incoming call of category OData/API, this endpoint isn't available in metadata. If the endpoint used to work, the issue might be due to uninstalling or updating an app/extension. For UI pages exposed as OData, check if the page was disabled by an administrator. Note: We recommend that you move to APIs. For an outgoing call, you need to examine the URI specified in the AL code calling the endpoint and compare it with the endpoints available on the server you call.
405	Method not allowed	The HTTP verb used in the request isn't allowed for the specific URI that an HTTP client requested.	For an incoming call, you need to examine the HTTP method and compare that with what the endpoint supports. One example could be a client calling an API query (which is read-only) using HTTP POST. For on-premises environments where SOAP, OData, or APIs have been turned off, any call to such an endpoint will also return 405 (so check your server configurations if you see those). For an outgoing call, you need to examine the AL code that calls out and compare the Httpclient method used with what the external service offers.
408	Request timed out	This status code is returned when the server didn't receive a complete request message within the time that it was prepared to wait.	For an incoming call, this is returned by the Business Central server when the request fails to complete within the limits setup in the server configuration. For online environments, the limits are documented here: Operational Limits for Business Central Online. For on-premises environments, you might be able to adjust the server limits to allow more time for requests to complete. For on-premises environments, you need to change the AL code for the endpoint or call break up the request into multiple requests to make them fit within the time limits. For an outgoing call, you need to investigate the similar limits setup for the external service. Then either change your AL code or break up the request into multiple requests to make them fit within the time limits.
409	Conflict	Indicates that the request couldn't be processed because of conflict in the current state of the resource, such as an edit conflict between multiple simultaneous updates.	For an incoming call, this happens when another user has already changed the record(s) that the request is trying to modify. For an outgoing call, you need to look into the documentation for the service you're calling.
410	Gone	Indicates that the resource requested was previously in use but is no longer available and won't be available again.	Check you client code/configuration and stop calling this endpoint.
412	Precondition Failed	The server doesn't meet one of the preconditions that the requester put on the request header fields.	You need to examine the code that calls the endpoint and check if the settings of request or content HTTP headers match what the service expects. For an incoming call, check if you have supplied the required headers as specified in the API documentation, see API (v2.0) for Business Central for details. For an outgoing call, a common error is to forget to set the Content-Type header (which by default is set to 'text/plain; charset=utf-8'. For more information about content headers, see HttpContent Data Type.)
413	Payload Too Large / Request Entity Too Large	The request is larger than the server is willing or able to process.	You need to examine the code that calls the endpoint and adjust the size of the data you send. For OData/API calls, the limit is called `Max body size` and its current value for BC online is documented here: OData request limits. For SOAP calls, the limit is called `Max message size` and its current value for Business Central online is documented here: SOAP request limits
415	Unsupported Media Type	The request uses a media type that isn't supported by the server/resource.	Could very well be due to wrong Content-Type or Content-Encoding headers in the request.
418	I'm a teapot	This HTTP status is exposed as an Easter egg in some services.	If you experience this, there's probably nothing else to do than have a nice cup of tea. Then get back to work afterwards refreshed and ready to fix some more HTTP issues.
429	Too Many Requests	This status code is returned when you call too aggressively to the server and it asks you to relax a bit.	The Business Central performance tuning guide explains how you can implement different retry strategies to deal with this type of issue. For more information, see Performance Articles For Developers.
502	Bad Gateway	This status code is returned when services in the Business Central data plane are temporarily unavailable.	If you experience this, please retry your request. Note that the Business Central service adds a `retry-after` HTTP header when returning HTTP status 502. Use this in your web service client code. The Business Central performance tuning guide explains how you can implement different retry strategies to deal with this type of issue. For more information, see Performance Articles For Developers.
503	Service Temporarily Unavailable	This status code is returned when the server is down for maintenance or is overloaded.	If you experience this, please retry your request. Note that the Business Central service adds a `retry-after` HTTP header when returning HTTP status 503. Please use this in your web service client code. The Business Central performance tuning guide explains how you can implement different retry strategies to deal with this type of issue. For more information, see Performance Articles For Developers.
504	Gateway Timeout	This status code is returned when your call takes longer than the timeout threshold defined on the server.	The Business Central performance tuning guide explains how you can implement different retry strategies to deal with this type of issue. For more information, see Performance Articles For Developers.