What are the rules for Azure Application Insights data throttling?

Question

What are the rules for Azure Application Insights data throttling?

Emin Sehic 0

We use Azure Application Insights to collect data from our frontend Angular Application. We noticed some logs are missing (we know that by providing payment ID - some value of internal use), but we do not know why they are missing.

We haven't implemented any of these:

Adaptive sampling (not even supported for JavaScript SDK)
Fixed-rate sampling
Ingestion sampling (ingestion sampling on Azure Portal for the Application Insights resource is set to collect all data - 100%)

We used the query recommended in the official documentation to check if any logs have been sampled, and RetainedPercentage is 100, which means no sampling occurred.

What caught our attention however are these statements from the official documentation:

"Sampling also helps you avoid Application Insights throttling your telemetry."
"Sampling reduces the likelihood that your application sees throttling occur."
"...without causing data loss from throttling."
"...You may still experience telemetry data loss related to throttling..."

As we do not have any sampling configured by us, is it possible that we lost our data due to the throttling? I could not find any more info on throttling, like when it occurs and in which amount.

Can someone help here, please? What if we want to keep all of our logs for 90 days, with no sampling and no throttling, is it even possible?

Thank you!

AnuragSingh-MSFT 21,551 Reputation points Moderator

2023-09-05T08:54:13.6+00:00

@Emin Sehic , Following-up to check if you had a chance to review the answer provided.

If the answer did not help, please add more context/follow-up question for it, and we will help you out. Else, if the answer helped, please click Accept answer so that it can help others in the community looking for help on similar topics.
Emin Sehic 0 Reputation points

2023-09-05T09:21:03.0866667+00:00

I am looking at the limitations explained on the link provided in the answer.

What is interesting in our case is that for some users, the whole session is missing on the frontend (but we have API calls logged on the backend, so we know the session happened for sure), not some logs in the session but all of them. That being said, we are going to focus on the investigation of the possibility of the front end blocking the logging from itself. This is not the case with 99% of the users but may be the case for some.

2 answers

Your answer

AnuragSingh-MSFT 21,551 Reputation points Moderator

2023-09-05T08:54:13.6+00:00

@Emin Sehic , Following-up to check if you had a chance to review the answer provided.

If the answer did not help, please add more context/follow-up question for it, and we will help you out. Else, if the answer helped, please click Accept answer so that it can help others in the community looking for help on similar topics.
Emin Sehic 0 Reputation points

2023-09-05T09:21:03.0866667+00:00

I am looking at the limitations explained on the link provided in the answer.

What is interesting in our case is that for some users, the whole session is missing on the frontend (but we have API calls logged on the backend, so we know the session happened for sure), not some logs in the session but all of them. That being said, we are going to focus on the investigation of the possibility of the front end blocking the logging from itself. This is not the case with 99% of the users but may be the case for some.

Answer 1

Hi,

Application insights limits are outlined here: https://learn.microsoft.com/en-us/azure/azure-monitor/service-limits#application-insights. To understand if your event was lost due to throttling you would have to evaluate the period you where expecting the event against these limits. If you have hit these limits contacting Microsoft support would be the best option as they will up these limits if possible.

However, I note that these logs are coming from a frontend client application. This is important as these apps run in a much more variable environment and therefore it is possible that these logs were dropped due to a transient issue on the client side or a network issue. It maybe worth exploring this as a potential as well.

It being a client side app I'd review the importance of this data and if there are other ways of logging it to app insights through a medium you control such as a backend API. This will give a greater guarantee that the data will be sent and ingested.

Hope this is helpful.

Emin Sehic 0 Reputation points

2023-09-05T09:22:57.53+00:00

Thank you so much for the answer!

I am looking at the limitations explained on the link provided in the answer.

What is interesting in our case is that for some users, the whole session is missing on the frontend (but we have API calls logged on the backend, so we know the session happened for sure, and no network errors occurred), not some logs in the session but all of them.

That being said, we are going to focus on the investigation of the possibility of the front end blocking the logging from itself. This is not the case with 99% of the users but may be the case for some.

Answer 2

Emin Sehic, thank you for the reply. As you mentioned, the blocking of requests from front/end (client browser) to AppInsights API endpoint could be one of the possibilities. For details on URLs/Ports requirement, see Application Insights Outgoing Ports

Another possibility could be the limit reached for the daily cap - Application Insights daily cap configuration and queries to check

If you do identify certain client machines/IPs from which the telemetry does not reach the AppInsights workspace, I would suggest opening a support case with Azure Support so that it can be investigated along with the logs from backend, if required.

Share via

What are the rules for Azure Application Insights data throttling?

2 answers

Your answer