Elastic Job Agent control database generates constant SQL audit volume despite a single nightly job

Question

We have server-level SQL auditing enabled on Azure SQL logical servers with a Log Analytics destination, using the default audit action groups.

The Elastic Job Agent's control database lives on the same server in each of our 3 environments, and its internal polling is generating about 5.85 GB and 3 million audit records per day, all authenticated as ##MS_InstanceCertificate##.

Real application and user audit activity across every other database on the server totals roughly 15 MB per day, so the job agent is about 400+ times the volume of the data we expect.

The top statements are IF OBJECT_ID('sys.sp_cloud_connection_set_sds', 'X')... at 254,000 executions per 4 hours, plus two Entity Framework job_execution polling queries at 126,000 each.

What makes this look wrong rather than merely noisy: the agent has on 1 job scheduled, and it runs once a day at 2 AM.

Volume is flat at approximately 127,000 records per hour across a full 30 day window, with no dip overnight and no bump at 2 AM. The polling appears entirely decoupled from the actual 2AM job workload.

At pay-as-you-go Log Analytics ingestion rates this is roughly $500 per month per each environment, and we run 3 exactly the same, so about $1,500+ per month that has been accruing for several months with no operational or security value attached to it.

Is this expected behavior for the Elastic Job Agent?

Is the polling cadence configurable?

And is there a supported way to exclude the job control database from server-level auditing while keeping auditing on the application databases, without dropping BATCH_COMPLETED_GROUP from the action groups, since that would remove the legitimate activity we do want to capture?

We are aware that a Workspace Transformation DCR would likely filter out this at ingestion, but would prefer a fix or configure this a different way or even at the audit configuration layer if one exists.

Thank you,

Answer 1

Hi Daniel,

Yes, this is expected behavior, even though it definitely looks wrong at first glance. The Elastic Job Agent works by constantly polling its job database to check for new jobs, job definition changes, and other CRUD operations. It's a fixed internal heartbeat of the agent service, completely independent of your job schedules. That's why your volume is flat at ~127K records/hour with no bump at 2 AM and no overnight dip. The sp_cloud_connection_set_sds checks and the EF job_execution queries are the agent's own plumbing running under ##MS_InstanceCertificate##and Microsoft has actually added a note about exactly this audit noise to the Elastic Jobs documentation.

Is the polling cadence configurable? No. There's no setting on the job agent (at any JA tier) to slow it down; the tiers only change capacity/concurrency, not the polling behavior.

Is there a supported way to exclude the job database at the audit layer? Yes, and it's the fix Microsoft recommends for this: a predicate expression on the server audit policy. It's not exposed in the portal, but it's fully supported via PowerShell/CLI/REST:

powershell

Set-AzSqlServerAudit -ResourceGroupName "rg-name" `
    -ServerName "sql-server-name" `
    -PredicateExpression "database_name <> 'YourJobDatabase'"

This keeps your action groups intact (including BATCH_COMPLETED_GROUP), keeps auditing on all application databases, including ones created later, which is the main benefit of server-level auditing, and drops the job DB records before they're written, so you stop paying Log Analytics ingestion on them entirely. That's better than a transformation DCR, where the data still transits the pipeline.

A couple of caveats:

• The predicate applies to the entire server audit policy, so test it in one environment and confirm your application-database events still land as expected.
• If you want to keep visibility into the actual 2 AM job step activity while cutting only the heartbeat, you can use a more surgical predicate instead, e.g. application_name = 'Microsoft Azure SQL Database elastic jobs - control' AND statement NOT LIKE '%sys.sp_cloud_connection_set_sds%' etc. But the plain database-name exclusion is simpler to reason about and easier to get past compliance review.
• I'd avoid filtering on server_principal_name <> '##MS_InstanceCertificate##', since that internal principal can appear for platform operations beyond the job agent.

If a predicate doesn't fit your compliance posture, the other supported option is to move the job database to a dedicated logical server with no (or reduced) server-level auditing. The agent can target databases on other servers, so the control DB doesn't need to live alongside your application databases. Switching to per-database auditing on each app DB would also work, but you'd lose automatic coverage of new databases, so I'd treat that as a last resort.

Since your 3 environments are identical, the predicate is a one-line change per server and should eliminate all of that spend essentially. Just document the exclusion in your audit standard, so it's clear the omission is deliberate.

Hope this helps. Please mark as accepted if it resolves your question.