Best practice for calling Azure OpenAI from an ADF pipeline for row-level enrichment without hitting 429 throttling?

Question

I'm building a pipeline in Azure Data Factory that lands semi-structured support-ticket data into Snowflake, and I want to enrich each record with a short summary and a category label generated by Azure OpenAI (gpt-4o-mini deployment) before the load step.

Current setup:

• ADF pipeline with a Copy activity landing raw JSON in ADLS Gen2
• A ForEach over the resulting rows calling a Web activity against the Azure OpenAI /chat/completions endpoint
• Copy activity into Snowflake via the Snowflake V2 connector

This works for small batches, but at +50k rows/day I'm running into two problems:

1. The ForEach + Web activity approach generates a huge number of activity runs, and I'm getting frequent HTTP 429 responses from the Azure OpenAI endpoint even with batch count reduced to 5. Retry policy helps but pipeline duration becomes unpredictable.
2. Per-activity-run cost in ADF is becoming significant relative to the token cost itself.

Questions:

• Is ForEach + Web activity the wrong pattern here entirely? Should this be offloaded to an Azure Function or Databricks notebook called as a single activity, with batching and retry handled in code?
• For PTU vs standard deployments, is there guidance on which is appropriate for bursty batch enrichment workloads like this rather than interactive ones?
• Does anyone have experience doing the enrichment after the load, using Snowflake Cortex functions instead, and how does that compare on cost and governance when the source system is Azure-side?

Any references to a documented reference architecture for this would be appreciated, most of what I've found covers RAG/interactive scenarios rather than batch enrichment.

Answer 1

Hello Sahib Gasimov,

Greetings! Thanks for raising this question in Q&A forum.

The main issue is that Azure Data Factory (ADF) ForEach + Web Activity is not typically the best pattern for high-volume LLM enrichment workloads. At 50K+ records per day, it can create excessive activity runs, increase costs, and make Azure OpenAI throttling (HTTP 429) more likely.

1. Consider moving the Azure OpenAI calls out of the ADF ForEach loop and into a single compute layer such as an Azure Function, Azure Databricks notebook, or Azure Container App.
2. Batch multiple records into a single request where your use case allows it. This significantly reduces the number of API calls and activity executions.
3. Implement throttling control, exponential backoff, and retry logic in code rather than relying solely on ADF retry policies.
4. Use ADF primarily for orchestration (landing data, triggering enrichment, loading results into Snowflake) rather than record-by-record AI processing.
5. For bursty batch workloads, evaluate Provisioned Throughput Units (PTU) if you require predictable throughput and consistently hit rate limits. Standard deployments are often sufficient when batch processing can be scheduled and throttled appropriately.
6. If your data ultimately resides in Snowflake, you can also evaluate performing enrichment after the load using Snowflake-native AI capabilities and compare governance, latency, and cost against Azure OpenAI.

A common reference architecture is: ADF → ADLS/Snowflake staging → Azure Function/Databricks batch enrichment → Write enriched results → Snowflake. This avoids thousands of ADF activity executions and provides much better control over concurrency and rate limiting.

The next action is with you to evaluate whether the enrichment logic can be moved into a dedicated compute service (Function or Databricks). That approach is generally recommended for large-scale batch enrichment scenarios and should help address both the 429 throttling and ADF cost concerns.

If this answer helps you kindly accept the answer which will help others who have similar questions

Best Regards,

Jerald Felix.