question

ALMK-2343 avatar image
0 Votes"
ALMK-2343 asked MartinJaffer-MSFT commented

Azure pipeline works in debuyg mode, but not on a trigger

I have an Azure Data Factory pipeline that utilizes an API call to our HR system, to copy records into a SQL database.

When running the pipeline in debug, the API correctly calls and returns 100 records at a time (the max allowed by the vendor) within an Until loop.

However, when the pipeline is executed via a trigger, the API only retrieves 1-2 records at a time.

There is a variable that's passed to the API call as a parameter; that variable is incremented by 100 in every loop, and the until loop continues until the count of records in the table matches the source count.

Any thoughts on why a pipeline would work in debug mode, but not when kicked off by a trigger?

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @ALMK-2343,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .

Thanks
Martin

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered

Hello @ALMK-2343,
Thanks for the question and using MS Q&A platform.

As we understand the ask here is why the effects of a pipeline might be different between debug run and trigger run .
The most likely cause is a difference between the published version and the version on your screen when you do debug run. Trigger runs are always started using the last published version. This can be different from what is on your screen, especially when you have a devops/git repository involved. Publishing is different from saving or committing changes to the repository.

Another, similar possibility, is any parameter values saved in the published trigger, being different from what you enter for a debug run. This assumes the pipeline / trigger use parameters. This is more complicated with event triggers compared to scheduled.

Please do let me if you have any queries.

Thanks
Martin


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.