Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,669 questions
adf_client.activity_runs.query_by_pipeline_run while debugging pipeline
hitler 007
1
Reputation point
I am using adf_client.activity_runs.query_by_pipeline_run inside my azure notebook to get pipeline details (such as output/pipeline status/child pipeline execution time). I am passing the pipeline run id as the base parameter to this notebook. However, I am able to get the details when i publish and trigger it but not when i debug the pipeline.
{count} votes
-
MartinJaffer-MSFT 25,476 Reputation points Microsoft Employee
2021-06-28T20:07:05.803+00:00 Hello @hitler 007 and welcome to Microsoft Q&A.
The reason you are able to query published & triggered pipelines, but not debug pipelines, is because they are considered to be different types of runs. Debug pipeline run data is not kept as long (I think?).
I will inquire as to how to retrieve activity data of debug runs.
Hi @MartinJaffer-MSFT ,
Thank you for the response.
So, any update as to how to retrieve activity data of debug runs?
I'm sorry, @hitler 007 ,
I haven't gotten responses on any of my recent escalations. Just in case they tell me debug details aren't available via SDK, I have done some investigation into alternatives.
By inspecting browser traffic when using the ADF UI, I found some endpoints that may be of use. While the endpoints aren't a class, they are a starting point for a work-around. I expect they work similar to the other ADF REST API endpoints.
base url:
https://management.azure.com/subscriptions/{subscription_id}/resourcegroups/{resource_group_name}/providers/Microsoft.DataFactory/
... factories/{factory_name}/querydebugpipelineruns?api-version=2018-06-01
(with no other parameters)
....factories/{factory_name}/sandboxPipelineRuns/{run_id}/sandboxActivityRuns?
with parameters
Any updates on this matter?
Sign in to comment