Cost Analysis for dynamic adf pipelines

Question

Hello Azure Data Factory Developers,

Following starting situation:
We have a parameterized adf pipeline to be able to load and transform data from different source systems with only one generalized pipeline.

Goal:
We want to have a cost analysis of the pipeline per source system load.
For example:
Loading from Source S1 with Pipeline P costs us 100 dollar per month
Loading from Source S2 with Pipeline P costs us 70 dollar per month
Loading from Source S3 with Pipeline P costs us 20 dollar per month
and so on.

Question:
I know there is the possibility to tag the pipeline run with annotations.
But is there any possibility to use tags or annotations of pipeline runs in the cost analysis of the whole subscription?

Thank you much!

Answer 1

Hi @Volkmann Stephan ,

Welcome to Microsoft Q&A forum and thanks for your query.

Unfortunately it is not possible. Currently annotations field in ADF pipeline doesn't support dynamic content, hence you won't be able to differentiate the runs of a particular pipeline for various source systems using annotations.

I would recommend you to please log a feature request in ADF user voice forum: https://feedback.azure.com/forums/270578-azure-data-factory
All the feedback share in this form are actively monitored and reviewed by ADF engineering team. And also do share the link here so that other users with similar idea can upvote and/or comment on your suggestion which would help to increase the priority of feature request.

Here is an existing feedback to support dynamic content for annotations : Allow dynamic content in annotations field Please feel free to upvote/comment on it.

Hope this info helps.

----------

Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.