Programetically getting data flow source "source options -> Folder path" for inline source type

Sateesh Battu 41 Reputation points
2022-05-12T10:14:56.273+00:00

Hi ,

In a data flow i have defined online source type . In the source options, defined the ADLS delta file location

Now i would like to programmatically ( any possible programming language using azure - cli/JAVA/Python/powershell/APIs ) get the value of the source option folder path dynamically from run time metadata.
If it is a dataset type, i can use "az datafactory dataset show " command in cli and get the schema table information.
How to get the same in case it is inline ADLS delta file.

201421-image.png

201260-image.png

The path highlighted in the second image want to get from activities metadata. or any other command in azure. Please help

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
{count} votes

Accepted answer
  1. MartinJaffer-MSFT 26,021 Reputation points
    2022-05-13T19:33:39.483+00:00

    @Sateesh Battu
    I notice that your path is concatenating parameters. If these parameters are passed into the Data Flow from the pipeline, then one solution is to simply reconstruct the path from the same parameters. I think the evaluations inside a dataflow are not exposed, so as to better keep information privacy.

    The Data Flow script itself will tell you the default value of a parameter, and the expressions used. properties.scriptLines

    The activity runs will tell you what the input parameter (to the Data Flow) values. value[n].input.dataflow.parameters[parameterName]

    If your parameter values are being obtained from another source, things can get more complicated.


0 additional answers

Sort by: Most helpful