Share via

Error occurs at source transformation when trying to source csv file from folder in data lake storage gen2 while debugging the pipeline

Rushabh Shah 1 Reputation point
2020-08-31T03:05:19.917+00:00

While using mapping data flow activity, In source transformation I am trying to read a csv file from a folder in Azure data lake gen2. I am able to preview the data in "Data Preview" in source transformation but when trying to debug the pipeline I get the error : "message":"at Source 'source1'(Line 4/Col 0): store is not defined. Details:at Source 'source1'(Line 4/Col 0): store is not defined","failureType":"UserError","target":"dataflow1","errorCode":"DFExecutorUserError"}.

I am not sure what the above error actually means. Any help in explaining the error in detail will be appreciated.

Thank You.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,389 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,845 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
    2020-09-03T23:16:45.56+00:00

    Hello @RusabhShah,

    Thanks for the reply . I see that in the script you have the date as string parameter , but then the on the first image , you dont have that . I am not clear on this .
    Anyways I dont think i have all the info in hand at this time to let you as to whats wrong . Can I ask you if the date which a parameter , are you using any conversion on that and if yes may be the incoming date format is not correct , request you to please check .

    While trying to repro i did created a pipeline and i am sharing that with you below ,not that it solves the issue inhand , but may be it helps you .

    My understanding is that you are trying to read some CSV filke from the GEN2 storage and you have structure like env\path\filrname.csv . I have tried to implement that below . I am using the paramterized dataset ( DS) .

    22510-transformationissue.gif

    Please drop in a email at azcommunity@microsoft.com ( with the subject "Attention : Himanshu" ) with more info on your pipeline and I can work offline .

    Please do keep us posted how it goes .

    Thanks
    Himanshu