Help with a ADF error

Divya Rajendran 21 Reputation points
2021-09-03T15:41:13.937+00:00

Hi Team,

I am running a pipeline that has a metadata activity which in turn calls on a dataflow that uses a date parameter as shown below and this pipeline always failed with the below error.

Parameters:

-------------

  1. startDtTm --> @addhours(pipeline().TriggerTime, -24)
  2. endDtTm --> @pipeline().TriggerTime

129212-parameters-dt.png

Error
-------
{"StatusCode":"DF-Executor-StoreIsNotDefined","Message":"Job failed due to reason: The store configuration is not defined. This error is potentially caused by invalid parameter assignment in the pipeline.","Details":""}

On research, I found a similar ticket on GitHub, link mentioned below. In this ticket, the issue was resolved by checking the Expression checkbox
However, when I tried to do the same the checkbox is disabled for me.

https://github.com/MicrosoftDocs/azure-docs/issues/79325

Could you please guide me on how to overcome this issue?

Thanks!

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,950 questions
{count} votes

Accepted answer
  1. svijay-MSFT 5,226 Reputation points Microsoft Employee
    2021-09-06T22:05:05.123+00:00

    Hello @Divya Rajendran ,

    Thanks for the question and using MS Q&A platform.

    You could use the below expression for startDtTm & endDtTm respectively :

    toTimestamp(left('@{addhours(pipeline().TriggerTime,-2)}',23), 'yyyy-MM-dd\'T\'HH:mm:ss.SSS')  
    toTimestamp(left('@{pipeline().TriggerTime}',23), 'yyyy-MM-dd\'T\'HH:mm:ss.SSS')  
    

    Code, @{addhours(pipeline().TriggerTime,-2)} that needs to be run is passed as string. Left is used to handle the limitation of the DataFlow Expressions. The totimestamp is used to convert the resultant string to type timestamp - that matches the parameter datatype.

    I tested at my end and was able to get the above expression work successfully. The error is because there is a crossover between the dataflow expression and pipeline expression when using pipeline().triggerTime or utcnow(). The reference can be found in this document.

    129753-image.png

    Hope this helps. Do let us know if you any further queries.

    ---------------------------------------------------------------------------

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

    0 comments No comments

0 additional answers

Sort by: Most helpful