How to use timestamp type parameters set in pipeline with timestamp type in data flow

Kakehi Shunya (筧 隼弥) 201 Reputation points
2022-12-23T03:20:00.183+00:00

Hello.
I have an error in an Azure pipeline where I want to set up a variable with 52 timestamps as an array and pass them in turn to the dataflow for use.
The pipeline design is shown in the image below.

273533-image.png

LookUp1:Get a list of timestamps from a file.
273541-image.png

Foreach1:Stores a list of timestamps in a variable of type array.
273503-image.png
273495-image.png

Foreach2:The variables of foreach1 are taken out in order and passed to the data flow. The data flow is then executed.
273490-image.png
273551-image.png
273496-image.png

And this is dataflow parameters...
273542-image.png
273534-image.png

start_date<= toTimestamp($as_of_date) && end_date>= toTimestamp($as_of_date)  

My understanding is that the variable is a String, so I first convert it to a String type when passing it to the dataflow. (*1)
Then I convert it to a Timestamp because I need to use it as a Timestamp type in the data flow. (*2)
However, an error occurs.
The error message is here.

{"message":"Job failed due to reason: Parameter stream has parsing errors. Parameter(s) with errors: as_of_date. Not honoring the datatype of parameter(s) could be one of the causes.. Details:","failureType":"UserError","target":"Data flow1","errorCode":"DF-Executor-ParameterParseError"}

Anyone can solve this?
Any answer would be appreciated.
If you need other more information, please tell me.
Thank you for reading.
I look forward to your answer.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,241 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,348 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,516 Reputation points Microsoft Employee
    2023-01-12T08:14:04.07+00:00

    Hi @Kakehi Shunya (筧 隼弥) ,

    Thank you for posting query in Microsoft Q&A Platform.

    Kindly try below and see if that helps.

    • In Append variable activity, actually we should write expression as @item().Horizon_Week_2022.
    • In filter transformation I see, start_date and end_date columns. Are these columns datatype is timestamp type only? If not, consider them also to convert.
    • Did you tried by passing hardcoded values in dataflows parameters? Is that worked? If yes, kindly make sure the value which we are getting is actually coming in similar format and see.

    Please let me know how it goes.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.