Using FilterBy LastModified Data Factory Data Flow

Question

I am setting a pipeline variable of in string type and passing it as a parameter to data flow.I want to use this to filter files using LastModified Option.My source of files is azure data lake.

My pipeline variable is produced in this format.

"name": "lastModified",  
"value": "2021-06-07T12:31:20Z"  

}

It is passed to data flow and used in LastModified option using following expression.
toTimestamp(left($lastModified, 23), 'yyyy-MM-dd\'T\'HH:mm:ss.SSS')

and

toTimestamp($lastModified, 'yyyy-MM-ddTHH:mm:ss')

I have tried another option without left string function as well. However I am getting following error.

"StatusCode":"DF-Executor-ParseError","Message":"Job failed due to reason: Expression cannot be parsed","Details":"Parameter stream has parsing errors\nLine 3 Position 26: missing '=' at ':'\nLine 3 Position 27: mismatched input '31' expecting {'parameters', 'functions', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER}\nLine 5 Position 0: mismatched input '[<' expecting '='"}

What is the right way to pass LastModified Parameter to data flow?

Answer 1

Hello @Jogalekar, Mandar ,

Welcome to Microsoft Q&A Platform.

There is syntax error in the expression. Please use below expression to resolve the issue.

toTimestamp($lastModified,'yyyy-MM-dd\'T\'HH:mm:ss')  

Hope this helps. Do let us know if you any further queries.

---------------------------------------------------------------------------

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.