Contains does not work in Dataflow expression builder

Sven Peeters 0 Reputation points
2024-10-31T07:23:22.47+00:00

I have an array field containing strings, this was created via an Aggregate stream on a Azure SQL Source.

The preview properly shows me the content of this array field

User's image

I try to check if a certain string is present in the array field in my Assert stream, if so then a particular field is mandatory. I use the expression below.

When I try to preview the results, I get following error :

Spark job failed: { "text/plain": "{"runId":"9a96d49a-2f20-4508-bb68-7233675bd5e1","sessionId":"b4375467-269e-4906-b3d0-66caf101cc80","status":"Failed","payload":{"statusCode":400,"shortMessage":"DF-EXPR-021 at Assert 'ValidateRows'(Line 519/Col 67): Return types do not match\nDF-EXPR-021 at Assert 'ValidateRows'(Line 520/Col 69): Return types do not match","detailedMessage":"Failure 2024-10-31 07:14:51.284 failed DebugManager.processJob, run=9a96d49a-2f20-4508-bb68-7233675bd5e1, errorMessage=DF-EXPR-021 at Assert 'ValidateRows'(Line 519/Col 67): Return types do not match\nDF-EXPR-021 at Assert 'ValidateRows'(Line 520/Col 69): Return types do not match"}}\n" } - RunId: 9a96d49a-2f20-4508-bb68-7233675bd5e1

(isNull(LICENSE_PLATE) || LICENSE_PLATE == "") && !contains(vab_fieldnames, 'platenumber')
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,865 questions
{count} votes

1 answer

Sort by: Most helpful
  1. ShaktiSingh-MSFT 15,501 Reputation points
    2024-11-08T06:03:00.15+00:00

    Hi Sven Peeters •,

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    Error Message:

    • Error Message...

    Issue:

    • Issue Description...

    Solution:

    After correcting the expression it worked:

    • (isNull(LICENSE_PLATE) || LICENSE_PLATE == "") && !contains(vab_fieldnames, lower(#item) == 'platenumber')

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    I hope this helps!

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    I hope this helps!

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.