Dataflow filtering on 2 combinations

Question

Hi all,

Having a strange result from filtering a dataflow with a combination of notEquals() with && or and()

Example source has:

id	subkey
00	AB
00	BC
00	CD

in SQL I have the following clause:

WHERE id != '00' AND subkey != 'AB'

The result is returning 2 rows. Excellent.

In dataflow, I've added a filter and tried the following code:

id != "00" && subkey != "AB"

and( id != "00" , subkey != "AB" )

notEquals(id, "00") && notEquals(subkey, "AB")

and( notEquals(id, "00"), notEquals(subkey, "AB") )

The result filters all 00, so I have no rows!

What am I doing wrong? The logic looks correct but if someone can help that would be great.

Answer 1

@Alex Darling - I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask: Dataflow filtering on 2 combinations

Solution: The issue is resolved. Managed to get this working by trying lots of different options but I still think that there is some type of bug here.

But the solution is:

not( id == '00' && subkey =='AB' )

It's possible that there might be some issues with the notEquals() function in the data flow expression language.

As an alternative, you can try using the equals() function with the negation operator (!) to achieve the same result. Here's an example of how your filter expression would look like using the equals() function with the negation operator:

not(id == '00' && subkey == 'AB')

This expression should return all rows where id is not equal to '00' or subkey is not equal to 'AB'.

For more details, refer to Expressions and functions in Azure Data Factory and Azure Synapse Analytics

I hope this helps! Let me know if you have any further questions.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Managed to get this working by trying lots of different options but I still think that there is some type of bug here.

But the solution is:

not( id == '00' && subkey =='AB' )