filter on jobid

Vineet S 185 Reputation points
2024-05-15T01:24:17.71+00:00

Hi,

if we have 2 table table1 =jobid

and table2- jobid,jobname

i want to filter the data in table 1 where jobid not equal to jobname='test1'

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,689 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Gowtham CP 1,005 Reputation points
    2024-05-15T06:22:46.58+00:00

    Hello @Vineet S

    Thanks for reaching out in the Microsoft Q&A!

    To filter table1 data in Azure Data Factory (ADF) based on jobid not equal to jobname='test1' in table2, use the Filter transformation. Set up two Source datasets for table1 and table2, with a Sink dataset for filtered data. Add a Filter transformation after table1, defining the condition "NOT table1.jobid == lookup(table2, 'jobid', 'jobname', 'test1')". This filters out rows from table1 where jobid doesn't match jobid associated with jobname='test1' in table2. Optionally, consider the Merge transformation if you need data from both tables, or Filter if only data from table1 is required.

    Don't forget to upvote! If you found this solution helpful, consider accepting it.