Hi, I have been learning working in ADF and was trying a scenario to copy a table "Customer" from sourcesqldb to destinationsqldb using a pipeline with conditions: load Customer table from SourceDB to DestinationDB only if the column Enabled in config table has a value of 1. If the Enabled column in config table has a value of 0 then the pipeline should not load Customer table from sourcedb to destinationdb I have created this pipeline:

Hi @Anonymous , Thank you for posting query in Microsoft Q&A Platform and Thanks for sharing details on clarifications. Filter activity will help you to filter your items from input array and it returns array back. In your it seems you are trying to filter value array which you got from lookup activity. And, it seems your condition is to take only items which has Enables property as 1. You need to correct expression inside filter activity --> Settings --> Condition as @equals(item().Enabled,'1') . This will filter your value array to return only items which has Enabled property as 1. Hope this will help. Thank you. Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how . Want a reminder to come back and check responses? Here is how to subscribe to a notification .

error {code":"BadRequest","message":null,"target":"pipeline//runid/7ea8dfa0-538a-4d37-9164-477d34174cbb","details":null,"error":null} in azure adf

Sona Jain 21

Hi,

I have been learning working in ADF and was trying a scenario to copy a table "Customer" from sourcesqldb to destinationsqldb using a pipeline with conditions:

load Customer table from SourceDB to DestinationDB only if the column Enabled in config table has a value of 1.
If the Enabled column in config table has a value of 0 then the pipeline should not load Customer table from sourcedb to destinationdb

I have created this pipeline:

Sona Jain 21 Reputation points

2021-07-18T17:52:56.8+00:00

lookup->filter->copydata
Ryan Abbey 1,181 Reputation points

2021-07-18T21:28:28.167+00:00

Go to the Monitor screen and look at the error message output there...

Unrelated to the error message, but your description of what you're trying to do sounds like you should be selecting only rows where Enabled = 1 and then have a foreach to do the copy. I don't think the filter is going to stop the copy from executing
ShaikMaheer-MSFT 38,441 Reputation points Microsoft Employee

2021-07-19T06:50:38.287+00:00
Hi @Anonymous ,

Welcome to Microsoft Q&A Platform. Thank you for posting your query here.

Could you please help on below clarifications. So that We can follow along your details to reproduce scenario and help with resolution.

In Which activity your getting error?

Could you please share output json of every activity execution, so that we can understand what exactly coming from lookup and what you are filtering in filter activity?

Could you please share expressions what written in each activity along with output json screenshots of activity executions?

By seeing your error message, It seems you written some wrong expression inside activities some where. We need to correct it. Once, we get above asked details. Then we can help you to resolve issue. Thank you.

Please check below link which helps to troubleshoot this kind of issue
https://github.com/MicrosoftDocs/azure-docs-pr/blob/702e4767193391009cd20803fcfdc638f5196493/articles/data-factory/pipeline-trigger-troubleshoot-guide.md#error-message---codebadrequest-messagenull
Sona Jain 21 Reputation points

2021-07-21T16:43:37.423+00:00
Hi @ShaikMaheer-MSFT ,

You guessed it correct. Somewhere I am making mistake in writing dynamic content as this is my first time working on it.
To answer your questions:

I am getting an error in Filter activity

LookupSourceDB output json file:
-> {
"count": 2,
"value": [
{
"Id": 1,
"Name": "Customer",
"Type": "Table",
"Enabled": 1
},
{
"Id": 2,
"Name": "Department",
"Type": "Table",
"Enabled": 0
}
],
"effectiveIntegrationRuntime": "DefaultIntegrationRuntime (East US)",
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "DIUHours"
}
]
},
"durationInQueue": {
"integrationRuntimeQueue": 0
}
}

Filter output json file:
-> {
"ItemsCount": 2,
"FilteredItemsCount": 0,
"Value": [
"[]"
]
}

In filter activity:
Settings->Items-> @activity('LookupRefDB').output.value
Settings->condition->@equals(concat('select ', activity('LookupRefDB').output.Column('Enabled').value, ' from ', activity('LookupRefDB').output, ' where ', activity('LookupRefDB').output.Column('Name').value, ' like ''%Customer%'''),1)

Kindly let me know if any other details are required.

Thanks,
Ryan Abbey 1,181 Reputation points

2021-07-21T20:19:11.907+00:00

you don't use "activity('LookupRefDB')..." but "item()"

you aren't referring to them correctly either... as a suggestion, make it a simple condition and run the job, you will then be able to view the output of the Lookup so that you can see what format you need to write it in

Accepted answer

ShaikMaheer-MSFT 38,441 Reputation points Microsoft Employee

2021-07-22T10:21:48.05+00:00
Hi @Anonymous ,

Thank you for posting query in Microsoft Q&A Platform and Thanks for sharing details on clarifications.

Filter activity will help you to filter your items from input array and it returns array back.

In your it seems you are trying to filter value array which you got from lookup activity. And, it seems your condition is to take only items which has Enables property as 1.

You need to correct expression inside filter activity --> Settings --> Condition as @equals(item().Enabled,'1'). This will filter your value array to return only items which has Enabled property as 1.

Hope this will help. Thank you.

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

error {code":"BadRequest","message":null,"target":"pipeline//runid/7ea8dfa0-538a-4d37-9164-477d34174cbb","details":null,"error":null} in azure adf

0 additional answers

Your answer