question

karthikshanth avatar image
0 Votes"
karthikshanth asked ShaikMaheer-MSFT commented

mongodb atlas filters in ADF source is not working. returning no records!

Hello, I have connected the MongoDB atlas as a source in the copy data tool and pulled the entire collections which worked flawlessly. Now, I m trying to apply the filters to pull the last few months' records, and I don't get any records.

This is what I have added inside the Filters box for source to pull data from 2022-01-12
{_id: ObjectId("61de60500000000000000000")}

is there any other way, we can pass the filters?

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I tried the same earlier too, but no luck. gets 0 records!

0 Votes 0 ·

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered ShaikMaheer-MSFT commented

Hi @karthikshanth ,

Thank you for posting query in Microsoft Q&A Platform.

I have gone through documentation of MongoDB connector in ADF. It looks The service supports consuming BSON document in Strict mode. Make sure your filter query is in Strict mode instead of Shell mode. More description can be found at MongoDB manual.

Below is sample filter expression.
"filter": "{datetimeData: {$gte: ISODate(\"2018-12-11T00:00:00.000Z\"),$lt: ISODate(\"2018-12-12T00:00:00.000Z\")}, _id: ObjectId(\"5acd7c3d0000000000000000\") }"

Kindly check below documentation for more details and see if that helps. Please let us know how it goes.
https://docs.microsoft.com/en-us/azure/data-factory/connector-mongodb?tabs=data-factory


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

how do I send a specific type in the filter?

For ex, something like Departname IN (HR, SALES, MARKETING) or DEPARTNAME = SALES ?

0 Votes 0 ·

Did you get chance try using $filter? Kindly try and see if that helps. Below is link for more details on filter in MongoDB.
https://www.mongodb.com/docs/manual/reference/operator/aggregation/filter/

0 Votes 0 ·