cosmosSearch vector search aggregation pipeline filtering on an id

Reza S 10

Jun 30, 2023, 11:06 AM

Trying out the new vector search feature in Mongodb vCore. Not able to find a lot of documentation in usage but basically trying to search on vector content but filter on or have exact match on a specific id in addition to the vector search. The below code returns empty array as result.

			{$search: {
                "cosmosSearch": {
                    "vector": queryVector,
                    "path": "vectorContent",
                    "k": 2
                },
                "returnStoredSource": true
            }},
            {$match: { customer_id: "597846e76b573e0011897dcdd1" }}

GeethaThatipatri-MSFT 29,517 Reputation points Microsoft Employee

Jun 30, 2023, 6:37 PM
Hi,@Reza S Welcome to Microsoft Q&A forum and thanks for reaching out here

If I understand correctly you are trying to use the new vector search feature in MongoDB vCore and having trouble filtering on or having an exact match on a specific ID

can you try as to filter on a specific ID in addition to the vector search using the Cosmos DB API for MongoDB.

{$search: { "cosmosSearch": { "vector": queryVector, "path": "vectorContent", "k": 2 }, "filter": { "customer_id": "597846e76b573e0011897dcdd1" }, "return": { "score": true, "payload": true }

Regards

Geetha

Reza S 10

Jul 1, 2023, 11:20 AM

Thanks Geetha,

I get error:

Unrecognized pipeline stage name: $search

Thanks,

This is how I call $search:

let result = await KnowledgeContent.aggregate([

        {
            "$search": {
                "cosmosSearch": {
                    "vector": queryVector,
                    "path": "vectorContent",
                    "k": 2
                },
                "filter": {
                    "customer_id": "5b100aa48544e1001a9c5111"
                },
                "return": {
                    "score": true,
                    "payload": true
                }
            }
        }
    ]);

Thanks,

Regards,

Reza

Yuvaraj Madheswaran 390 Reputation points

Jul 6, 2023, 5:58 PM

Hi Reza,

This might be related to the Aggregation Pipeline stage is only available in MongoDB Atlas as an Atlas Search 37 index is required.

Please refer the comment from alexbevi in the below link,

https://www.mongodb.com/community/forums/t/unrecognized-pipeline-stage-name-search/111883/3

Can you please try as per the suggestion in the above link?
Yuvaraj Madheswaran 390 Reputation points

Jul 16, 2023, 10:09 PM

Hi @Reza S,

Does the answer helped you to fix the issue?

Thanks!
GeethaThatipatri-MSFT 29,517 Reputation points Microsoft Employee

Jul 18, 2023, 2:13 PM

@Reza S Did you get a chance to check the above community response? is your issue resolved

Regards

Geetha
Reza S 10 Reputation points

Jul 31, 2023, 10:25 AM

Hi,

Not sure I understand this. I´m not running this locally as I´m testing against running on Mongodb vCore on Azure Cosmos db.
Himasagar Mallepalli 5 Reputation points

Sep 21, 2023, 3:06 PM
pipeline = [ { "$search" : { "cosmosSearch": { "vector": query_vector, "path": "vectorContent", "k": 2, }, "filter": { "price": {"$lt": 2000} }, "returnStoredSource": True, } } ] result = list(collection.aggregate(pipeline))

I'm also looking for the same solution , first to filter and then loop on the filtered results..

The above query resulting in below exception

OperationFailure: Unrecognized $search option: filter, full error: {'ok': 0.0, 'errmsg': 'Unrecognized $search option: filter', 'code': 40324, 'codeName': 'UnrecognizedCommand'}

Hammad Ali 0

Sep 12, 2024, 8:39 AM

Hello we are facing the similar problem,
@Geetha when trying to use the filter we receive an error

OperationFailure: $filter is not supported for vector search yet., full error: {'ok': 0.0, 'errmsg': '$filter is not supported for vector search yet.', 'code': 115, 'codeName': 'CommandNotSupported'} .

The pipeline looks as following

pipeline = [
    {
        "$search": {
            "cosmosSearch": {
                "vector": query_vector,
                "path": "vectorContent",
                "k": 1000,  # Top 10 results
                "filter": [
                    {
                        "path": "metadata.license_tag",
                        "values": ["PL1097"]
                    }
                ]
            }
        },
    },
    {
        "$project": {
    
            "document": "$$ROOT",
           # Exclude the vectorContent field
        }
    }
]


docs = collection_db.aggregate(pipeline)

Your comment would be highly appreciated

1 answer

Hunt, Gabe 0 Reputation points

Sep 27, 2023, 3:07 PM
Looking at: https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/compatibility we see that $filter is not supported, but $match is.

Here is a tested and validated answer that actually works.

pipeline = [ { "$search": { "cosmosSearch": { "vector": myVector, "path": "myVectorField", "k": limit }, "returnStoredSource": True } }, { "$match": { "myOtherField": { "$eq": "myValue" } } } ] ```
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

cosmosSearch vector search aggregation pipeline filtering on an id

1 answer

Your answer