Azure search indexer index entire Cosmos db data source

Evgeny 1 Reputation point
2020-06-30T15:44:53.64+00:00

Adding custom query to cosmos db data source causes my indexer to go through entire source, not only the newest elements.
Also, it's worth to mention that adding query like SELECT * FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts doesn't break anything. But as soon as I add fields into select statement (e.g. SELECT c.id FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) my indexer ignores previous progress.

I noted that if I add @HighWaterMark as one of my search fields to query (SELECT c.id, @HighWaterMark as {some field} FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) this field becomes -1. I'm not sure if it is a correct way to get @HighWaterMark value.

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
862 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. brtrach-MSFT 15,791 Reputation points Microsoft Employee
    2020-07-24T20:33:31.707+00:00

    @Evgeny-1093, If you are still having trouble, can you please reach out to us at azcommunity@microsoft.com with your Azure subscription ID? We'd like to provide you with a free support case to get you the support needed.

    If you found a solution, we would appreciate it if you shared the solution with the community to help others with similar issues.

    0 comments No comments