Azure search indexer index entire Cosmos db data source

Question

Adding custom query to cosmos db data source causes my indexer to go through entire source, not only the newest elements.
Also, it's worth to mention that adding query like SELECT * FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts doesn't break anything. But as soon as I add fields into select statement (e.g. SELECT c.id FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) my indexer ignores previous progress.

I noted that if I add @HighWaterMark as one of my search fields to query (SELECT c.id, @HighWaterMark as {some field} FROM c WHERE c._ts >= @HighWaterMark ORDER BY c._ts) this field becomes -1. I'm not sure if it is a correct way to get @HighWaterMark value.

Answer 1

@Evgeny-1093, If you are still having trouble, can you please reach out to us at azcommunity@microsoft.com with your Azure subscription ID? We'd like to provide you with a free support case to get you the support needed.

If you found a solution, we would appreciate it if you shared the solution with the community to help others with similar issues.