(Cosmos db) After uploading data ,Need to wait for a period of time (a few minutes ~ half an hour) to retrieve the latest added data

xialingti 1

Hi
After I import a large amount of data (more than 10000) into the cosmosdb container at one time,
I try to retrieve data in the Data Explorer , but I can’t retrieve the newly added data.
Only the existing data can be retrieved.
I need to wait for a while (a few minutes~
Half an hour) to query the latest other data.
But I checked the timestamp in the newly added data and it is consistent with the added time point.
Can you tell me what causes this to happen
I tryed to change the scale
change to Autoscale
and increase the maximum RU/s

The situation has improved, but sometimes the above situation still occurs

Are there any known issues related to this problem?

please provide me help in solving this .thanks.

Anurag Sharma 17,571 Reputation points

2021-04-19T10:21:04.47+00:00
Hi @xialingti , welcome to Microsoft Q&A forum.

Could you please provide below information to look further into the issue:

How are you importing the docs?

Manual or autoscale throughput, and what is the RU set?

Which region you have cosmos account created in?

Could you provide a sample doc please?
xialingti 1 Reputation point

2021-04-21T07:37:42.247+00:00

Hi

How are you importing the docs?

The number of parallel tasks is 10
Each task handles the upload of 1000 pieces of data
and then
I use the following method provided by Microsoft.Azure.DocumentDB.Core.dl to import data
Task<ResourceResponse<Document>> UpsertDocumentAsync(Uri documentCollectionUri, object document, RequestOptions options = null, bool disableAutomaticIdGeneration = false, CancellationToken cancellationToken = default);

Manual or autoscale throughput, and what is the RU set?

Manual throughput 1700RU/s

Which region you have cosmos account created in?

Japan West

Could you provide a sample doc please?

json file cannot be added, so I attached a sample with txt type89710-sample.txt
Anurag Sharma 17,571 Reputation points

2021-04-22T05:58:42.13+00:00

Hi @xialingti , thanks for providing more details. This information is helpful. I am checking on this and will get back at the earliest.
Anurag Sharma 17,571 Reputation points

2021-04-23T14:20:58.123+00:00

Hi @xialingti , thanks for your patience.

I tried uploading 20,000 documents to Japan West region and there was not any delay. I kept the RU at Manual 1700.

Could you please let me know how are you querying the documents after loading?
xialingti 1 Reputation point

2021-04-27T02:45:50.66+00:00

Hi @AnuragSharma-MSFT ,thanks for you support.

I select the data by the following query:
e.g.

WHERE c.RelationId="1_1-1" and c.Code= "1"

btw.
The same Code can have multiple document results in cosmos db
And so far, there are about 600,000 documents saved in cosmosdb
Anurag Sharma 17,571 Reputation points

2021-04-28T03:45:01.367+00:00

Hi @xialingti , thanks for replying back. This issue seems to be very specific to the cosmos account you created. But in my case it was just 20,000 documents in the container till now. I can try adding more documents to see the latency if any and get back to you..
KalyanChanumolu-MSFT 8,316 Reputation points

2021-05-06T11:33:11.487+00:00

@xialingti We will not be able to troubleshoot this issue remotely.
Please reach out to Azure Support so that an engineer can work with you and help you resolve this issue.