How to order documents in a Cosmos DB collection by the time inserted

Rune Larsen 1 Reputation point
2020-06-14T06:11:14.85+00:00

We are building a large platform where we want to expose events to many subscribers.

We have decided we want to expose the events through a REST-API to be able to support all our use cases. We are not able to use event hub or Kafka since we need millions of topics. We also need to authorize access to the different events based on type of event and which person/business is related to.

One option is to use Cosmos DB as storage, but I am not able to see how we could get the order of the elements correctly. I understand that _ts property contains the epoch time since last changed, but that is only in seconds and would cause us to have hundreds of events at the same time.

Is there a way? I guess we could use the change feed to trigger something but does the change feed have some index we could set back on the document?

Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,437 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. FelixP 86 Reputation points
    2020-06-14T11:12:18.32+00:00

    Hi - just starting answer questions on this platform so bear with me.

    Are you looking to order event reads or writes? It might be worth considering what consistency level you need, as this could affect the ordering you see.

    This page of docs might be helpful:

    If you need stricter consistency guarantees than the ones provided by session consistency and single-digit-millisecond latency for writes, it is recommended that you use bounded staleness consistency level.

    https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels-choosing

    The cosmos dB change feed might also be what you’re looking for:

    There is guaranteed order in the change feed within a partition key value but not across partition key values. You should select a partition key that gives you a meaningful order guarantee.

    https://learn.microsoft.com/en-us/azure/cosmos-db/change-feed-design-patterns

    Please let me know if this helps.

    0 comments
  2. Mark Brown - MSFT 2,761 Reputation points Microsoft Employee
    2020-06-14T15:48:01.54+00:00

    Cosmos only has second granularity with _ts. If you need a finer grain you will need to create your own timestamp property. Data is always stored in the order it is committed. However I would look to use a composite index with the partition key and order by on your timestamp property as queries will be run from this index and will be less expensive than without where you use order by in your queries.