Logical and Physical Partition Relation

Question

Logical and Physical Partition Relation

George Luiz Bittencourt 21 Microsoft Employee

Hello,

I have a container with a single logical partition (LP) because I don't have a partition key defined. From the docs, I see that a LP will be assigned to only one physical partition (PP).

However, if I query this container in the CDBPartitionKeyRUConsumption table in the log analytics workspace I can see that this LP is assigned to more than on PP.
My understanding is that the PartitionKeyRangeId is the PP, so, in this case, I would see only one row from the Kusto query below. However, I am seeing several rows.

CDBPartitionKeyRUConsumption
| where DatabaseName == 'SampleDB' and CollectionName == 'Persons'
| distinct PartitionKeyRangeId

What I want to understand is what makes a LP to be assigned to more than one PP.
This information is important so we can better understand our RU usage.

Thanks for any information.

Anurag Sharma 17,636 Reputation points

2021-10-04T06:40:20.643+00:00

Hi @George Luiz Bittencourt , welcome to Microsoft Q&A forum.

You are right as one logical partition can only belong to one physical partition.

Would it be possible for you to share the screenshot of query result? Also just wanted to understand how have you created a container without partition key.
George Luiz Bittencourt 21 Reputation points Microsoft Employee

2021-10-04T16:43:45.657+00:00

Hello,

Thanks for your response!

What I meant by having a container without a partition key is that I have one defined for a property I don't use. So all documents are allocated in a single logical partition in a "null" partition.

I have attached two screenshots showing the results of my queries. I have erased the table, database and resource name to hide the name for NDA compliance.

Thanks for any help.

1 answer

Your answer

Anurag Sharma 17,636 Reputation points

2021-10-04T06:40:20.643+00:00

Hi @George Luiz Bittencourt , welcome to Microsoft Q&A forum.

You are right as one logical partition can only belong to one physical partition.

Would it be possible for you to share the screenshot of query result? Also just wanted to understand how have you created a container without partition key.
George Luiz Bittencourt 21 Reputation points Microsoft Employee

2021-10-04T16:43:45.657+00:00

Hello,

Thanks for your response!

What I meant by having a container without a partition key is that I have one defined for a property I don't use. So all documents are allocated in a single logical partition in a "null" partition.

I have attached two screenshots showing the results of my queries. I have erased the table, database and resource name to hide the name for NDA compliance.

Thanks for any help.

Answer 1

Anurag Sharma 17,636

Thanks @GeorgeLuizBittencourt-414 for your response.

Logical partitions are created based on the partition key value. In case we don't pass any value(null) for partition key, Cosmos DB can assign them into multiple logical containers or multiple physical partitions as nulls are treated differently than same value. Now if we talk about just a few hundred documents, chances are there that they can fall into same logical/physical partition but if there are many items they could go to different partitions.

You can try this by creating documents with same partition key and observe they should fall into same logical partition.

Please let us know if this helps.

George Luiz Bittencourt 21 Reputation points Microsoft Employee

2021-10-05T09:51:13.047+00:00

Hello @AnuragSharma-MSFT!

Thanks for your response.

Can you please elaborate more about how null is treated or point me to some documentation?
II am asking this because my request charges are not evenly balanced for this container.

Thanks for any help!
Anurag Sharma 17,636 Reputation points

2021-10-05T11:02:46.787+00:00
From the log analytics it looks that most of the request charge is consumed by partition 26, that should mean most of the create operations are happening on that partition. Could you please run below query and let me know the result?

CDBPartitionKeyRUConsumption | where DatabaseName == '<yourdb>' and CollectionName == '<yourcollection>' |summarize TotalRUs = sum(RequestCharge) by OperationName,PartitionKeyRangeId
George Luiz Bittencourt 21 Reputation points Microsoft Employee

2021-10-05T12:22:46.753+00:00

Hello,

I am sending the data you requested. The PP 26 is definitely a hot partition.
However I still don't understand the reason for the queries on the other PPs.

If I submit a cross-partition query (query below) from Data Explorer and download the query metrics per partition I can see that other partition are queried. It seems like Cosmos DB doesn't know I have only one partition and the only way to resolve this query is sending to all PPs. I haven't said, but I have other containers in this database.

SELECT COUNT(1) AS columnACount, c.ColumnA
FROM c
GROUP BY c.ColumnA
Anurag Sharma 17,636 Reputation points

2021-10-06T08:06:41.693+00:00

Thanks for your response.

From the queries/screenshot provided it looks like the documents are part of physical partition 26 only based on Request Charges. I tried replicating the same at my side and I noticed that documents with no partition key go to same physical partition, but I also observed we have multiple physical partitions created. In my case I was having a throughput of 50K and as per limit one Physical partition has the limit of 10K, so it created 5 physical partitions. What is the throughput you have configured for this container?

Now the request charges that we see for other partitions is related to 'Query' operation performed throughout all the partitions, this could be scanning done across partitions before inserting the documents. However I am checking this as well with product team on the same.

Please let us know if this helps or we can discuss further.
George Luiz Bittencourt 21 Reputation points Microsoft Employee

2021-10-06T13:27:44.377+00:00

Hello!

This container is in a database with other containers and is using the throughput defined at the database level. It was 150K RUs, manually provisioned, and now is 20K RU in auto-scale mode.

We are also using version 2 of the SDK, so the class used is the DocumentClient instead of CosmosClient. Maybe it is related to the client issuing "wrong" queries.

Thanks for all your help.
Anurag Sharma 17,636 Reputation points

2021-10-07T06:51:20.787+00:00

Thanks for responding back. For 150k provisioned RUs, it would have created multiple physical partitions but all non-partitioned docs should be in same partition. Also version 2 sdk is getting retired by end August 2024, V3 offers many performance improvement. You can refer to below docs for more details:

Version 2.x of the Azure Cosmos DB .NET SDK will be retired on 31 August 2024

Share via

Logical and Physical Partition Relation

1 answer

Your answer