Nota
L-aċċess għal din il-paġna jeħtieġ l-awtorizzazzjoni. Tista’ tipprova tidħol jew tibdel id-direttorji.
L-aċċess għal din il-paġna jeħtieġ l-awtorizzazzjoni. Tista’ tipprova tibdel id-direttorji.
Azure DocumentDB supports sharding to horizontally distribute data and traffic. The documents within a collection are divided into chunks called logical shards.
Approaches to horizontal scalability
Azure DocumentDB provides two complementary ways to distribute data and traffic across the physical shards in a cluster:
- Shard a collection with a shard key: Distribute a single collection's documents across the physical shards by choosing a shard key. Use this approach when a collection's storage or throughput exceeds the capacity of a single physical shard.
- Place collections across shards: Keep collections unsharded and place or colocate them on specific physical shards. Use this approach when individual collections fit on a single physical shard, but you want to control which collections share a shard. For example, you can isolate a cache-sensitive collection onto its own shard.
These approaches complement each other. Sharded and unsharded collections coexist in the same cluster, so you can shard the collections that outgrow a single physical shard and place the remaining unsharded collections across the cluster.
Logical shards
Define sharding individually for each collection by using a designated shard key from the collection's document structure. Bucket data into chunks, with each chunk corresponding to a logical partition. Documents for each unique value of the shard key property reside in the same logical shard.
For each document you insert into a sharded collection, hash the value of the shard key property to compute the designated logical shard. The service abstracts and fully manages the placement of the logical shard and the distribution of all logical shards within the cluster.
All documents containing the same value for the shard key belong to the same logical shard.
For example, let's consider a collection called Employees with the document structure below.
This table shows a mapping of shard key values to logical partitions.
| Document ID | Shard Key Value | Logical Shard |
|---|---|---|
| "12345" | "Steve Smith" | Shard 1 |
| "23456" | "Jane Doe" | Shard 2 |
| "34567" | "Steve Smith" | Shard 1 |
| "45678" | "Michael Smith" | Shard 3 |
| "56789" | "Jane Doe" | Shard 2 |
There are no limits to the number of logical shards for a collection. A collection can have as many logical shards as documents with a unique value for the shard key property in each document.
There are also no limits to the size of a single logical shard.
In addition, the service doesn't limit transactions to the scope of a logical shard. Azure DocumentDB supports read and write transactions that are applicable across multiple logical shards and across multiple physical shards in the cluster.
Physical shards
Physical shards are the underlying machines and disks responsible for persisting the data and fulfilling database transactions. Unlike logical shards, the service manages physical shards under the covers.
The number of physical shards is defined when a cluster is created and can be increased if database size grows over time. Single shard clusters have one physical shard (node) that is entirely responsible for the cluster's storage and database transactions. Multishard clusters distribute the data and transaction volume across the physical shards in the cluster.
Mapping logical shards to physical shards
When new logical shards are added, the cluster seamlessly updates the mapping of logical to physical shards. Similarly, the assignment of the address space to each physical shard is changed as new physical shards are added to the cluster after which, logical shards are rebalanced across the cluster.
The hash range used to map logical and physical shards is evenly distributed across the physical shards in the cluster. Each physical shard owns an evenly sized bucket of the hash range. For every document that is written, the value of the shard key property is hashed and the hash value determines the mapping of the document to the underlying physical shard. Internally, several logical shards map to a single physical shard. Moreover, logical shards are never split across physical shards and all the documents for a logical shard only map to one physical shard.
Building on the prior example using a cluster with two physical shards, this table shows a sample mapping of documents to physical shards.
| Document ID | Shard Key Value | Logical Shard | Physical Shard |
|---|---|---|---|
| "12345" | "Steve Smith" | Shard 1 | Physical Shard 1 |
| "23456" | "Jane Doe" | Shard 2 | Physical Shard 2 |
| "34567" | "Steve Smith" | Shard 1 | Physical Shard 1 |
| "45678" | "Michael Smith" | Shard 3 | Physical Shard 1 |
| "56789" | "Jane Doe" | Shard 2 | Physical Shard 2 |
Capacity of physical shards
The cluster tier that is selected when the cluster is provisioned determines the CPU and memory capacity of a physical shard. Similarly the storage SKU determines the storage and IOPS capacity of a physical shard. Larger cluster tiers provide more compute power and larger memory while larger storage disks provide more storage and IOPS. Read heavy workloads benefit from a larger cluster tier while write heavy workloads benefit from a larger storage SKU. The cluster tier can be scaled up and down after the cluster is created based on the changing needs of the application.
In a multishard cluster, the capacity of each physical shard is the same. Scaling up the cluster tier or the storage SKU doesn't change the placement of logical shards on the physical shards. After a scale up operation, the number of physical shards remains the same thus avoiding the need to rebalance the data in the cluster.
The compute, memory, storage, and IOPS capacity of the physical shard determine the resources available for the logical shards. Shard keys that don't have an even distribution of storage and request volumes can cause uneven storage and throughput consumption within the cluster. Hot partitions can cause physical shards to be unevenly utilized leading to unpredictable throughput and performance. Thus sharded clusters require careful planning upfront to ensure performance remains consistent as the requirements of the application change over time.
Replica sets
Each physical shard consists of a set of replicas, also referred to as a replica set. Each replica hosts an instance of the database engine. A replica set makes the data store within the physical shard durable, highly available, and consistent. Each replica that makes up the physical shard inherits the physical shard's storage and compute capacity. Azure DocumentDB automatically manages replica sets.
How to shard a collection
Consider the following document within the 'cosmicworks' database and 'employee' collection
{
"firstName": "Steve",
"lastName": "Smith",
"companyName": "Microsoft",
"division": "Azure",
"subDivision": "Data & AI",
"timeInOrgInYears": 7
}
The following sample shards the employee collection within the cosmicworks database on the firstName property.
use cosmicworks;
sh.shardCollection("cosmicworks.employee", {"firstName": "hashed"})
The collection can also be sharded using an admin command:
use cosmicworks;
db.adminCommand({
"shardCollection": "cosmicworks.employee",
"key": {"firstName": "hashed"}
})
While it isn't ideal to change the shard key after the collection has grown significantly in storage volume, the reshardCollection command can be used to alter the shard key.
use cosmicworks;
sh.reshardCollection("cosmicworks.employee", {"lastName": "hashed"})
The collection can also be resharded using an admin command:
use cosmicworks;
db.adminCommand({
"reshardCollection": "cosmicworks.employee",
"key": {"lastName": "hashed"}
})
As a best practice, an index must be created on the shard key property.
use cosmicworks;
db.runCommand({
createIndexes: "employee",
indexes: [{"key":{"firstName":1}, "name":"firstName_1", "enableLargeIndexKeys": true}],
blocking: true
})
Collection placement across physical shards
By default, the service automatically places each collection on a physical shard and distributes collections across the cluster to evenly use compute and storage. You don't need to manage placement for most workloads.
In some cases, you want explicit control over where a collection lives. For example, a collection that serves unfiltered find() queries performs well only when it stays fully in memory. When such a collection shares a physical shard with other high-throughput collections, they compete for the same in-memory cache. Cache evictions then add latency to the collection that can least afford it. You can isolate these collections onto a dedicated physical shard, or colocate related collections on the same shard, without introducing a shard key or changing how the application queries the data.
Collection placement applies to unsharded collections. A sharded collection is already distributed across the cluster by its shard key.
Move a collection to a specific shard
Use the moveCollection admin command to move an unsharded collection to a specific physical shard. The move operation doesn't convert the collection to a sharded collection and doesn't change how the application queries it.
use cosmicworks;
db.adminCommand({
"moveCollection": "cosmicworks.employee",
"toShard": "shard_0"
})
Consider the following behaviors when you move a collection:
- The collection must be unsharded. You can't pin a sharded collection to a single shard.
- If the collection is already on the target shard, the command is a no-op.
- Move a collection after it's created. If you run the command immediately after you create the collection, it might return a transient error. Wait briefly, then retry the command.
To colocate several collections on the same shard, iterate over the list of collections and run one moveCollection command for each namespace.
use cosmicworks;
const toShard = "shard_0";
const collections = ["employee", "department", "project"];
for (const coll of collections) {
db.adminCommand({ "moveCollection": `cosmicworks.${coll}`, "toShard": toShard });
}
View collection placement
To list the physical shards in the cluster, query the shards collection in the config database. Each _id value is a shard identifier that you can pass to the toShard parameter.
db.getSiblingDB("config").shards.find({}, { "_id": 1 })
To see which shard holds each collection, aggregate the chunks collection in the config database and group by shard. The result contains one entry per shard with the list of collection namespaces on that shard.
db.getSiblingDB("config").chunks.aggregate([
{ "$group": { "_id": "$shard", "collections": { "$addToSet": "$ns" } } },
{ "$sort": { "_id": 1 } }
])
Best practices for sharding data
You don't need to shard data in Azure DocumentDB unless the collection's storage and transaction volumes can exceed the capacity of a single physical shard. For example, the service provides 32 TB disks per shard. If a collection requires more than 32 TB, shard it.
You don't need to shard every collection in a cluster with multiple physical shards. Sharded and unsharded collections can coexist in the same cluster. The service optimally distributes the collections within the cluster to evenly utilize the cluster's compute and storage resources.
For read-heavy applications, select the shard key based on the most frequent query patterns. Choose the most commonly used query filter for a collection as the shard key to optimize the highest percentage of database transactions by localizing the search to a single physical shard.
For write-heavy applications that favor write performance over reads, choose a shard key that evenly distributes data across the physical shards. Shard keys with the highest cardinality provide the best opportunity to uniformly distribute data.
For optimal performance, keep the storage size of a logical shard less than 4 TB.
For optimal performance, distribute logical shards evenly in storage and request volume across the physical shards of the cluster.
A collection doesn't need a shard key to be placed on a specific physical shard. You can isolate a latency-sensitive, unsharded collection onto a dedicated shard so that it doesn't compete for in-memory cache with other collections. For more information, see Collection placement across physical shards.