Overview of the Azure Event Hubs Dedicated tier
Azure Event Hubs dedicated clusters are designed to meet the needs of the most demanding mission-critical event streaming workloads. These clusters provide a high-performance, low-latency, scalable, and reliable event streaming service for your event streaming applications that are based on the Advanced Message Queuing Protocol (Event Hubs SDK) or Apache Kafka APIs.
Note
The Dedicated tier isn't available in all regions. Try to create a dedicated cluster in the Azure portal. See the supported regions in the Location dropdown list on the Create Event Hubs Cluster page.
Why dedicated clusters?
The Dedicated tier of Event Hubs offers several benefits to customers who need to run mission-critical workloads at enterprise-level capacity.
Low-latency event streaming
These clusters are optimized for low end-to-end latency and high performance. These clusters enable businesses to handle high-velocity and high-volume data streaming.
Stream large volumes of data
Dedicated clusters can stream events at the scale of gigabytes per second or millions of events per second for most of the use cases. You can also scale these clusters to accommodate changes in event streaming volume.
Guaranteed consistent performance
Event Hubs dedicated clusters minimize the latency jitter and ensure consistent performance with guaranteed capacity.
Zero interference
Event Hubs dedicated clusters operate on a single-tenant architecture. This architecture ensures that the allocated resources aren't being shared with any other tenants. Unlike with other tiers, you won't see any cross-tenant interference in a dedicated cluster.
Self-serve scaling
The dedicated cluster offers self-serve scaling capabilities that allow you to adjust the capacity of the cluster according to dynamic loads and to facilitate business operations. You can scale out during spikes in usage and scale in when the usage is low.
High-end features and generous quotas
Dedicated clusters include all features of the Premium tier and more. The service also manages load balancing, operating system updates, security patches, and partitioning. You can spend less time on infrastructure maintenance and more time on building your event streaming applications.
Supports streaming large messages
In most streaming scenarios, data is lightweight, typically less than 1 MB, and requires high throughput. There are instances where messages can't be divided into smaller segments. Self-serve dedicated clusters can accommodate events up to 20 MB of size at no extra cost. This capability allows Event Hubs to handle a wide range of message sizes to ensure uninterrupted business operations. For more information, see Send and receive large messages with Azure Event Hubs.
Capacity units
Dedicated clusters are provisioned and billed by capacity units (CUs), which is a preallocated amount of CPU and memory resources.
How much you can ingest and stream per CU depends on factors such as the:
- Number of producers and consumers.
- Number of partitions.
- Producer and consumer configuration.
- Payload size.
- Egress rate.
To determine the necessary number of CUs, you should carry out your anticipated event streaming workload on an Event Hubs dedicated cluster while you observe the cluster's resource utilization. For more information, see When should I scale my dedicated cluster.
Cluster types
Event Hubs dedicated clusters come in two distinct types: self-serve scalable clusters and legacy clusters. These two types differ in their support for the number of CUs, the amount of throughput each CU provides, and the regional and zone availability.
As a dedicated cluster user, you can determine the type of cluster by examining the availability of the capacity scaling feature in the portal. If this capability is present, you're using a self-serve scalable cluster. Conversely, if it isn't available, you're using a legacy dedicated cluster. Alternatively, you can look for the Azure Resource Manager properties related to dedicated clusters.
Self-serve scalable clusters
Event Hubs self-serve scalable clusters are based on new infrastructure and allow users to scale the number of CUs allocated to each cluster. By creating a dedicated cluster through the Event Hubs portal or Azure Resource Manager templates (ARM templates), you gain access to a self-service scalable cluster. To learn how to scale your dedicated cluster, see Scale Event Hubs dedicated clusters.
Approximately one CU in a self-serve scalable cluster provides ingress capacity ranging from 100 MB/sec to 200 MB/sec, although actual throughput might fluctuate depending on various factors.
With self-serve scalable clusters, you can purchase up to 10 CUs for a cluster in the Azure portal. In contrast to traditional clusters, these clusters can be scaled incrementally with CUs ranging from 1 to 10. If you need a cluster larger than 10 CUs, you can submit a support request to scale up your cluster after its creation.
Important
To create or scale an AZ-enabled self-serve cluster, you must submit a support request requesting three CUs or greater. A three CU (or greater) self-serve cluster created via Portal is NOT AZ-enabled.
Legacy clusters
Event Hubs dedicated clusters created before the availability of self-serve scalable clusters are referred to as legacy clusters.
To use these legacy clusters, direct creation through the Azure portal or ARM templates isn't possible. Instead, you must submit a support request to create one.
Approximately one CU in a legacy cluster provides ingress capacity ranging from 50 MB/sec to 100 MB/sec, although actual throughput might fluctuate depending on various factors.
With a legacy cluster, you can purchase up to 20 CUs.
Legacy Event Hubs dedicated clusters require at least eight CUs to enable availability zones. Availability zone support is only available in Azure regions with availability zones.
Important
Migrating an existing legacy cluster to a self-serve cluster isn't currently supported. For more information, see Migrating a legacy cluster to a self-serve scalable cluster.
Determine the cluster type
You can determine the cluster type that you're using with the following methods.
| Method | Action | Self-serve scalable clusters | Legacy clusters | Notes |
|---|---|---|---|---|
| Use the portal | Check for the presence of the Scale tab under the cluster. | The Scale page is available in the cluster UI. | No Scale page is available in the cluster UI. | |
| Use Azure Resource Manager | Check for the supportsScaling Azure Resource Manager property on the cluster. |
Check for the presence of the Scale page under the cluster. | No Scale page is available in the cluster UI. | Check this property in the portal, the Azure CLI, or PowerShell. Needs API version 2022-01-01-preview or newer. |
Use nslookup |
Run the nslookup command on a namespace in a cluster. |
CNAME maps to *.cloudapp.azure.com. |
CNAME maps to *.cloudapp.net. |
Example: nslookup ns.servicebus.windows.net. |
Quotas and limits
The Event Hubs Dedicated offering is billed at a fixed monthly price with a minimum of four hours of usage. The Dedicated tier offers all the features of the Premium plan, but with enterprise-scale capacity and limits for customers with demanding workloads.
For more information about quotas and limits, see Event Hubs quotas and limits.
FAQs
What can I achieve with a cluster?
For an Event Hubs cluster, how much you can ingest and stream depends on factors such as your producers, consumers, and the rate at which you're ingesting and processing.
The following table shows the benchmark results that we achieved during our testing with a legacy dedicated cluster.
| Payload shape | Receivers | Ingress bandwidth | Ingress messages | Egress bandwidth | Egress messages | Total TUs | TUs per CU |
|---|---|---|---|---|---|---|---|
| Batches of 100x1KB | 2 | 400 MB/sec | 400k messages/sec | 800 MB/sec | 800k messages/sec | 400 TUs | 100 TUs |
| Batches of 10x10KB | 2 | 666 MB/sec | 66.6k messages/sec | 1.33 GB/sec | 133k messages/sec | 666 TUs | 166 TUs |
| Batches of 6x32KB | 1 | 1.05 GB/sec | 34k messages/sec | 1.05 GB/sec | 34k messages/sec | 1,000 TUs | 250 TUs |
In the testing, the following criteria were used:
- A Dedicated-tier Event Hubs cluster with four CUs was used.
- The event hub used for ingestion had 200 partitions.
- The data that was ingested was received by two receiver applications receiving from all partitions.
Can I scale up or scale down my cluster?
If you create the cluster with the Support scaling option set, you can use the self-serve experience to scale out and scale in, as needed. You can scale up to 10 CUs with self-serve scalable clusters. Self-serve scalable dedicated clusters are based on new infrastructure, so they perform better than dedicated clusters that don't support self-serve scaling. The performance of dedicated clusters depends on factors such as resource allocation, number of partitions, and storage. We recommend that you determine the required number of CUs after you test with a real workload.
Submit a support request to scale out or scale in your dedicated cluster in the following scenarios:
- You need more than 10 CUs for a self-serve scalable dedicated cluster (a cluster that was created with the Support scaling option set).
- You need to scale out or scale in a cluster that was created without selecting the Support scaling option.
- You need to scale out or scale in a dedicated cluster that was created before the self-serve experience was released.
Warning
You won't be able to delete the cluster for at least four hours after you create it. You're charged for a minimum of four hours of usage of the cluster. For more information on pricing, see Event Hubs pricing.
Can I migrate from a legacy cluster to a self-serve scalable cluster?
Because of the difference in the underlying hardware and software infrastructure, we don't currently support migration of clusters that don't support self-serve scaling to self-serve scalable dedicated clusters. If you want to use self-serve scaling, you must re-create the cluster. To learn how to create a scalable cluster, see Create an Event Hubs dedicated cluster.
When should I scale my dedicated cluster?
CPU consumption is the key indicator of the resource consumption of your dedicated cluster. When the overall CPU consumption begins to reach 70% (without observing any abnormal conditions, such as a high number of server errors or a low number of successful requests), that means your cluster is moving toward its maximum capacity. You can use this information as an indicator to consider whether you need to scale up your dedicated cluster or not.
To monitor the CPU usage of the dedicated cluster, follow these steps:
On the Metrics page of your Event Hubs dedicated cluster, select Add metric.
Select CPU as the metric and use Max as the aggregation.
Select Add filter and add a filter for the Property type Role. Use the equal operator and select all the values (Backend and Gateway) from the dropdown list.
Then you can monitor this metric to determine when you should scale your dedicated cluster. You can also set up alerts against this metric to get notified when CPU usage reaches the thresholds you set.
How does geo-disaster recovery work with my cluster?
You can geo-pair a namespace under a Dedicated-tier cluster with another namespace under a Dedicated-tier cluster. We don't encourage pairing a Dedicated-tier namespace with a namespace in the Standard offering because the throughput limit is incompatible and results in errors.
Can I migrate my Standard or Premium namespaces to a Dedicated-tier cluster?
We don't currently support an automated migration process for migrating your Event Hubs data from a Standard or Premium namespace to a dedicated one.
Why does a legacy zone-redundant dedicated cluster have a minimum of eight CUs?
To provide zone redundancy for the Dedicated offering, all compute resources must have three replicas across three datacenters in the same region. This minimum requirement supports zone redundancy (so that the service can still function when two zones or datacenters are down) and results in a compute capacity equivalent to eight CUs.
We can't change this quota. It's a restriction of the current architecture with a Dedicated tier.
Related content
Contact your Microsoft sales representative or Microsoft Support to get more details about Event Hubs Dedicated. You can also create a cluster or learn more about Event Hubs pricing tiers:
- Create an Event Hubs cluster through the Azure portal.
- Check out Event Hubs Dedicated pricing. You can also contact your Microsoft sales representative or Microsoft Support to get more details about Event Hubs Dedicated capacity.
- See the Event Hubs FAQ for pricing information and answers to frequently asked questions about Event Hubs.