Overview of Azure Event Hubs Dedicated tier

Event Hubs Dedicated clusters are designed to meet the needs of most demanding mission-critical event streaming workloads. These clusters provide a high-performance, low-latency, scalable, and reliable event streaming service for your event streaming applications that are based on AMQP(Event Hubs SDK) or Apache Kafka APIs.

Note

The Dedicated tier isn't available in all regions. Try to create a Dedicated cluster in the Azure portal and see supported regions in the Location drop-down list on the Create Event Hubs Cluster page.

Why Dedicated cluster?

The Dedicated tier of Event Hubs offers several benefits to customers who need run mission-critical workloads at enterprise-level capacity.

Low latency event streaming

These clusters are optimized for low end-to-end latency and high performance. Therefore these clusters enable businesses to handle high-velocity and high-volume data streaming.

Streaming large volumes of data

Dedicated clusters can stream events at the gigabytes per second or millions of events per second scale for Most of the use cases. Also, these clusters can be easily scaled to accommodate changes in event streaming volume.

Guaranteed consistent performance

Event Hubs Dedicated clusters minimize the latency jitter and ensure consistent performance with guaranteed capacity.

Zero interference

Event Hubs Dedicated Clusters operate on a single-tenant architecture. Therefore it ensures that the allocated resources being not shared with any other tenants. Therefore, unlike other tiers, you wouldn't see any cross tenant interference in Dedicated cluster.

Self-serve scaling

The Dedicated cluster offers self-serve scaling capabilities that allow you to adjust the capacity of the cluster according to dynamic loads and to facilitate business operations. You can scale out during spikes in usage and scale in when the usage is low.

High-end features and generous quotas

Dedicated clusters include all features of the Premium tier and more. The service also manages load balancing, operating system updates, security patches, and partitioning. So, you can spend less time on infrastructure maintenance and more time on building your event streaming applications.

Capacity Units(CU)

Dedicated clusters are provisioned and billed by capacity units (CUs), a pre-allocated amount of CPU and memory resources.

How much you can ingest and stream per CU depends on various factors, such as the following ones:

  • Number of producers and consumers
  • Number of partitions.
  • Producer and consumer configuration.
  • Payload size
  • Egress rate

Therefore, to determine the necessary number of CUs, you should carry out your anticipated event streaming workload on an Event Hubs Dedicated cluster while observing the cluster's resource utilization. For more information, see When to scale my Dedicated cluster.

Cluster Types

Event Hubs Dedicated Clusters come in two distinct types: Self-serve scalable clusters and Legacy clusters. These two types differ in their support for the number of CUs, the amount of throughput each CU provides, and the regional and zone availability.

As a Dedicated cluster user, you can determine the type of cluster by examining the availability of the capacity scaling feature in the portal. If this capability is present, you're using a self-serve scalable cluster. Conversely, if it isn't available, you're utilizing a legacy Dedicated cluster. Alternatively you can look for the Azure Resource Manager properties related to Dedicated clusters.

Self-serve scalable clusters

Event Hubs Self-serve scalable clusters are based on new infrastructure and allow users to easily scale the number of capacity units allocated to each cluster. By creating a Dedicated cluster through the Event Hubs portal or ARM templates, you gain access to a self-service scalable cluster. To learn how to scale your Dedicated cluster, see Scale Event Hubs Dedicated clusters.

Approximately, one capacity unit (CU) in a self-serve scalable cluster provides ingress capacity ranging from 100 MB/s to 200 MB/s, although actual throughput may fluctuate depending on various factors.

With self-serve scalable clusters, you can purchase up to 10 CUs for a cluster in the Azure portal. In contrast to traditional clusters, these clusters can be scaled incrementally with CUs ranging from 1 to 10. If you need a cluster larger than 10 CU, you can submit a support request to scale up your cluster after its creation.

Important

Self-serve scalable Dedicated can be deployed with availability zones enabled with 3 CUs but you won't be able to use the self-serve scaling capability to scale the cluster. To create or scale an AZ enabled self-serve cluster you must submit a support request.

Legacy clusters

Event Hubs Dedicated clusters created prior to the availability of self-serve scalable clusters are referred to as legacy clusters.

To use these legacy clusters, direct creation through the Azure portal or ARM templates isn't possible and you must instead submit a support request to create one.

Approximately, one capacity unit (CU) in a legacy cluster provides ingress capacity ranging from 50 MB/s to 100 MB/s, although actual throughput may fluctuate depending on various factors.

With Legacy cluster, you can purchase up to 20 CUs.

Note

Legacy Event Hubs Dedicated clusters require at least 8 Capacity Units(CUs) to enable availability zones. Availability zone support is only available in Azure regions with availability zones.

Important

Migrating an existing Legacy cluster to a Self-Serve Cluster isn't currently support. For more information, see migrating a Legacy cluster to Self-Serve Scalable cluster..

Determining cluster type

You can determine the cluster type that you're using the following methods.

Method Action Self-serve scalable clusters Legacy clusters Notes
Using Portal Check for presence of ‘Scale’ tab under the cluster ‘Scale’ page available in the cluster UI. No scale page available in the cluster UI.
Using Azure Resource Manager Check for supportsScaling Azure Resource Manager property on cluster. Check for presence of ‘Scale’ page under the cluster.  No scale page available in the cluster UI. Check this property on Portal, CLI or PowerShell. Needs API version 2022-01-01-preview or newer.
Using nslookup Run nslookup command on a namespace in cluster. CNAME maps to *.cloudapp.azure.com CNAME maps to *.cloudapp.net. Example: nslookup ns.servicebus.windows.net.

Quotas and limits

The Event Hubs Dedicated offering is billed at a fixed monthly price, with a minimum of 4 hours of usage. The Dedicated tier offers all the features of the premium plan, but with enterprise-scale capacity and limits for customers with demanding workloads.

For more information about quotas and limits, see Event Hubs quotas and limits

FAQs

What can I achieve with a cluster?

For an Event Hubs cluster, how much you can ingest and stream depends on various factors such as your producers, consumers, the rate at which you're ingesting and processing, and much more.

The following table shows the benchmark results that we achieved during our testing a Legacy dedicated cluster:

Payload shape Receivers Ingress bandwidth Ingress messages Egress bandwidth Egress messages Total TUs TUs per CU
Batches of 100x1KB 2 400 MB/sec 400k messages/sec 800 MB/sec 800k messages/sec 400 TUs 100 TUs
Batches of 10x10KB 2 666 MB/sec 66.6k messages/sec 1.33 GB/sec 133k messages/sec 666 TUs 166 TUs
Batches of 6x32KB 1 1.05 GB/sec 34k messages / sec 1.05 GB/sec 34k messages/sec 1000 TUs 250 TUs

In the testing, the following criteria were used:

  • A dedicated-tier Event Hubs cluster with 4 capacity units (CUs) was used.
  • The event hub used for ingestion had 200 partitions.
  • The data that was ingested was received by two receiver applications receiving from all partitions.

Can I scale up/down my cluster?

If you created the cluster with the Support Scaling option set, you can use the self-serve experience to scale out and scale in as needed. You can scale up to 10 CUs with self-serve scalable clusters. As self-serve scalable dedicated clusters are based out of new infrastructure, they're bound to be performant over dedicated clusters that don't support self-serve scaling. As the performance of dedicated clusters depends on various factors such as resource allocation, number of partitions, storage, and so on, we recommend you to determine the required number of CUs after testing with a real workload.

Submit a support request in the following scenarios to scale out or scale in your dedicated cluster.

  • You need more than 10 CUs for a self-serve scalable dedicated cluster (a cluster that was created with the Support scaling option set).
  • You need to scale out or scale in a cluster that was created without selecting the Support scaling option
  • You need to scale out or scale in a dedicated cluster that was created before the self-serve experience was released

Warning

You won't be able to delete the cluster for at least 4 hours after you create it. Therefore, you will be charged for a minimum 4 hours of usage of the cluster. For more information on pricing, see Event Hubs - Pricing.

Can I migrate from a Legacy cluster to a Self-Serve Scalable cluster?

Due to difference in the underlying hardware and software infrastructure, we don't currently support migration of clusters that don't support self-serve scaling to self-serve scalable dedicated clusters. If you would wish to use self-serve scaling, you must recreate the cluster. To learn how to create scalable cluster, see Create an Event Hubs dedicated cluster.

When to scale my dedicated cluster?

CPU consumption is the key indicator of the resource consumption of your dedicated cluster. When the overall CPU consumption is reaching 70% (without observing any abnormal conditions such as high number of server errors or low successful requests), that means your cluster is moving towards its maximum capacity. Therefore you can use this as an indicator to consider whether you need to scale up your dedicated cluster or not.

To monitor the CPU usage of the dedicated cluster, you need to follow these steps.

  • In the metrics page of your Event Hubs Dedicated cluster, select Add metric.

  • Select CPU as the metric and use the Max as the aggregation.

    Screenshot showing the Metrics page with the CPU metric.

  • Then, select Add filter, and add a filter for the property type Role, use the equal operator and select all three values(Backend, Gateway) from the dropdown.

    Screeshot showing the metrics page with CPU consumption metric and roles.

    Then you can monitor this metric to determine when you should scale your dedicated cluster. You can also set up alerts against this metric to get notified when CPU usage reaches the thresholds you set.

How does Geo-DR work with my cluster?

You can geo-pair a namespace under a dedicated-tier cluster with another namespace under a dedicated-tier cluster. We don't encourage pairing a dedicated-tier namespace with a namespace in our standard offering because the throughput limit will be incompatible and result in errors.

Can I migrate my standard or premium namespaces to a Dedicated-tier cluster?

We don't currently support an automated migration process for migrating your event hubs data from a standard or premium namespace to a dedicated one.

Why does a zone redundant dedicated cluster have a minimum of 8 CU?

In order to provide zone redundancy for the dedicated offering, all compute resources must have 3 replicas across 3 datacenters in the same region. This is the minimum requirement to support zone redundancy (so that the service can still function when 2 zones/datacenters are down) and results in a compute capacity equivalent to 8 CUs.

So this is not a quota that we can change but rather a restriction of the current architecture with a dedicated tier.

Next steps

Contact your Microsoft sales representative or Microsoft Support to get more details about Event Hubs Dedicated. You can also create a cluster or learn more about Event Hubs pricing tiers by visiting the following links: