How to participate in Private Preview?

Next Steps

Proxy support over WebSockets

com.microsoft.azure.eventhubs

New Features

• WebSockets Support (#362) Amqp over WebSockets is particularly useful when enterprise policies (ex: firewall outbound port rules) restrict traffic on the default Amqp secure port (5671). To send or receive to Microsoft Azure Event Hubs over websockets - which uses port 443, set the TransportType on ConnectionStringBuilder, like this:

Bug fixes

• Correct the ExceptionContract when request-response channel closes with transient error (#372)
• PartitionReceiver and PartitionSender creation should participate in RetryPolicy (#373)

com.microsoft.azure.eventhubs

New Features

• Proxy Support - with Basic Authentication (#362) Amqp over WebSockets with proxy support is particularly useful when enterprises want to stream data from on-premise to cloud and IT policies restrict all outbound traffic to flow only via a Proxy Server. To send to or receive from Microsoft Azure Event Hubs via proxy server, set the HTTP proxy settings & set the TransportType on ConnectionStringBuilder to AMQP_WEB_SOCKETS, and you are all set!

// step-1: set the proxy server settings 
ProxySelector.setDefault(new ProxySelector() {            
                   @Override            
                   public List<Proxy> select(URI uri) {                
                      LinkedList<Proxy> proxies = new LinkedList<>();                
                      proxies.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress(proxyIpAddressStr, proxyPort)));                
                      return proxies;     
                  } 
                  @Override
                  public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
                      // trace and follow up on why proxy server is down
                  } 
              }); 
// step-2: configure EventHubClient transport to AmqpWebSockets
connectionStringBuilder.setTransportType(TransportType.AMQP_WEB_SOCKETS);
EventHubClient ehClient = EventHubClient.createSync(connectionStringBuilder.toString()....); 

// hereon, this ehClient instance is ready to talk to EventHub service via the proxy server

Next Steps

What regions will this be offered in?

• Central US
• East US 2
• France Central

How can you enable Availability Zones on your Standard Event Hubs namespace?

Learn more about Azure Event Hubs.

• Event Hubs for Apache Kafka Ecosystems Overview
• Create Apache Kafka enabled Event Hubs
• Event Hubs for Apache Kafka Ecosystems Quickstart
• Using Apache Kafka MirrorMaker with a Kafka enabled Event Hub
• Examples on GitHub

• Data is 1 minute latent compared to 20 minutes latent for the older pipeline you are currently using
• Metrics are available immediately
• View 30 days of history
• Metrics have dimensions that allow you to segment usage and get more insights into the Service
• Setup alerts on metrics that allow you to receive notifications when the metric crosses a threshold you set
• Going forward, Azure Monitor is the pipeline that we will add newer metrics to for further insights into your workload

So what is an alternate name?

1. An alternate name is simply an identifier name that you give to your primary namespace.
2. If you provide your primary namespace to be your alias, you current primary connection string is your alias connection string as well.
3. When you do a failover in the event of a disaster – your secondary namespace becomes your primary with the same connection string (remember, your alias connection string is same as your current existing primary connection string).
4. This means, when you want to connect back to your old primary (to drain any data or connect for any other reasons), this is not possible with the connection string.
5. Your alternate name now comes into consideration, as this is the only way you can access your old primary.
6. Your primary namespace name in this context remains the same. The connectionstring though will change to reflect the alternate name
7. In addition, after the fail-over your new primary (old secondary) will now have the existing primary connection string and the alias connection string.
8. If you want to pair the same set of namespaces again, this will not be possible to avoid confusions with the connection string

#Step1 - Login to your Azure subscription
az login

#Optional - set your desired subscription
az account set --subscription "your subscription name"

#Step 2 – following parameters are used while creating the resources.

#geo location where the primarynamespace is
export location1=SouthCentralUS

#geo location where the secondarynamespace should be created
export location2=NorthCentralUS

#provide your exisitng resourcegroup which has the primarynamespace
export resourcegroup=myresourcegroup

#provide your existing primary namespace
export primarynamespace=myexistingprimarynamespace
export secondarynamespace=myexistingsecondarynamespace

# here your primarynamespace is your alias name
export aliasname=myexistingprimarynamespace
export alternatename=testehalternatename1

#Step 3 - create your secondary namespace. Copy the ARM ID from the output as you need it later for -PartnerPartnernamespace.
# sample ARM ID looks like this - /subscriptions/your subscriptionid/resourceGroups/$resourcegroup/providers/Microsoft.EventHub/namespaces/secondarynamespace
az eventhubs namespace create --name $secondarynamespace --resource-group $resourcegroup --location $location2 --sku Standard

#Step 4 - Create a geo-dr configuration with primarynamespace name as the alias name and pair the namespaces. Note, you will also provide the alternatename here.
# It is using this alternatename that you will access your old primary once you have triggered the failover
az eventhubs georecovery-alias set --resource-group $resourcegroup --alias $primarynamespace --namespace-name $primarynamespace --partner-namespace "ARM ID of secondary namespace" --alternate-name $alternatename

#Optional - Once your geo-dr configurations are set, you can get the details using this
az eventhubs georecovery-alias show --resource-group $resourcegroup --alias $aliasname --namespace-name $primarynamespace

#Step 5 - you can now do a failover on your secondary namespace with the following command
az eventhubs georecovery-alias fail-over --resource-group $resourcegroup --alias $aliasname --namespace-name $secondarynamespace

#Optional - check you geo-dr configuration details to reflect the fail-over
#Note - your secondarynamespace after the failover will be your new primary, but the connection string remains the same
az eventhubs georecovery-alias show --resource-group $resourcegroup --alias $aliasname --namespace-name $secondarynamespace -- this is your current primary

Create a new Event Hub
 az eventhubs eventhub create --resource-group myresourcegroup --namespace-name mynamespace --name myeventhub --message-retention 4 ---partition-count 15
 
 Create an Authorization Rule
 az eventhubs eventhub authorization-rule create --resource-group myresourcegroup --namespace-name mynamespace --eventhub-name myeventhub --name myauthrule --rights Listen
 
 
 Delete a Consumer Group
 az eventhubs eventhub consumer-group delete --eventhub-name
  --name
  --namespace-name
  --resource-group

• We added dependency changes to support NewtonSoft.Json version 10.0.3
• We also added a bug fix - A lease might have empty offset stored if there are no events on the partitions. This could result in host initialization failures. This has been fixed.

<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-eventhubs</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-eventhubs-eph</artifactId>
<version>1.0.0</version>
</dependency>

1.0.0

• Support user threadPool : All asynchronous tasks are handed off to the ‘Executor’ that is provided while creating the ‘EventHubClient’. This means EventHubs library no longer creates any threads or a threadPool.
• Support for ‘PartitionReceiver’: ‘PartitionReceiver’ is now based out of ‘EventData.SequenceNumber’, you can now use ‘EventPosition.fromSequenceNumber(sequenceNumber)’ to create the ‘PartitionReceiver’ from a sequence number
• ConnectionStringBuilder supports setters for all properties in fluent-style. Supports adding properties to an existing ConnectionString
• PartitionReceiveHandler is changed from abstract class to interface.
• PartitionReceiver.END_OF_STREAM and PartitionReceiver.START_OF_STREAM are replaced by EventPosition.fromEndOfStream() and EventPosition.fromStartOfStream() respectively.

• Previously there was a central loop which periodically scanned the state of all leases and was responsible for taking unowned leases and renewing leases owned by the host instance. The two parts (taking and renewing) have been separated and both are now parallel rather than iterative. This greatly reduces the impact of Azure Storage latency when using the default lease manager.
• All code has been refactored to use CompletableFuture and composition. There are no long-running threads and all sleeps have been replaced with scheduled execution.

Breaking API changes

• EventData constructors are replaced with EventData.create(...) variants
• Factory method to create an instance of EventHubClient from connection string is renamed from fromConnectionString to create.
• Factory method to create an instance of EventHubClient requires additional parameter - Executor.
• Factory methods to create PartitionReceiver instances require a new parameter EventPosition - as an abstraction to the position of the EventData (as opposed to taking offset, dateTime or sequenceNumber).

• All deprecated constructors and methods have been removed
• EventProcessorHost no longer offers constructors, which automatically generate a hostname. This means that all constructors will requires the user to supply a hostname. To generate a unique hostname, we recommend using the static EventProcessorHost.createHostName utility method, which appends the UUID to a user-supplied string.
• If you are using Azure Storage for your lease management and checkpointing, the EventProcessorHost constructor requires a user supplied Storage container name. This container name should be the same for all EventProcessorHost instances operating on the name event hub and consumer group. This shared store is what allows the instances to divide ownership of the event hub partitions amongst them.
• For EventProcessorHost constructors which take a user-supplied threadpool, the type is now changed from ExecutorService to ScheduledExecutorService
• The EventProcessorHost.setAutoExecutorShutdown and forceExectorShutdown methods are removed. An internal threadpool will always be shut down when the EventProcessorHost instance is shut down via unregisterEventProcessor, and a user-supplied threadpool will never be shut down.
• The Client for Java now uses class EventPosition to represent a position in the stream of events. The initial offset provider previously in EventProcessorOptions has been renamed to initial position provider and supports EventPosition.
• The EventProcessorHost.register* methods now return a CompletableFuture which completes when initialization is finished. These methods no longer throw directly. All errors during initialization are signaled by completing the CompletableFuture with an exception
• unregisterEventProcessor is now async and returns a CompletableFuture which completes when the EventProcessorHost instance has finished shutting down. Failures during shutdown are signaled by completing the CompletableFuture with an exception
• The PartitionContext.checkpoint methods are now async and return a CompletableFuture. The checkpoint is not guaranteed to be persisted until the CompletableFuture completes. Failures during checkpointing are signaled by completing the CompletableFuture with an exception
• The ICheckpointManager and ILeaseManager interfaces have been changed to use CompletableFuture instead of Future. This change only affects those implementing checkpoint and lease managers.
• As part of the parallelization effort, the internals were refactored to use CompletableFutures and composition. There are no long-running threads and all sleeps have been replaced by scheduling. If the user is supplying a threadpool, there is no minimum number of available threads required to avoid starvation or deadlock: Event Processor Host can make progress, although slowly, as long as even one thread is available. More threads are recommended, of course, and achieving high performance requires multiple threads

What is next?