Add custom endpoint source to an eventstream

If you want to connect your own application with an eventstream, you can add a custom endpoint (i.e., Custom App in standard capability) source. Then you can send real-time events to the eventstream from your own application with the connection endpoint exposed on the custom endpoint (i.e., Custom App in standard capability). Furthermore, with the Apache Kafka protocol available as an option for custom endpoints (i.e., Custom App in standard capability), you can send real-time events using the Apache Kafka protocol. This article shows you how to add a custom endpoint (i.e., Custom App in standard capability) source to an eventstream.

Note

If you want to use enhanced capabilities that are in preview, select Enhanced Capabilities at the top. Otherwise, select Standard Capabilities. For information about enhanced capabilities that are in preview, see Introduction to Fabric event streams.

Prerequisites

• Access to the Fabric premium workspace with Contributor or higher permissions.

Important

Enhanced capabilities of Fabric event streams are currently in preview.

Note

The maximum number of sources and destinations for one eventstream is 11.

Add custom endpoint data as a source

Follow these steps to add a custom endpoint source:

1. To create a new eventstream, select Eventstream from the Home screen. Make sure the Enhanced Capabilities (preview) option is enabled.

   

2. To add custom endpoint source, on the get-started page, select Use custom endpoint.

   

   Or, if you already have a published eventstream and want to add custom endpoint data as a source, switch to Edit mode. Then select Add source in the ribbon, and select Custom endpoint.

   

3. On the Custom endpoint screen, enter a name for the custom source under Source name, and then select Add.

   

4. After you create the custom endpoint source, you see it added to your eventstream on the canvas in Edit mode. To implement this newly added custom app source data, select Publish.

   

Once you complete these steps, the custom endpoint data is available for visualization in Live view.

Get endpoint details in Details pane to send events

The Details pane has three protocol tabs: Event Hub, AMQP, and Kafka. Each protocol tab has three pages: Basics, Keys, and Sample code which offer the endpoint details with the corresponding protocol for connecting.

Basic shows the name, description, type, and status of your custom endpoint.

Keys and Sample code pages provide you with the connection keys information and the sample code with the corresponding keys embedded that you can use to stream the events to your eventstream. The Keys and Sample code information varies by protocol.

Event hub

The Keys in the Event hub protocol format contain information related to an event hub connection string, including the Event hub name, Shared access key name, Primary key, and Connection string-primary key. The Event hub format is the default for the connection string and works with Azure Event Hubs SDK. This format allows you to connect to your eventstream via the Event Hubs protocol. The following example shows what the connection string looks like in Event hub format:

Endpoint=sb://eventstream-xxxxxxxx.servicebus.windows.net/;SharedAccessKeyName=key_xxxxxxxx;SharedAccessKey=xxxxxxxx;EntityPath=es_xxxxxxx

The Sample code page in Event Hubs tab offers ready-to-use code with the required connection keys information in Event hub included. Simply copy and paste it into your application for use.

Kafka

The Kafka format is compatible with the Apache Kafka protocol, which is a popular distributed streaming platform that supports high-throughput and low-latency data processing. You can use the Keys and Sample code in Kafka protocol format to connect to your eventstream and stream the events.

Likewise, the Sample code page in Kafka tab provides you with ready-made code, including the necessary connection keys in Kafka format. Simply copy it for your use.

AMQP

The AMQP format is compatible with the AMQP 1.0 protocol, which is a standard messaging protocol that supports interoperability between different platforms and languages. You can use this format to connect to your eventstream using the AMQP protocol.

The Sample code page in AMQP tab also provides you with the ready-to-use code with connection keys information in AMQP format.

You can choose the protocol format that suits your application needs and preferences and copy and paste the connection string into your application. You can also refer to or copy the sample code that we provide in the Sample code tab, which shows how to send or receive events using different protocols.

Related content 

To learn how to add other sources to an eventstream, see the following articles:

• Amazon Kinesis Data Streams
• Azure Blob Storage events
• Azure Cosmos DB
• Azure Event Hubs
• Azure IoT Hub
• Azure SQL Database Change Data Capture (CDC)
• Confluent Kafka
• Fabric workspace event 
• Google Cloud Pub/Sub
• MySQL Database CDC
• PostgreSQL Database CDC
• Sample data

Prerequisites

Before you start, you must complete the following prerequisites:

• Get access to a premium workspace with Contributor or above permissions where your eventstream is located.

Note

The maximum number of sources and destinations for one eventstream is 11.

Add a custom app as a source

If you want to connect your own application with an eventstream, you can add a custom app source. Then, send data to the eventstream with your own application with the connection endpoint exposed in the custom app. Follow these steps to add a custom app source:

1. Select New source on the ribbon or "+" in the main editor canvas and then Custom App.

2. Enter a Source name for the custom app and select Add.

   

3. After you have successfully created the custom application source, you can switch and view the following information in the Details tab in the lower pane:

   

Get endpoint details in Details pane to send events

The Details pane has three protocol tabs: Event Hub, AMQP, and Kafka. Each protocol tab has three pages: Basics, Keys, and Sample code which offer the endpoint details with the corresponding protocol for connecting.

Basic shows the name, description, type, and status of your custom app.

Keys and Sample code pages provide you with the connection keys information and the sample code with the corresponding keys embedded that you can use to stream the events to your eventstream. The Keys and Sample code information varies by protocol.

Event hub

The Keys in the Event hub protocol format contain information related to an event hub connection string, including the Event hub name, Shared access key name, Primary key, and Connection string-primary key. The Event hub format is the default for the connection string and works with Azure Event Hubs SDK. This format allows you to connect to your eventstream via the Event Hubs protocol. The following example shows what the connection string looks like in Event hub format:

Endpoint=sb://eventstream-xxxxxxxx.servicebus.windows.net/;SharedAccessKeyName=key_xxxxxxxx;SharedAccessKey=xxxxxxxx;EntityPath=es_xxxxxxx

The Sample code page in Event Hubs tab offers ready-to-use code with the required connection keys information in Event hub included. Simply copy and paste it into your application for use.

Kafka

The Kafka format is compatible with the Apache Kafka protocol, which is a popular distributed streaming platform that supports high-throughput and low-latency data processing. You can use the Keys and Sample code in Kafka protocol format to connect to your eventstream and stream the events.

Likewise, the Sample code page in Kafka tab provides you with ready-made code, including the necessary connection keys in Kafka format. Simply copy it for your use.

AMQP

The AMQP format is compatible with the AMQP 1.0 protocol, which is a standard messaging protocol that supports interoperability between different platforms and languages. You can use this format to connect to your eventstream using the AMQP protocol.

The Sample code page in AMQP tab also provides you with the ready-to-use code with connection keys information in AMQP format.

You can choose the protocol format that suits your application needs and preferences and copy and paste the connection string into your application. You can also refer to or copy the sample code that we provide in the Sample code tab, which shows how to send or receive events using different protocols.

Related content 

To learn how to add other sources to an eventstream, see the following articles:

• Azure Event Hubs
• Azure IoT Hub
• Sample data