Create a Kafka connection

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

Create a Kafka connection in Catalog Explorer to store authentication credentials for Lakeflow Connect ingestion. Any user with the USE CONNECTION privilege on the connection can then create Kafka ingestion pipelines without needing direct access to the credentials.

For privilege requirements, see Connect to managed ingestion sources.

Create a connection

1. In the Azure Databricks workspace, click Catalog > Create > Create a connection.

2. On the Connection basics page of the Set up connection wizard, specify a unique Connection name.

3. In the Connection type drop-down menu, select Kafka.

4. In the Auth type drop-down menu, select either Username and Password or Service Credential. For instructions, see

   Authentication for Kafka.

5. (Optional) Add a comment.

6. (Username and Password) Enter the user identity and password of the Kafka cluster.

7. (Service Credential) Select your service credential from the drop-down or click Create new service credential. Enter the bootstrap servers of the Kafka cluster.

8. (Optional) Enter the URL, API key, and API secret for your schema registry.

9. Click Create connection.

Next steps

Create an ingestion pipeline.