Can Azure Streaming Analytics read from Kafka on HDInsight and write to Deltalake table on Synapse lake.

sql-seek 61 Reputation points
2022-06-01T21:16:30.287+00:00

Hello

I am looking for guidance on building a new event driven platform. The options we are exploring for processing are -

  1. Azure Stream Analytics
  2. Apache Spark Structured Streaming in Synapse

Source is like going to be Kafka on HDInsight
Destination is going to be Deltalake table on ADLS (Synapse)

Thank you!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,003 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
215 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
361 questions
0 comments No comments
{count} votes

Accepted answer
  1. Sid Ramadoss 161 Reputation points
    2022-06-02T20:17:38.097+00:00

    Hi,
    We don't yet support Kafka input or writing to delta lake output but have it on the roadmap. Could you please email askasa@microsoft.com so that we can understand your use case better?


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.