Can Azure Streaming Analytics read from Kafka on HDInsight and write to Deltalake table on Synapse lake.

sql-seek 61 Reputation points
2022-06-01T21:16:30.287+00:00

Hello

I am looking for guidance on building a new event driven platform. The options we are exploring for processing are -

  1. Azure Stream Analytics
  2. Apache Spark Structured Streaming in Synapse

Source is like going to be Kafka on HDInsight
Destination is going to be Deltalake table on ADLS (Synapse)

Thank you!

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,395 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
199 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
331 questions
0 comments
Accepted answer
  1. Sid Ramadoss 161 Reputation points
    2022-06-02T20:17:38.097+00:00

    Hi,
    We don't yet support Kafka input or writing to delta lake output but have it on the roadmap. Could you please email askasa@microsoft.com so that we can understand your use case better?

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest