Streaming ETL with eventhub

Tommy Wang 1 Reputation point


We have SAP and Hybris hosted onprem, which are the book of record for product and promo. The goal is to build a ETL solution to send to partner systems on products on promotion (effectively joining the feeds from both systems), the technical objective is to leverag azure cloud to build something scalable and reusable.

Shown in the diagram, I am thinking to use Azure Event Hub to build a steaming ETL, with SAP to replicate the product catalog to SQL Azure then use asa job to filter out by the Hybris promo events. However, with the size of SAP product, it does not seems to be a better fit.

Please help share some thoughts or best practices for this use case. Thanks

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
380 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
235 questions
{count} votes

1 answer

Sort by: Newest
  1. Tommy Wang 1 Reputation point

    thanks for the update, at this point only use of event hub, i have no go ahead for the proposed alternatives.

    what if i subscribe the function directly to the event hub then do a query to the on prem database while process the event directly from there, is this anti pattern?