processing different message types in stream analytics

Pálmi Símonarson 21 Reputation points
2022-10-31T08:43:29.923+00:00

An event stream has a few message types, all json, very different schemas, the system that produces the events provides an AMQP interface The idea is to use event hubs and stream analytics to capture the events to an sql database. Can a single stream analytics job process mutliple message types? If not what would be the alternative?

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
555 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
330 questions
{count} votes

Accepted answer
  1. Wilko van de Velde 2,146 Reputation points
    2022-11-01T06:47:49.69+00:00

    Hi,

    You can create multiple outputs in Stream Analytics. So if you can identify the different message types than split them.
    For example:

    WITH TMP_Input as(  
        SELECT  
            *   
        FROM  
            [Input]  
        )  
          
      
    SELECT   
            *   
    INTO          
            [Output1]   
    FROM  
            [TMP_Input]  
    WHERE  
            Type = 'Type1'  
      
          
      
    SELECT   
            *   
    INTO          
            [Output2]   
    FROM  
            [TMP_Input]  
    WHERE  
            Type = 'Type2'  
    

0 additional answers

Sort by: Most helpful