Data Lake Central alerting

Dondapati, Navin 281 Reputation points
2020-12-17T21:42:13.313+00:00

Hi Guys

We are planning to build centralize alert system on Azure

Source systems
Splunk
Solarwinds
SAP Alert Management
Oracle Alert log
Talend Job Logs
Cisco Information Service
Azure Logs

what would the easy way to gather all the logs? [Event hubs, Stream analytics,Kafaka etc to gather near real time logs, what connectors to look for to fetch data]

How to send Message if there is any failure log in central system?

Regards,
Navin

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,359 questions
Azure Logic Apps
Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
2,873 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
564 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
333 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pramod Valavala 20,591 Reputation points Microsoft Employee
    2020-12-18T11:56:52.213+00:00

    I'm not familiar with all the source systems you've mentioned but Azure Event Hubs is a good service to gather all the logs into one location. If the sources already support exporting logs to Azure Event Hubs, it will make things a lot simpler.

    For others, you could ingest logs using Azure Functions or Azure Logic Apps (if a connector exists).

    As for processing the logs, either Azure Functions or Azure Logic Apps would do depending on how complex the processing is and the same could be used for sending out alerts as required.

    For Logic Apps, you could use the Outlook 365 connector to send out an email for example.

    0 comments