Data Lake Central alerting

Question

Hi Guys

We are planning to build centralize alert system on Azure

Source systems
Splunk
Solarwinds
SAP Alert Management
Oracle Alert log
Talend Job Logs
Cisco Information Service
Azure Logs

what would the easy way to gather all the logs? [Event hubs, Stream analytics,Kafaka etc to gather near real time logs, what connectors to look for to fetch data]

How to send Message if there is any failure log in central system?

Regards,
Navin

Answer 1

I'm not familiar with all the source systems you've mentioned but Azure Event Hubs is a good service to gather all the logs into one location. If the sources already support exporting logs to Azure Event Hubs, it will make things a lot simpler.

For others, you could ingest logs using Azure Functions or Azure Logic Apps (if a connector exists).

As for processing the logs, either Azure Functions or Azure Logic Apps would do depending on how complex the processing is and the same could be used for sending out alerts as required.

For Logic Apps, you could use the Outlook 365 connector to send out an email for example.