Azure AD and Azure Subscription Log Streaming to ArcSight

Question

Hi Guys

We’re trying to setup centralized security event log management for logs from Azure AD, Azure ARM subscription and Security Center. So, the following logs will be captured:

1. Azure Active Directory logs

a. Sign-in Logs
b. Audit Logs

2. Azure ARM subscription:

a. Activity logs: Provides data related to write operations that were performed on each Azure resource in the subscription from the outside (the management plane), for example creating a new resource or starting a virtual machine. This is information about the what, who, and when for any write operations (PUT, POST, DELETE) taken on the resources in your subscription.

b. Diagnostic logs / resource logs: Provides data related to operations performed within an Azure resource (the data plane), for example getting a secret from a Key Vault or making a request to a database. The content and structure of resource logs varies by the Azure service and resource type.

3. Azure Security Center:

a. Security alerts: Provides data related to security actions performed on Azure Security Center in a subscription.

b. Recommendations: Provides data related to prevention recommendations provided for the resources in a subscription.

The on-prem solution is ArcSight SIEM, so we have a plan to route all the events to an Event Hub instance per subscription (we have two subscriptions in a typical Hub-Spoke model) and configure ArcSight Syslog NG Daemon SmartConnector to grab these events and push them into on-prem ArcSight SIEM solution:

Have reviewed the following documents:

Stream Azure monitoring data to an event hub or external partner https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/stream-monitoring-data-event-hubs

Tutorial: Stream Azure Active Directory logs to an Azure event hub https://learn.microsoft.com/en-us/azure/active-directory/reports-monitoring/tutorial-azure-monitor-stream-logs-to-event-hub

Integrate Azure Active Directory logs with ArcSight using Azure Monitor https://learn.microsoft.com/en-us/azure/active-directory/reports-monitoring/howto-integrate-activity-logs-with-arcsight

Continuously export Security Center data https://learn.microsoft.com/en-us/azure/security-center/continuous-export?tabs=azure-portal

However, got some very fundamental questions:

The documentation here (https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/monitor-azure-resource) says that resource logs are generated automatically, but not collected without a Diagnostic setting. Create a diagnostic setting to send entries to Azure Monitor Logs:

1. Firstly, does that mean we need to enable Diagnostic Setting for each resource deployed in our subscription that we want to monitor and send the logs to the Event Hub OR this can simply be enabled universally by going to Azure Monitor--> Activity log--> Diagnostic Setting? as shown below?

In other words, do we need to create Diagnostic setting for each resource and directly route logs through these setting to the single Event Hub instance that’s integrated with ArcSight Syslog NG Daemon SmartConnector? OR there’s a single subscription wide diagnostic setting (somewhere in Azure Monitor) that we can point to Event Hub and it will automatically send these logs for all the resources (regardless of whether resource-level diagnostic settings are enabled or not)?

2. When we enable Diagnostic setting for a resource, we only get the above 4 options, there’s nothing called “Azure Monitor Logs” as stated in:

Create a diagnostic setting to send entries to Azure Monitor Logs

1. Since we have two Azure subscriptions, does that mean we need to perform all the Azure side (Event Hub namespace, Event Hub, and Azure Monitor) and the on-prem side (ArcSight connector) needs to be done twice – one for each subscription separately?

Is there a better architecture that’s easy to scale and manage when we add more subscriptions to the equation?

4. Is it good to create separate Event Hubs in Event Hub Namespace – one for ArcSight and another for Splunk OR a single Event Hub with separate Consumer Groups – one for ArcSight and another for Splunk?

Thanks
Taranjeet Singh

Answer 1

Trying to answer as many questions as possible:

• There isn't a way of enabling resource diagnostic logs for every resource in a centralized manner. You have to enable it for each resource. Of course, you can use a deployIfNotExists Policy initiative that will ensure that diagnostic logs are turned on for every new resource, provided the initiative covers all your resource types (one policy definition per resource type).
• Azure Monitor--> Activity log--> Diagnostic Setting applies only to the Activity Log and is enabled once per each subscription.
• You don't need to have an Event Hub per subscription. You could/should use a centralized Event Hub (e.g., in the hub subscription) where you point all your diagnostic settings to. It's a more manageable approach.

Answer 2

Hey @Helder Pinto

Thanks for your responses. Can you please share some guidance or some article that does on how a centralized Evet Hub can be used ?

Appreciate the support extended.

Taranjeet Singh