send windows OS logs to Event Hub

Question

send windows OS logs to Event Hub

Jezo, Libor 20

I would like to see more details than the information on this page: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/diagnostics-extension-stream-event-hubs

Everyone refers to this page but to really follow the direction from the link you will end up with many questions that are not asnwered anywhere I could find.

json files are well described but the missing information is: where do I upload those json files? where do they reside? what should I do with the json files? Please spell it out so it is clear.
what do I need to do inside the virtual machine to send the event logs to the Event Hub?
it would help to see, step by step how to configure it and verify that the Event Hub receives the operating level logs from virtual machines. There are steps that I followed how to enable the "diagnostics for a VM" but that in itself did not work. A video would be helpful too.
My goal is to send the existing virtual machine's logs, Windows server logs, to Splunk. I was able to configure the Event Hub Namespace and one hub. I even see the portal level logs in the Splunk but the operation system logs, windows logs I dont see even though I enabled the 'diagnostics' on a virtual machine. thank you for your help

Answer 1

Limitless Technology 15,101

Hello there,

The path for your Windows Terminal settings.json file may be found in one of the following directories:

Terminal (stable / general release): %LOCALAPPDATA%\Packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState\settings.json

Terminal (preview release): %LOCALAPPDATA%\Packages\Microsoft.WindowsTerminalPreview_8wekyb3d8bbwe\LocalState\settings.json

Terminal (unpackaged: Scoop, Chocolately, etc): %LOCALAPPDATA%\Microsoft\Windows Terminal\settings.json

The easiest way to accomplish what you're trying is to install the Azure Diagnostics Extension on the VM and configure it to output log data to an Event Hub sink.

Azure diagnostic logs can be streamed in near real-time to any application using the built-in “Export to Event Hubs” option in the Portal, or by enabling the Event Hub Authorization Rule ID in a diagnostic setting via the Azure PowerShell Cmdlets or Azure CLI.

After data is displayed in the event hub, you can access and read the data in two ways:

Configure a supported SIEM tool. To read data from the event hub, most tools require the event hub connection string and certain permissions to your Azure subscription.

This article provides a brief description of how to stream data and then lists some of the partners with whom you can send it. Some partners have special integration with Azure Monitor and might be hosted on Azure. https://learn.microsoft.com/en-us/azure/azure-monitor/agents/diagnostics-extension-stream-event-hubs

Hope this resolves your Query !!

--If the reply is helpful, please Upvote and Accept it as an answer–

Jezo, Libor 20 Reputation points

2023-03-03T17:56:43.64+00:00
thank you for your reply.

I ran these commands:

az vm extension set \ --resource-group myResourceGroup \ --vm-name myVM \ --name IaaSDiagnostics \ --publisher Microsoft.Azure.Diagnostics \ --protected-settings protected-settings.json \ --settings public-settings.json

I followed through and was able to send the configuration as instructed.

Then I followed the json conifguration from this page https://learn.microsoft.com/en-us/azure/azure-monitor/agents/diagnostics-extension-stream-event-hubs and I updated the configuration with the information about my Event Hub.

Now, how do I verify that Event Hub received the OS level logs?

Btw, I already have Splunk receiving data from the Event Hub but not the OS level logs.

Is there a need to specify some special permission for the API that is used for the Splunk in connection with the OS level logs? For example I already see in the splunk the activity about me updating the configuration

thank you for your help

Jezo, Libor 20

an update: I wa able to receive the logs in the Splunk. so the above configuration worked. I have another question if you can help me with.

the sample public settings.json is somehow limited to the level of logs I would like to see in Splunk.

This is the json. Do you know about a guide how to specify "IIS Logs" SharePoint Logs that are in a specific directory etc?

thank you again for your help.

"DiagnosticMonitorConfiguration": {
      "overallQuotaInMB": 5120,
      "PerformanceCounters": {
        "scheduledTransferPeriod": "PT1M",
        "PerformanceCounterConfiguration": [
          {
            "counterSpecifier": "\\Processor Information(_Total)\\% Processor Time",
            "unit": "Percent",
            "sampleRate": "PT60S"
          }
        ]
      },
      "WindowsEventLog": {
        "scheduledTransferPeriod": "PT1M",
        "DataSource": [
          {
            "name": "Application!*[System[(Level=1 or Level=2 or Level=3)]]"
          }
        ]
      }
    }

send windows OS logs to Event Hub

0 additional answers

Activity