Event ID 8000 on all servers?

Article
09/15/2015

Question

_{Tuesday, September 15, 2015 8:29 PM}

Attempting to set up Operational Insights for the first time in our environment and I've successfully connected it via SCOM.

However when I add a computer object via the "Operational Insights Managed", I later check the event log on the server I'm managing and I find a whole bunch of these:

Event ID 8000 Data Publisher Manager: "A subscriber data source in management group XXX has posted items to the workflow but has not received a response in 1000+ minutes. Data will be queued to the disk until a response has been received."

Any idea how I can resolve this issue?

I presume the server is sending the information to SCOM, and then SCOM is uploading the data into OMS in the cloud? Is the connection between the server and SCOM broken somehow?

All replies (5)

_{Monday, September 28, 2015 4:42 PM ✅Answered}

Hi Luke, Darren,

The 8000 events suggest that there is some form of connectivity issue.

There are likely additional errors and warnings in the Event log related to this.

Most data is actually sent directly from the agent to the OMS Service and does not go through the SCOM management group.

Are your agents behind a proxy or a firewall? If so, you may need to configure a proxy:

https://technet.microsoft.com/en-us/library/mt484101.aspx

This page also has details on validating that data is flowing.

Note: When you configure a proxy for the agent, it is an application specific proxy that gets configured which means only the agent is able to send data to the internet, and other services on the box are not able to send data.

Regards

Richard

This posting is provided "AS IS" with no warranties, and confers no rights.

_{Wednesday, September 23, 2015 4:23 AM}

Attempting to set up Operational Insights for the first time in our environment and I've successfully connected it via SCOM.

However when I add a computer object via the "Operational Insights Managed", I later check the event log on the server I'm managing and I find a whole bunch of these:

Event ID 8000 Data Publisher Manager: "A subscriber data source in management group XXX has posted items to the workflow but has not received a response in 1000+ minutes. Data will be queued to the disk until a response has been received."

Any idea how I can resolve this issue?

I presume the server is sending the information to SCOM, and then SCOM is uploading the data into OMS in the cloud? Is the connection between the server and SCOM broken somehow?

We have the same issue. Data is going to the cloud, so I'm perplexed to what is causing it.

* Workflow Id : Microsoft.SystemCenter.CollectHighVolumeDirectChannelEventDataToCloud*

We also have this

The health service HTTP module in management group "xxxxxx_01" timed out connecting to service. Please check configuration settings. *
* Provided configuration: *
* Service Url https://db7ef280-bf22-4379-84a5-2a6d7b55c38b.ods.opinsights.azure.com/EventDataService.svc/PostDataItems. *
* Proxy Provided .

No proxy is configured in SCOM for OMS.

_{Tuesday, September 29, 2015 7:59 PM}

So far we have found the actual upload to OMS is running really slow. No proxy is configured, and data is reaching OMS.

It came about from investigating another issue where the agent would fall behind in processing events, which could be related to this error.

(we hammer our security logs around 250-300 entries per second, so it's a lot of events!)

Checking our firewall logs we can see batches of data sent, and to upload a few MB is can take 5 or more minutes.

Running a traceroute I can see a slowdown outside our network to Microsoft.

If data upload is really slow, could this cause the 8000 events?

I've got an open case now with MS support for our other issue, but I suspect when they resolve the slow connection, it may resolve this 8000 event.

_{Tuesday, September 29, 2015 8:13 PM}

Finally managed to solve this for the most part.

Learnings:

- Each SCOM agent loaded into Operational Insights console actually communicates to Azure directly as well as via the SCOM Management Servers.

- The proxy settings that had been set up were not working correctly (Webmarshal is notorious for failing proxy authentications if the settings aren't quite right).

At the moment I've solved it by authenticating the agents via IP address proxy authentication in Webmarshal rather than User authentication. 8000 events gone ... and a lot more data going into OMS :)

Now I'm looking at the frequent appearance of 21404,21405, and 6028 events in the Operations Manager event logs on the clients.

_{Wednesday, September 30, 2015 3:12 PM}

Luke - glad to hear this seems to be resolved for you. The Event Id 21405 and 6028 can be ignored if they don't list your SCOM management group

Darren - can you add me to the e-mails with Microsoft Support, or have the support engineer contact me? Richard.Rundle@microsoft.com. Our engineers are interested in having a closer look at your log files.