Event 4503: A module reported an error 0x80FF0004 from a callback Microsoft.Linux.RHEL.7.Process.Audit.Monitor

Question

On a gatewayserver which monitors about 25 linux servers i constantly get 4503 events on 2 linux agents, various workflows.
The workflows are out of the box from the redhat 7 MP, like the one above which seems to monitor if auditd is running.

Searching on this error i found not much, some mentioning the account or distribution which i tested and seem to work fine. I can run tasks like run vmstat from the scom console on the computer, both computers are healthy in scom so heartbeats seems to work fine.

I've exported the pack and it seems that this wsman command is run by the workflow:
winrm e http://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_UnixProcess?__cimnamespace=root/scx

I ran this manually on the gateway and it does give output, an xml of over 8000 lines. The managementpack contains a select statement for auditd, i'm not sure if this is done in the winrm command or after the feedback is received.
The error indicates to me that the command was ok but someting in the output (is that the callback?) is not correct.

any idea of what this can be?

Answer 1

The select is done after the full xml is received.
At this point I would try running a workflow trace, that might provide you with some useful details : https://monitoringguys.com/2020/12/15/tracing-scom-workflows-with-powershell/

Answer 2

Hi,

Since we suspect it is caused by the output from the 2 problematic computers, is there any chance for us to check if there any difference with the working ones?

Alex
If the response is helpful, please click "Accept Answer" and upvote it.

Answer 3

Thanks both for your answers.
The tracing script looks very good but did not give me anything useful. The script it'self doesnt work at all after you download it but after a minor adjustment it did. I've created the special override and used the specific method to trace, waited till the event was created but noting usefull in the logs.
I also used the old way of tracing by creating the empty file EnableOpsMgrModuleLogging and also could not find anything useful.

I also compared the output with an agent that does not give this error.

The specific part of xml of the process auditd looks simular, only the paths differ but that should not be a problem i guess.
bad server = <p:ModulePath>/usr/sbin/auditd</p:ModulePath>
good server = <p:ModulePath>/sbin/auditd</p:ModulePath>

I do note that the output of the good server contains about 25000 lines of xml, the failing server contains over 50000 lines, so maybe that is the issue.

I will open a supportcase for this to troubleshoot further.