Azure HDInsight Spark job is failing with Logger Error

Aishwarya Gopikrishnan 1

Hello Team,

Our jobs are recently failing with this error- ERROR RawSocketSender [MdsLoggerSenderThread]: org.fluentd.logger.sender.RawSocketSender
java.net.SocketException: Broken pipe (Write failed)
All these pyspark jobs were running fine until recently and suddenly they are failing with this error. Also we didnt make any kind of code changes. Please help me in understanding the error

MartinJaffer-MSFT 26,036 Reputation points

2022-09-30T01:25:32.33+00:00

Hello and welcome to Microsoft Q&A @Aishwarya Gopikrishnan

A broken pipe is a very deep level error. Deeper than I can help with here. I'd like to guide you to support services where they can take a deep investigation into your logs.

If you do not have a support plan, I can give you a 1 time free support ticket. Let me know if you need that. Directions on that to follow in private message.
MartinJaffer-MSFT 26,036 Reputation points

2022-10-04T16:32:49.827+00:00

Hello @Aishwarya Gopikrishnan ,
We haven’t received any email from you as requested in the last response and was just checking back to see if you have a resolution. In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, please respond back with the more details as requested and we will try to help .

Thanks
Martin