Not able to limit the size of STDERR and STDOUT logs for Azure HDInsight deployed Spark Application!!!

Goel, Akanksha 66

I have tried editing the log4j properties file in

https://CLUSTERNAME.azurehdinsight.net/#/main/services/SPARK2/configs,

as mentioned in the following link:

https://learn.microsoft.com/en-us/azure/hdinsight/spark/troubleshoot-debug-wasb

I have done the following additions as shown in screenshot:

log4j.appender.audit.triggeringPolicy=org.apache.log4j.rolling.SizeBasedTriggeringPolicy
log4j.appender.audit.triggeringPolicy.maxFileSize=50MB

![89416-image.png ]2
But, the changes are not working, **what can I do to limit the size of logs as you can see the stderr log size is increasing **

PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee

2021-04-20T11:21:11.04+00:00

Hello @Goel, Akanksha ,

Welcome to the Microsoft Q&A platform.

We are reaching out to the internal team to get more help on this, I will update you once we hear back from them.
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee

2021-04-22T06:49:14.663+00:00

Hello @Goel, Akanksha ,

We are still awaiting for the response from the product team. I will update you once we hear back from them.

Thank you for your patience.
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee

2021-04-27T10:44:21.183+00:00

Hello @Goel, Akanksha ,

Apologize for the delay in response.

Unfortunately, I haven't got any response from the product team.

For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket, else please do let us know.
PRADEEPCHEEKATLA-MSFT 77,081 Reputation points Microsoft Employee

2021-05-04T04:24:45.77+00:00

Hello @Goel, Akanksha ,

Did you get a chance to open a support ticket on this issue?