Active worker nodes usage in HDinsight cluster for the period of last one year


Hi Team,

Can someone help us to get the Active worker nodes usage in HDinsight cluster for the period of last one year.

Thanks & Regards,

Azure Metrics Advisor
Azure Metrics Advisor
An Azure artificial intelligence analytics service that proactively monitors metrics and diagnoses issues.
68 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
170 questions
{count} votes

1 answer

Sort by: Most helpful
  1. MartinJaffer-MSFT 25,371 Reputation points Microsoft Employee

    Hello and welcome to Microsoft Q&A @UDAYA SRINIVASARAO KOTHAMASU .

    If I understand correctly, you want to get historical data for active nodes. With the appropriate preparations this would be easy to do.

    Have you configured Azure Monitor to save the logs from your HDInsight cluster, or set up the Diagnostic Settings to save your logs? If so, this should be easy to do. If not, I am uncertain whether the entire year's data can be retrieved. Generally, there is a limit to historical data.

    Some data can be retrieved from the portal without setting up Diagnostic Settings or Azure Monitor:

    Under Monitoring: Metrics, the number of active worker nodes is available. This data extends to the past 30 days.

    When autoscale is enabled, the scale-up, scale-down is recorded in the activity log. This records the number of nodes. The activity logs can be downloaded/exported.