Share via

Azure Databricks autoscale disk setting

Ken Kasai a 101 Reputation points
2021-06-29T05:03:28.24+00:00

I have a question about Databricks in Azure.

The "Autoscaling local storage" link below states that the virtual machine's Disk size can grow up to 5TB.
https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#---autoscaling-local-storage

When we tried it, it didn't autoscale and stayed at 250GB.
Would you please tell me if you know how to configure it?

Thanks

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.


Answer accepted by question author

  1. Ken Kasai a 101 Reputation points
    2021-07-06T03:01:17.28+00:00

    Share the answer.

    Autoscaling works only on local disks.

    This means that Autoscaling is working with "local_disk0" disk usage.
    Therefore, even if you operate a file on another Disk area and tighten the area, automatic scaling will not work.

    Thank you

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.