Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,514 questions
How can I clear the cache in my Databricks Cluster
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 16%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERH%3C/text%3E%3C/svg%3E)
richardthomas66@hotmail.com
0
Reputation points
Ive tried many articles and links without success.
Has anyone managed to successfully clear the cache on their Databricks cluster? This causes many issues as my memory is near its peak, meaning my notebooks often crashes. Any help or advised would be much appreciated.
Azure Databricks
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator2024-06-18T17:01:11.2466667+00:00 Hello ******@hotmail.com,
Have you tried
spark.catalog.clearCache()method? This will clear the cached data in Spark. This method removes all cached data from memory and disk.You can also see the below threads:
I hope this helps.
-
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator
2024-06-19T19:35:52.0033333+00:00 Hello ******@hotmail.com,
I am checking to see if you have any further questions here.
Sign in to comment
Sign in to answer