Databricks costs

Question

Databricks costs

Ales Ventus 46

I have tags for databricks job and databricks cluster assigned to the job. I can see costs based on these tags in azure cost analysis.

I want to know if in azure cost analysis is cost for cluster included in the job cost.

job cost = 0.80$

cluster cost = 0.14$

Total for process is 0.80$ or 0.94$ ?

Thanks.

1 answer

Your answer

Answer 1

Bhargava-MSFT 31,361 Microsoft Employee Moderator

Hello Admin - Ales Ventus,

Welcome to the Microsoft Q&A forum.

There is no direct official document given the variability in costs, but from the below blog post and other documents, my understanding is the cost for a Databricks job and the cost for a Databricks cluster are separate.

The cost for the Databricks job is based on the number of job runs and the duration of each run, while the cost for the Databricks cluster is based on the size and duration of the cluster.

In your example, if the cost of the Databricks job is $0.80 and the cost of the Databricks cluster is $0.14, then the total cost for the process would be $0.94.

https://medium.com/microsoftazure/understanding-azure-databricks-costs-using-azure-cost-management-for-observability-and-chargebacks-86e7911fb989

https://learn.microsoft.com/en-us/azure/databricks/lakehouse-architecture/cost-optimization/best-practices

https://learn.microsoft.com/en-us/answers/questions/1188051/azure-databricks-how-to-calculate-the-cost-for-a-j

I hope this helps. Please let me know if you have any further questions.

If this answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.

Ales Ventus 46 Reputation points

2024-01-09T07:55:14.93+00:00

Thanks for reply. Actually, when I analyzed other jobs/cluster costs, I realized that cost of the job is equal to cost of the cluster. So, I would say, that cost of the job is equal to cluster cost. What other costs could be there except of cluster costs? Ok some other side costs are related to storage and virtual network, but these are external.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-01-09T20:46:16.0433333+00:00

Hello Ales Ventus,

Here are the more accurate details after further discussion with my internal team.

The job cost is VM + DBU. These two are a little complicated to calculate if the cluster scales up and down. You can also have different discount rates for VMs and DBU. VM depends on your Azure discount or any reservations you may have, or if you are using Spot. DBU depends on your EA discount or if you have a Databricks commit, as well as the type of cluster you are using, jobs or otherwise

A simplified forumula to calcuate Job Costs = VM_costjob_durationVM_count + VM_DBU_Costjob_durationVM_Count

Please see this github document for more deatis.

so the total cost of a Databricks job and cluster, you would need to add the cost of the cluster to the job cost. The cost of the cluster would depend on the number and type of virtual machines used by the cluster, as well as any associated costs such as storage and virtual network.

I hope this helps.

If this answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.
Bhargava-MSFT 31,361 Reputation points Microsoft Employee Moderator

2024-01-17T16:23:22.77+00:00

Hello Ales Ventus, I am checking to see if you have any further questions here. If this answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.

Share via

Databricks costs

1 answer

Your answer