Calling databricks Python notebook in Azure function.

Sanju 21 Reputation points
2022-06-17T19:16:07.767+00:00

Hello,

I have a python databricks notebook(pyspark) which does an aggregation based on the inputs provided to the notebook via parameters.

  1. Is it possible to run this notebook from the Azure function app.
  2. Can we pass the parameters to the notebook from the Azure function. If so, kindly let me know the approach.

Thank you.

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,214 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
2,607 questions
No comments
{count} votes

Accepted answer
  1. Vidhya Sagar Karthikeyan 391 Reputation points
    2022-06-20T15:59:28.95+00:00

    Just wanted to summarize,

    Databricks has an API which can perform many actions within the data bricks, saying that you can use the same API to trigger a job. You can create a job which executes the notebook, and then you can invoke the job via databricks API from Azure functions and also pass required parameters to it. Refer to the documentation https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/jobs for more details about using the job API.

    Mark this as an answer if it helps you.

    No comments

1 additional answer

Sort by: Most helpful
  1. Samy Abdul 3,001 Reputation points
    2022-06-19T16:31:48.193+00:00
    No comments