Integrating HDInsight Zeppelin Notebook in Azure Data Factory

Question

What's the optimal approach for invoking a Zeppelin notebook from an HDInsight cluster within Azure Data Factory? I have a Zeppelin notebook on HDInsight, and I want to initiate it through Azure Data Factory. Additionally, what's the most efficient method for crafting Spark code and triggering it within Data Factory? I'm seeking guidance on seamlessly integrating these elements for effective data processing.

Answer 1

You can call Databricks notebook using ADF's Databricks Notebook activity, but there is no equivalent activity for Zeppelin notebooks in HDInsight. I believe it is still not possible to invoke Zeppelin notebooks in ADF.

You can call Spark scripts or JARs in HDInsight using ADF's Spark activity, in the HDInsight group in ADF Studio.

Answer 2

(Removed duplicate post -- ignore this)

Answer 3

Hi ClarissaJacquline , Thankyou for using Microsoft Q&A platform and thanks for posting your query here.

Unfortunately , there is no direct way to achieve this requirement as within HDinsights , there is no notebook activity.

However, you can try converting your zeppline notebook code to python code and run it using databricks notebook.

Kindly check, You might find this below documentation helpful: https://github.com/rdblue/jupyter-zeppelin

Hope it helps . Kindly accept the answer by clicking on Accept answer button. Thankyou