Hi @Dario Bertolino Welcome to Microsoft Q&A forum and thanks for reaching out here.
Here is the MS documentation which explains below steps to achieve your requirement - Install Jupyter Notebook on your computer and connect to Apache Spark on HDInsight
There are four key steps involved in installing Jupyter Notebook locally and connecting to Apache Spark on HDInsight.
- Configure Spark cluster.
- Install Jupyter Notebook.
- Install the PySpark and Spark kernels with the Spark magic.
- Configure Spark magic to access Spark cluster on HDInsight.
Another additional helpful resource but it a bit outdated - Remotely execute a Spark job on an HDInsight cluster
Hope this helps. Let us know how it goes.
Please don’t forget to
Accept Answer and
Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.