creating conda environment on azure hdinsight spark cluster taking hours

kunal kapoor 1 Reputation point
2020-09-10T14:14:09.1+00:00

trying to follow the code in this tutorial , https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-in-spark/train-in-spark.ipynb

when i send the run to run to azure ml, the run essentially stalls on the step creating the conda environment in the hdinsight cluster I am looking to run the pyspark code on.

I am wondering if anyone has run into the same problem where creating a conda environment on a hdinsight cluster for the first time is taking hours, already been 2 hours for me.

Thanks

23854-image.png

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,571 questions
Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
199 questions
{count} votes