HDInsight cluster creation Fails
{ "code": "InvalidNetworkConfigurationErrorCode", "message": "Virtual Network configuration is incompatible with HDInsight requirements. Error: 'Cluster Validation timed out.', Please follow…
Azure HDInsight
Unable to delete HDInsight cluster and resource group.
Unable to delete HDInsight cluster and resource group. I am getting and error and the deletion process keeps running indefinitely.
Azure HDInsight
I want to delete my HDInsight cluster
I cannot ssh into the HD cluster, and Ambari UI is not loading either. The cluster has the status deallocated. I want to delete the cluster and create a new one
Azure HDInsight
How to Disable or Remove Hive Warehouse Connector from HDInsight 5.1
Hello, We are running Spark on HDInsight 5.1 and are not using Hive Metastore at all. However, our Spark application is failing with the following exception: User class threw exception: java.lang.NoClassDefFoundError:…
Azure HDInsight
Azure HDInsight Cluster Not getting Deleted
Hi !!! I have a HDInsight Cluster which I am not able to delete. It is running for more than 30 mins. The name of the cluster is "sayakban-cc-spark-cluster". This is a high priority task as it is consuming my credits from my educational…
Azure HDInsight
HDInsight Spark-type cluster failing to be deployed due to failed start up of SPARK3_THRIFTSERVER service
In Microsoft Azure the Spark-type HDInsight cluster (with 5.1 stack version) is failing to be deployed with an AmbariClusterCreationFailedErrorCode error-code due to an internal error of Spark3 hriftserver service on Ambari: Execute['!…
Azure HDInsight

Creating Azure HDInsight process is stuck at "Status: HDInsight Configuration"
Hey, During the creation of Azure HDinsight (Hadoop Cluster) The process seems to be stuck at "Status: HDInsight Configuration" For more than 20 minutes; after it did manage to somewhat create it (note the dashboard itself is up and running),…
Azure HDInsight
Issue Creating HDInsight Cluster with Virtual Network
I am trying to create an HDInsight cluster using Terraform. However, I encounter the following error when connecting it to a virtual network: │ Error: creating Hadoop HDInsight Cluster (Subscription: "XXX" │ Resource Group Name:…
Azure HDInsight

Sku family standardev3family quota is not enough
I'm attempting to go through this tutorial: https://learn.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-jupyter-spark-sql-use-portal?source=docs I cannot create a HDInsight cluster because of this error that I'm seeing on the Configuration +…
Azure HDInsight
The resource api version is unavailable, how to delete resource
the service corresponding to the resource located under sub has been taken offline or the API version is unavailable, which makes it impossible to delete this resource. How to delete the reources in portal? The VM network and other resources…
Azure HDInsight
In Data Lake Storage Account Gen 2 unexpected reduction of calls API "GetPathStatus" and "ListFilesystemDir"
Hi, I am reaching out with an inquiry regarding the transactions quantity in an data lake storage account gen 2 on Azure. We have noticed that starting on 28th october, the transactions quantity associated with storage account have significantly…
Azure Storage Accounts
Azure HDInsight
Unable to create HDInsight On Demand Cluster in Azure Data Factory
I am getting the issue to creating the On Demand HDInsights in Azure Data Factory. The status is stuck at "PreparingCluster". Here's my LinkedService configuration. OS type : Linux
Azure HDInsight
Azure Data Factory
Compatibility of Azure HDInsight Kafka
Can you provide clarity on whether Azure HDInsight Kafka works with Azure Event Registry? Additionally, I’m interested in any compatibility details or constraints that might apply. Could you share more on this integration?
Azure HDInsight
Azure Data Factory
Difference between synapse and hdinsight
What are major difference between synapse and HD insight and under what circumstances one should use synapse and vice versa
Azure Synapse Analytics
Azure HDInsight
Spark HDInsight cluster deployment fails with an internal server error
I've tried deploying a 5.1.3 HDInsight cluster twice, and both times it fails with an internal server error. JSON below. I chose all the defaults except for using an existing storage account and virtual network. Any suggestions appreciated. { …
Azure HDInsight
Azure HDInsight: Managing Unhealthy Nodes and Sessions
Hi Team, We are currently facing an issue with Unhealthy Nodes on Azure HDInsight. We have one centralized HDInsight cluster shared between Data Teams, where multiple Spark and Hive jobs are being submitted continuously. However, due to either overload…
Azure HDInsight
spark notebooks with terraform
i want to create notebooks using automation for automation i am using terraform, can we invoke spark notebooks in terraform
Azure Synapse Analytics
Azure HDInsight

Stucking in Creating HDinsight Clusters
Hi, My regional quota for vCPUs is 20, but I'm still unable to create the HDInsight clusters. Can anyone help me?
Azure HDInsight
Unable to run Spark activity through data factory pipeline
After configuring the network and ensuring that the test connection between data factory and the respective spark cluster is successful, I am unable to submit nor view any spark jobs that are submitted through the data pipeline using the spark activity.…
Azure HDInsight
Azure Data Factory
Kafka REST proxy endpoint is corrupting and wrongly parsing data
Hello Team, I recently activated REST proxy endpoint for HdInsight kafka. The endpoint is used to produce data to a topic, and upon consuming from the topic I observed that the data is wrongly encoded and parsed. If I try to produce data from the client…
Azure HDInsight
