How to implement CI/CD on HDInsight ?

Federico Sardo 91 Reputation points
2023-07-26T15:51:14.8133333+00:00

Hi,

I need recommendations about how to apply CI/CD in HDinsight.

I want to create "dev" and "prod" cluster. One Spark and one Hive IQ each. Let's imagine:

my-cluster-spark-dev

my-cluster-hiveiq-dev

my-cluster-spark-prod

my-cluster-hiveiq-prod

Suppose that I am testing a flow and for that I created a hive table in dev. After running some test, everything goes well, and I need to deploy the create table script in the hive production cluster.

How can I achieve that? We are totally allowed to use Azure Devops.

Thanks,

Fede

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
214 questions
{count} vote

1 answer

Sort by: Most helpful
  1. Federico Sardo 91 Reputation points
    2023-08-09T12:24:53.9233333+00:00

    Hi SatishBoddu-MSFT

    I think that you pointed me to a document about how to create a cluster with Ifra as code. (We allready did that).

    My question was related about how can I move , for instance, a pyspark script, or a hive DDL script from dev cluster to prod cluster.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.