Dynamic Paramter Cluster option

Question

Hi

How can we parametrize above option in Azure Databricks Linked Services in ADF

Answer 1

Hello @Kuldeep Chitrakar ,

Thanks for the question and using MS Q&A platform.

As per my understanding you would like to make the Work options (selection between Fixed vs Autoscaling) parameterize and dynamically select at the runtime. Please correct me if my understanding is not accurate.

If my understanding is correct, then the answer to your question is that you cannot parameterize selection of Work options ( Fixed vs Autoscaling).

Workaround: But as an alternative you may have to have 2 Databricks linked services one with Work options ( Fixed) and the other with Work options (Autoscaling). Then design your pipeline in such a way that based on the criteria/requirement you will do a precheck to decide which Work options to be executed at that runtime using If condition activity. Then inside If condition activity you will have 2 paths If True and Else. You will have to have the same flow in both the paths (True/False) but referencing to unique (one choosing Work options ( Fixed) and the other path having Linked service with Work options ( Autoscaling)) Databricks linked service in each path.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how