How to use UA Managed Identity in Data factory On Demand HD Insight Linked Service

Question

When creating an on-demand HD Insight linked service, there's missing detail for how to configure a User Assigned managed identity instead of a service principal. Steps are shown on how to add a UA managed identity to the Data Factory, but what values should be populated in the linked service configuration. Here https://learn.microsoft.com/en-us/azure/data-factory/tutorial-transform-data-spark-portal?source=recommendations, we should skip the entry for service principal key. Do we put the UA managed identity's object principal key in the service principal id field? Then, we select Azure Key vault. I did create a key vault linked service and selected that, but what about the secret field? What do we put in there?

Any high definition insight would be greatly appreciated. ;)

Answer 1

Hello @Scoot-3223 ,

Welcome to the MS Q&A platform.

Please correct me if my understanding is wrong. You want to know how to use "User managed Identity" on the Azure HD insights linked service connection.

You can create HD Insights linked service connection with either Service principal key or Azure Key vault.

In either case, You will need to use Service principal key

Azure key vault is a more secure way of storing secrets. Once you select Azure Key vault, you will need to create an Azure key vault linked service connection.

And the secret name(on the Azure HDInsights) is from the Azure Key vault. You will need to create a secret on the Azure key vault.

Name: YourSecret name
Scret valut: Service principal key

Once you create the secrets, you will need to use the secret name on the "Secrete name" section on the HDInsights linked service connection.

Please see the below screenshot for your reference.

In case you need more clarity on the Azure key vault:

When you choose Azure Key vault:
You will need to create an Azure Key vault linked service first

for the Azure key vault linked service below are the two authentication methods.

1) System assigned managed identity (by default system managed identity is created when you spin up an azure resource and will be deleted once you delete the resource)
2) user-assigned managed identity

To use the user-assigned managed identity, you will need to create credentials.

The below document explains how to use user-assigned managed identity in ADF

https://techcommunity.microsoft.com/t5/azure-data-factory-blog/support-for-user-assigned-managed-identity-in-azure-data-factory/ba-p/2841013

For your questions:

1. Do we put the UA managed identity's object principal key in the service principal id field?
   No, you can use either one of the methods at a time (using service principal key or Azure key vault)

2) we select Azure Key vault. I did create a key vault linked service and selected that, but what about the secret field? What do we put in there?

You will need to use the secret name that was created on the Azure key vault(which is having service principal key)

In short: you will need the Service principal key in both cases. The only difference is, you will save the service principal key in Azure key vault secrets and reference the name when using -Azure Key vault in the Azure HDInsights linked service connection.

I hope this clarifies your questions. If you have any further questions, please let me know.

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators