ADF linked service for ADLS - Terraform

Question

ADF linked service for ADLS - Terraform

Rajat Srivastava 40

I have been trying to make a linked service for ADLS. I want to access the ADLS using the secrets stored in my Key Vault. I have made a linked service to access Key Vault.

Now I want to access the ADLS using the Key Vault linked service (like we are able to do on the Azure Portal) I need to write terraform script for the same.

I have been trying to find out how to do this, but have not been able to find any solution.

Bhargava-MSFT 31,356 Microsoft Employee Moderator

Hello Rajat Srivastava,

You can try the below script to access ADLS using Key Vault linked service

data "azurerm_client_config" "current" {}

resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

resource "azurerm_key_vault" "example" {
  name                = "example"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  tenant_id           = data.azurerm_client_config.current.tenant_id
  sku_name            = "standard"
}

resource "azurerm_data_factory" "example" {
  name                = "example"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
}

resource "azurerm_data_factory_linked_service_key_vault" "example_key_vault" {
  name            = "example_key_vault"
  data_factory_id = azurerm_data_factory.example.id
  key_vault_id    = azurerm_key_vault.example.id
}

resource "azurerm_data_factory_linked_service_data_lake_storage_gen2" "example_adls" {
  name                = "example_adls"
  data_factory_id     = azurerm_data_factory.example.id
  account_name        = "example"
  tenant_id           = "11111111-1111-1111-1111-111111111111"
  service_principal_id = data.azurerm_client_config.current.client_id
  service_principal_key = azurerm_data_factory_linked_service_key_vault.example_key_vault.get_secret("example_secret_name")
}

Reference documents: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_key_vault

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_data_lake_storage_gen2

I hope this helps. Please let me know if you have any further questions.

Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-13T16:50:23.6333333+00:00

Hello Rajat Srivastava,

I am checking to see if you got a chance to look into my earlier response. Please let us know if you need any further assistance.
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-16T21:15:30.97+00:00

Hello Rajat Srivastava,

I am checking to see if you have any further questions here.
Rajat Srivastava 40 Reputation points

2023-10-17T13:02:48.5+00:00

Thanks for your response. This is not what my question . I want to make a linked service for ADLS that uses the Key Vault Linked Service to Authenticate into the Storage account (using the secrets stored in the Key Vault)

check the ss below -
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-24T15:53:39.97+00:00

Hello Rajat Srivastava,

I am checking to see if you need any further assistance here.

1 answer

Your answer

Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-12T16:36:11.9566667+00:00

Hello Rajat Srivastava,

You can try the below script to access ADLS using Key Vault linked service

data "azurerm_client_config" "current" {} resource "azurerm_resource_group" "example" { name = "example-resources" location = "West Europe" } resource "azurerm_key_vault" "example" { name = "example" location = azurerm_resource_group.example.location resource_group_name = azurerm_resource_group.example.name tenant_id = data.azurerm_client_config.current.tenant_id sku_name = "standard" } resource "azurerm_data_factory" "example" { name = "example" location = azurerm_resource_group.example.location resource_group_name = azurerm_resource_group.example.name } resource "azurerm_data_factory_linked_service_key_vault" "example_key_vault" { name = "example_key_vault" data_factory_id = azurerm_data_factory.example.id key_vault_id = azurerm_key_vault.example.id } resource "azurerm_data_factory_linked_service_data_lake_storage_gen2" "example_adls" { name = "example_adls" data_factory_id = azurerm_data_factory.example.id account_name = "example" tenant_id = "11111111-1111-1111-1111-111111111111" service_principal_id = data.azurerm_client_config.current.client_id service_principal_key = azurerm_data_factory_linked_service_key_vault.example_key_vault.get_secret("example_secret_name") }

Reference documents: https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_key_vault

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_data_lake_storage_gen2

I hope this helps. Please let me know if you have any further questions.
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-13T16:50:23.6333333+00:00

Hello Rajat Srivastava,

I am checking to see if you got a chance to look into my earlier response. Please let us know if you need any further assistance.
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-16T21:15:30.97+00:00

Hello Rajat Srivastava,

I am checking to see if you have any further questions here.
Rajat Srivastava 40 Reputation points

2023-10-17T13:02:48.5+00:00

Thanks for your response. This is not what my question . I want to make a linked service for ADLS that uses the Key Vault Linked Service to Authenticate into the Storage account (using the secrets stored in the Key Vault)

check the ss below -
Bhargava-MSFT 31,356 Reputation points Microsoft Employee Moderator

2023-10-24T15:53:39.97+00:00

Hello Rajat Srivastava,

I am checking to see if you need any further assistance here.

Answer 1

Hello Rajat Srivastava,

Thanks for the details.

To create a ADLS gen2 linked service that uses the key valut, then you need to use either service principal or account key authentication.

here is an example using service principal

# Define the Key Vault linked service
resource "azurerm_data_factory_linked_service_key_vault" "key_vault" {
  name                = "key_vault_linked_service"
  data_factory_name   = azurerm_data_factory.data_factory.name
  resource_group_name = azurerm_resource_group.resource_group.name
  vault_uri           = "https://.vault.azure.net/"
  tenant_id           = ""
  client_id           = ""
  client_secret       = ""
}

# Define the ADLS Gen2 linked service
resource "azurerm_data_factory_linked_service_data_lake_storage_gen2" "adls_gen2" {
  name                = "adls_gen2_linked_service"
  data_factory_name   = azurerm_data_factory.data_factory.name
  resource_group_name = azurerm_resource_group.resource_group.name
  account_name        = ""
  authentication_type = "ServicePrincipal"
  service_principal_id = azurerm_data_factory_linked_service_key_vault.key_vault.get_secret("")
  service_principal_key = azurerm_data_factory_linked_service_key_vault.key_vault.get_secret("")
  tenant_id           = ""
}

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_key_vault

https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_service_data_lake_storage_gen2

Sorry, I don't have an environment to test the script and I have used AI to generate the script.

I hope this helps.

Share via

ADF linked service for ADLS - Terraform

1 answer

Your answer