Decrypt a column of data in Python Notebook

Question

Decrypt a column of data in Python Notebook

Visakh 211 Volunteer Moderator

Hello Team
I have a use case which required the PII source data maintained in a SQL Server to be encrypted prior to ingesting to our data lake (ADLS gen 2). Currently this is done through a SQL stored procedure utilizing EncryptByKey function.

The data in the encrypted form is maintained within Parquet files inside an ADLS account.

The requirement is to allow only a class of privileged users to decrypt this data while accessing through Python code within Synapse spark pools.

The certificate and private key originally used for encryption is backed up and made available to us.

Any ideas on how we can use the information to decrypt the data through Python notebook code only for an identified set of users within Synapse?
Thanks in advance for any advice provided.

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-03-01T23:04:56.4366667+00:00

@Visakh Just checking in to see if the below information was helpful. If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. If you have any further query, do let us know.

Thank you
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-03-02T17:31:47.0466667+00:00

@Visakh We still have not heard back from you. Just wanted to check if the below information was helpful? If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

1 answer

Your answer

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-03-01T23:04:56.4366667+00:00

@Visakh Just checking in to see if the below information was helpful. If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. If you have any further query, do let us know.

Thank you
KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2023-03-02T17:31:47.0466667+00:00

@Visakh We still have not heard back from you. Just wanted to check if the below information was helpful? If it answers your query, please do click Accept Answer and Yes for "was this answer helpful", as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you

Answer 1

Hi @Visakh ,

Welcome to Microsoft Q&A forum and thanks for reaching out here.

As per my understanding you would like to know how a particular set of privileged users can be able to decrypt the PII data while working on Synapse notebooks (interactive authoring) using python code.

Since you want only particular set of privileged users to decrypt the files data, what you can do is:

Create a AAD user group for those privileged users.
Create an Azure Key vault which store the Decryption key and certificate which can decrypt the parquet files in ADLS Gen2.
Then grant the AAD group access to the Azure Key vault.
Now you can utilize Azure Key Vault Secrets Client library for Python to retrieve the secrets which contain Your decryption key and certificate values, and you can use to decrypt your secured files from ADLS Gen2.

   from azure.identity import DefaultAzureCredential
   from azure.keyvault.secrets import SecretClient
   
   credential = DefaultAzureCredential()
   
   secret_client = SecretClient(vault_url="https://my-key-vault.vault.azure.net/", credential=credential)
   secret = secret_client.get_secret("secret-name")
   
   print(secret.name)
   print(secret.value)

By doing so, when those users' login to Azure Synapse studio with their login credentials and try to execute those Python notebooks, the key vault will try to validate their permission against their credentials and if the user doesn't have permission to the key vault, it will throw error. Here is video by one of my teammate which explains how to give permission to Azure Key Vault for Synapse Notebook in Synapse Analytics: Quickstart: Azure Key Vault secret client library for Python

Hope this info helps.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Share via

Decrypt a column of data in Python Notebook

1 answer

Your answer