Decryption or Encryption of pgp files in ADF

Jeevi Danda 71

Hi,

I am trying copy the encrypted/decrypted files from one server to another server using Data factory. Before I move these files, I need to encrypt them if they decrypted and vice versa. But, I don't find a way to do this through ADF. Please suggest a best way to achieve this.

Thanks,
Chiranjeevi

AnnuKumari-MSFT 31,731 Reputation points Microsoft Employee

2022-10-13T08:26:12.27+00:00

Hi @Jeevi Danda ,
Just checking in to see if the below answer helped. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well. If you have any further query do let us know.

1 answer

AnnuKumari-MSFT 31,731 Reputation points Microsoft Employee

2022-10-11T04:38:13.147+00:00
Hi @Jeevi Danda ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

Encrypt Azure Data Factory with customer-managed keys feature is there to encrypt the data factory environment i.e., to encrypt data that datafactory is storing in the system. Unfortunately, there is no out of box feature in Azure Data factory to perform encryption/decryption of files.

You can leverage Shell command to perform encryption and decryption of files using Azure functions or databricks. Kindly check the following blog post: PGP File Decrypt Using Azure Data Factory

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Please sign in to rate this answer.
Jeevi Danda 71 Reputation points

2022-10-11T12:46:02.247+00:00

Hi @AnnuKumari-MSFT , Thanks for sharing the blog post. This blog is talking about the decryption of files using databricks. But, in our project we are not using databricks. Is there any other solution that I can implement to achieve this?

Thanks,
@Jeevi Danda

Jeevi Danda 71 Reputation points

2022-10-11T15:05:01.613+00:00

Hi @AnnuKumari-MSFT To elaborate more,

From SFTP, I have to read .pgp files and decrypt each file using a password/key ID before copy them to another SFTP

From SFTP, I have to read .pgp files and encrypt each file using a password/key ID before copy them to another SFTP

Could you please let me know the best possible solution.

Thanks,
Jeevi

AnnuKumari-MSFT 31,731 Reputation points Microsoft Employee

2022-10-11T15:07:56.743+00:00

Hi @Jeevi Danda ,

Right now , pgp encryption and decryption is not directly supported by ADF. Few alternatives are:

To write a custom code in JAVA,python or C# using Azure function and call the activity in ADF pipeline. There is no out of the box feature available currently.

To write a custom code in C# and call the code using Custom activity in ADF pipeline.

Kindly check few references related to the query:
C# code to perform PGP encryption
AzureFunctionsPGPDecrypt
Decryption pgp file in Azure gen2 via ADF

---------------

Hope it helps. Please do consider clicking Accept Answer and Up-Vote for the same as accepted answers help community as well

AnnuKumari-MSFT 31,731 Reputation points Microsoft Employee

2022-10-12T05:42:02.4+00:00

Hi @Jeevi Danda ,

Thankyou for getting back with more details. As stated above, in ADF, there is no direct method of doing encryption and decryption of files yet. You need to write the custom code using Azure function, databricks or Custom batch activity.

Kindly check if the approach suggested in below posts help you:
SFTP: Encrypting or Decrypting Files with PGP
Automated SFTP with PGP encryption and decryption

Appreciate if you could vote for this request on our Data factory feedback channel, which would be open for the user community to upvote & comment on. This allows our product team to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

----------------------

If the suggested response helped you, please click Accept Answer as accepted answer helps community as well.
Sign in to comment

Share via

Decryption or Encryption of pgp files in ADF

1 answer