Azure Function apps Function to rename file in Azure Data lake storage Gen 2

Question

Azure Function apps Function to rename file in Azure Data lake storage Gen 2

AzeemK 516

I have few text files under Azure Data lake Storage Gen 2 raw container, I want to rename these file like strip some characters ProductIDs.20210623.075739.txt to ProductIDs.txt out of file name , also replace some characters inside the file using python or power shell , this really straight forward on prem file share, is there any good documentation on how to do that by using function in Azure Function App or any other simpler way from ADF pipeline??

1 answer

Your answer

Answer 1

ShaikMaheer-MSFT 38,636 Microsoft Employee Moderator

Hi @AzeemK ,

Thank you for posting your query in Microsoft Q&A Platform.

You can achieve your requirement either using ADF or using Azure Function App.

Requirement 1: Rename file name on ADLS Gen2

Using Azure Data Factory Pipeline
- Make API call to ADLS Gen2 using web activity. This API of ADLS Gen2 helps you to rename file name.
- Have PowerShell code to rename file on ADLS Gen2 and execute that using Custom Activity. Click here for documentation of PowerShell library
Using Azure Functions
- Have Python or C# or PowerShell code to make REST API call to ADLS Gen2 to rename file.

Requirement 2: Replace some characters inside file

Using Azure Data Factory Pipeline
- You need to consider data flows inside ADF to perform this transformation. replace() function will help to achieve this.
- You can consider REST API call to ADLS Gen2 as well, to update file.
Using Azure Functions
- You can write your own code in Python or C# or PowerShell to make REST API call to ADLS Gen2 to update file.

Click here for Azure data lake Gen2 REST APIs documentation.
Click here to Azure data lake Gen2 PowerShell library documentation.

Hope this will help. Please let us know if any further queries. Thank you.

----------------------

Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

AzeemK 516

@ShaikMaheer-MSFT

below is my power shell script , I have created Azure function app and a Blobtrigger Function as well , how do I run this power shell script in Azure function , I am new to Azure function app , see powershell code and function screen shot below

 ------//rename files in ADLS Gen2 Inbound Container   
 Get-ChildItem 'Inbound/BOB/2021/09/*.txt' | rename-item -NewName {$_.name.substring(0,$_.BaseName.length-16) + $_.Extension -replace "_"," "}  
 Get-ChildItem 'Inbound/BOB/2021/09/*.txt' | rename-item -newname { [string]($_.name).substring(8) }  
 -------------------------///Change double Pipe delimiter to single inside the files  
 $Content.replace('||','|')|Set-Content Inbound/BOB/2021/09/*.txt  
 $Content.replace('||','|')|Set-Content Inbound/BOB/2021/09/*.txt![128543-functionapp1.jpg][1]

ShaikMaheer-MSFT 38,636 Reputation points Microsoft Employee Moderator

2021-09-03T06:16:49.673+00:00

Hi @AzeemK ,

Thank you for follow-up query. Please consider Accept Answer if above helped you.

Regarding follow-up query, The Blob storage trigger starts a function when a new or updated blob is detected. Click here, to know more about Blob trigger functions. You test by adding blob to your storage.
ShaikMaheer-MSFT 38,636 Reputation points Microsoft Employee Moderator

2021-09-08T17:49:37.487+00:00

Hi @AzeemK ,

Following up to see is above answer helps you? If yes, Please consider accepting answer by clicking on button. Please let us know if any further queries. Thank you.
AzeemK 516 Reputation points

2021-09-08T18:11:49.237+00:00

the documentation link you have refer to is not much help , there is no comprehensive documentation as far as how to build and run Azure function using power shell
ShaikMaheer-MSFT 38,636 Reputation points Microsoft Employee Moderator

2021-09-10T16:05:02.723+00:00

Hi @AzeemK ,

Below documentation explains how to build and run a PowerShell function in Azure using Visual Studio Code.
https://learn.microsoft.com/en-us/azure/azure-functions/create-first-function-vs-code-powershell

Kindly consider clicking on Accept Answer if it helps. Accepted answers helps community as well. Please let us know if any further queries. Thank you.

Share via

Azure Function apps Function to rename file in Azure Data lake storage Gen 2

1 answer

Your answer