Powershell Scripts

Abhay Chandramouli 1,031 Reputation points
2022-07-06T04:31:27.073+00:00

Hi I have a powershell script that reads data from storage account (csv) and calls and api. Because the csv file has millions of records.. can you suggest a suitable way to run the script ?

Azure CLI / Azure Function App / Azure Data Factory ?
After running the script will we be able to shutdown our local systems and the scripts continue to run on azure ?

Azure Automation
Azure Automation
An Azure service that is used to automate, configure, and install updates across hybrid environments.
1,307 questions
Windows Server PowerShell
Windows Server PowerShell
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.PowerShell: A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
5,627 questions
{count} votes

1 answer

Sort by: Most helpful
  1. AnuragSingh-MSFT 21,476 Reputation points
    2022-07-11T12:36:19.28+00:00

    Hi anonymous user,

    Thank you for reaching out to Microsoft Q&A for this question.

    I understand that you have a script that reads data from storage account (csv) and calls some API, and you would like to run it in Azure. There are a number of ways that you can run it in Azure and one of the easiest ways would be to run it in Azure Automation which would not require having a machine to be in running state. I have provided a summary of steps required to create a simple runbook and run it in Azure below:

    1. Login to Azure portal and create an Automation account, if you already don't have one. Ref: Create a standalone Azure Automation account

    2. In the created Azure Automation account in Portal --> Click on "Runbooks" --> "Create a Runbook" (Runbooks are where you store the PowerShell script)

    219571-image.png

    3. In the next screen, provide a Name of your choice, Runbook Type as PowerShell, select the Runtime version of PowerShell (to ensure that it matches with the version that you have on your system where this script runs fine, get the output of $PSVersiontable. It would tell you the version of PowerShell where you are running the script)

    4. The steps above would take you to the script editor page, paste your script here. To test, you can click on Save --> Test pane --> Start. This will run the script right away and share the result. In the script you can use write-output cmdlet at different location to include status message as the script runs through. To go back to the editor, you can use the X button on the top right corner in the browser body.
    219518-image.png

    5. Once you are satisfied with the script, click on Save --> Publish. Note that the new runbook created is available in Azure Automation Account --> Runbooks. To run it, you can click on the runbook from here --> Start. From here, you can close the browser/computer and return back later to view the output. The output will be available under Jobs (for your runbook)

    6. If you want, you can also attach schedule to this runbook, so that it will be run regularly based on it.

    The steps above are for a simple scenario where no authentication is required from runbook. In case your runbook accesses other resources (for example storage account in current case), an authentication mechanism will have to be provided. In such case, Managed Identity can be used to for authentication and authorization. See the link below for more details - PowerShell Runbook with Managed-Identity.

    Note that when running the Runbook in azure automation, you need not continue using the portal/machine through which you are logged in.

    Also, Azure Batch would not be a good option as it is for running large-scale parallel and high-performance computing (HPC) batch jobs efficiently in Azure (which is not the requirement here). Another easy option available in Azure to run your script is Azure Functions.

    Please let me know if you have any questions.

    ---
    Please 'Accept as answer' if it helped so that it can help others in the community looking for help on similar topics.

    1 person found this answer helpful.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.