Get started with Azure Data Lake Analytics using Azure PowerShell

Article
12/20/2023

Important

New Azure Data Lake Analytics accounts can no longer be created unless your subscription has been enabled. If you need your subscription to be enabled contact support and provide your business scenario.

If you are already using Azure Data Lake Analytics, you'll need to create a migration plan to Azure Synapse Analytics for your organization by February 29th, 2024.

Learn how to use Azure PowerShell to create Azure Data Lake Analytics accounts and then submit and run U-SQL jobs. For more information about Data Lake Analytics, see Azure Data Lake Analytics overview.

Prerequisites

Note

We recommend that you use the Azure Az PowerShell module to interact with Azure. See Install Azure PowerShell to get started. To learn how to migrate to the Az PowerShell module, see Migrate Azure PowerShell from AzureRM to Az.

Before you begin this tutorial, you must have the following information:

An Azure Data Lake Analytics account. See Get started with Data Lake Analytics.
A workstation with Azure PowerShell. See How to install and configure Azure PowerShell.

Log in to Azure

This tutorial assumes you're already familiar with using Azure PowerShell. In particular, you need to know how to log in to Azure. See the Get started with Azure PowerShell if you need help.

To log in with a subscription name:

Connect-AzAccount -SubscriptionName "ContosoSubscription"

Instead of the subscription name, you can also use a subscription ID to log in:

Connect-AzAccount -SubscriptionId "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

If successful, the output of this command looks like the following text:

Environment           : AzureCloud
Account               : joe@contoso.com
TenantId              : "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
SubscriptionId        : "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
SubscriptionName      : ContosoSubscription
CurrentStorageAccount :

Preparing for the tutorial

The PowerShell snippets in this tutorial use these variables to store this information:

$rg = "<ResourceGroupName>"
$adls = "<DataLakeStoreAccountName>"
$adla = "<DataLakeAnalyticsAccountName>"
$location = "East US 2"

Get information about a Data Lake Analytics account

Get-AdlAnalyticsAccount -ResourceGroupName $rg -Name $adla

Submit a U-SQL job

Create a PowerShell variable to hold the U-SQL script.

$script = @"
@a  = 
    SELECT * FROM 
        (VALUES
            ("Contoso", 1500.0),
            ("Woodgrove", 2700.0)
        ) AS 
              D( customer, amount );
OUTPUT @a
    TO "/data.csv"
    USING Outputters.Csv();

"@

Submit the script text with the Submit-AdlJob cmdlet and the -Script parameter.

$job = Submit-AdlJob -Account $adla -Name "My Job" -Script $script

As an alternative, you can submit a script file using the -ScriptPath parameter:

$filename = "d:\test.usql"
$script | out-File $filename
$job = Submit-AdlJob -Account $adla -Name "My Job" -ScriptPath $filename

Get the status of a job with Get-AdlJob.

$job = Get-AdlJob -Account $adla -JobId $job.JobId

Instead of calling Get-AdlJob over and over until a job finishes, use the Wait-AdlJob cmdlet.

Wait-AdlJob -Account $adla -JobId $job.JobId

Download the output file using Export-AdlStoreItem.

Export-AdlStoreItem -Account $adls -Path "/data.csv" -Destination "C:\data.csv"

Share via