Manage Azure Data Lake Analytics using the Azure CLI

Important

Azure Data Lake Analytics retired on 29 February 2024. Learn more with this announcement.

For data analytics, your organization can use Azure Synapse Analytics or Microsoft Fabric.

Learn how to manage Azure Data Lake Analytics accounts, data sources, users, and jobs using the Azure CLI. To see management topics using other tools, select the tab select above.

Prerequisites

Before you begin this tutorial, you must have the following resources:

  • An Azure subscription. See Get Azure free trial.

  • Azure CLI. See Install and configure Azure CLI.

    • Download and install the pre-release Azure CLI tools in order to complete this demo.
  • Authenticate by using the az login command and select the subscription that you want to use. For more information on authenticating using a work or school account, see Connect to an Azure subscription from the Azure CLI.

    az login
    az account set --subscription <subscription id>
    

    You can now access the Data Lake Analytics and Data Lake Store commands. Run the following command to list the Data Lake Store and Data Lake Analytics commands:

    az dls -h
    az dla -h
    

Manage accounts

Before running any Data Lake Analytics jobs, you must have a Data Lake Analytics account. Unlike Azure HDInsight, you don't pay for an Analytics account when it isn't running a job. You only pay for the time when it's running a job. For more information, see Azure Data Lake Analytics Overview.

Create accounts

Run the following command to create a Data Lake account,

az dla account create --account "<Data Lake Analytics account name>" --location "<Location Name>" --resource-group "<Resource Group Name>" --default-data-lake-store "<Data Lake Store account name>"

Update accounts

The following command updates the properties of an existing Data Lake Analytics Account

az dla account update --account "<Data Lake Analytics Account Name>" --firewall-state "Enabled" --query-store-retention 7

List accounts

List Data Lake Analytics accounts within a specific resource group

az dla account list "<Resource group name>"

Get details of an account

az dla account show --account "<Data Lake Analytics account name>" --resource-group "<Resource group name>"

Delete an account

az dla account delete --account "<Data Lake Analytics account name>" --resource-group "<Resource group name>"

Manage data sources

Data Lake Analytics currently supports the following two data sources:

When you create an Analytics account, you must designate an Azure Data Lake Storage account to be the default storage account. The default Data Lake storage account is used to store job metadata and job audit logs. After you've created an Analytics account, you can add other Data Lake Storage accounts and/or Azure Storage account.

Find the default Data Lake Store account

You can view the default Data Lake Store account used by running the az dla account show command. Default account name is listed under the defaultDataLakeStoreAccount property.

az dla account show --account "<Data Lake Analytics account name>"

Add other Blob storage accounts

az dla account blob-storage add --access-key "<Azure Storage Account Key>" --account "<Data Lake Analytics account name>" --storage-account-name "<Storage account name>"

Note

Only Blob storage short names are supported. Don't use FQDN, for example "myblob.blob.core.windows.net".

Add other Data Lake Store accounts

The following command updates the specified Data Lake Analytics account with another Data Lake Store account:

az dla account data-lake-store add --account "<Data Lake Analytics account name>" --data-lake-store-account-name "<Data Lake Store account name>"

Update existing data source

To update an existing Blob storage account key:

az dla account blob-storage update --access-key "<New Blob Storage Account Key>" --account "<Data Lake Analytics account name>" --storage-account-name "<Data Lake Store account name>"

List data sources

To list the Data Lake Store accounts:

az dla account data-lake-store list --account "<Data Lake Analytics account name>"

To list the Blob storage account:

az dla account blob-storage list --account "<Data Lake Analytics account name>"

Screenshot that shows Azure C L I with the "dataLakeStoreAccounts:" information highlighted.

Delete data sources

To delete a Data Lake Store account:

az dla account data-lake-store delete --account "<Data Lake Analytics account name>" --data-lake-store-account-name "<Azure Data Lake Store account name>"

To delete a Blob storage account:

az dla account blob-storage delete --account "<Data Lake Analytics account name>" --storage-account-name "<Data Lake Store account name>"

Manage jobs

You must have a Data Lake Analytics account before you can create a job. For more information, see Manage Data Lake Analytics accounts.

List jobs

az dla job list --account "<Data Lake Analytics account name>"

Data Lake Analytics list data source

Get job details

az dla job show --account "<Data Lake Analytics account name>" --job-identity "<Job Id>"

Submit jobs

Note

The default priority of a job is 1000, and the default degree of parallelism for a job is 1.

az dla job submit --account "<Data Lake Analytics account name>" --job-name "<Name of your job>" --script "<Script to submit>"

Cancel jobs

Use the list command to find the job ID, and then use cancel to cancel the job.

az dla job cancel --account "<Data Lake Analytics account name>" --job-identity "<Job Id>"

Pipelines and recurrences

Get information about pipelines and recurrences

Use the az dla job pipeline commands to see the pipeline information previously submitted jobs.

az dla job pipeline list --account "<Data Lake Analytics Account Name>"

az dla job pipeline show --account "<Data Lake Analytics Account Name>" --pipeline-identity "<Pipeline ID>"

Use the az dla job recurrence commands to see the recurrence information for previously submitted jobs.

az dla job recurrence list --account "<Data Lake Analytics Account Name>"

az dla job recurrence show --account "<Data Lake Analytics Account Name>" --recurrence-identity "<Recurrence ID>"

Next steps