Manage Azure Data Lake Analytics using the Azure CLI
Important
Azure Data Lake Analytics retired on 29 February 2024. Learn more with this announcement.
For data analytics, your organization can use Azure Synapse Analytics or Microsoft Fabric.
Learn how to manage Azure Data Lake Analytics accounts, data sources, users, and jobs using the Azure CLI. To see management topics using other tools, select the tab select above.
Before you begin this tutorial, you must have the following resources:
An Azure subscription. See Get Azure free trial.
Azure CLI. See Install and configure Azure CLI.
- Download and install the pre-release Azure CLI tools in order to complete this demo.
Authenticate by using the
az login
command and select the subscription that you want to use. For more information on authenticating using a work or school account, see Connect to an Azure subscription from the Azure CLI.az login az account set --subscription <subscription id>
You can now access the Data Lake Analytics and Data Lake Store commands. Run the following command to list the Data Lake Store and Data Lake Analytics commands:
az dls -h az dla -h
Before running any Data Lake Analytics jobs, you must have a Data Lake Analytics account. Unlike Azure HDInsight, you don't pay for an Analytics account when it isn't running a job. You only pay for the time when it's running a job. For more information, see Azure Data Lake Analytics Overview.
Run the following command to create a Data Lake account,
az dla account create --account "<Data Lake Analytics account name>" --location "<Location Name>" --resource-group "<Resource Group Name>" --default-data-lake-store "<Data Lake Store account name>"
The following command updates the properties of an existing Data Lake Analytics Account
az dla account update --account "<Data Lake Analytics Account Name>" --firewall-state "Enabled" --query-store-retention 7
List Data Lake Analytics accounts within a specific resource group
az dla account list "<Resource group name>"
az dla account show --account "<Data Lake Analytics account name>" --resource-group "<Resource group name>"
az dla account delete --account "<Data Lake Analytics account name>" --resource-group "<Resource group name>"
Data Lake Analytics currently supports the following two data sources:
When you create an Analytics account, you must designate an Azure Data Lake Storage account to be the default storage account. The default Data Lake storage account is used to store job metadata and job audit logs. After you've created an Analytics account, you can add other Data Lake Storage accounts and/or Azure Storage account.
You can view the default Data Lake Store account used by running the az dla account show
command. Default account name is listed under the defaultDataLakeStoreAccount property.
az dla account show --account "<Data Lake Analytics account name>"
az dla account blob-storage add --access-key "<Azure Storage Account Key>" --account "<Data Lake Analytics account name>" --storage-account-name "<Storage account name>"
Note
Only Blob storage short names are supported. Don't use FQDN, for example "myblob.blob.core.windows.net".
The following command updates the specified Data Lake Analytics account with another Data Lake Store account:
az dla account data-lake-store add --account "<Data Lake Analytics account name>" --data-lake-store-account-name "<Data Lake Store account name>"
To update an existing Blob storage account key:
az dla account blob-storage update --access-key "<New Blob Storage Account Key>" --account "<Data Lake Analytics account name>" --storage-account-name "<Data Lake Store account name>"
To list the Data Lake Store accounts:
az dla account data-lake-store list --account "<Data Lake Analytics account name>"
To list the Blob storage account:
az dla account blob-storage list --account "<Data Lake Analytics account name>"
To delete a Data Lake Store account:
az dla account data-lake-store delete --account "<Data Lake Analytics account name>" --data-lake-store-account-name "<Azure Data Lake Store account name>"
To delete a Blob storage account:
az dla account blob-storage delete --account "<Data Lake Analytics account name>" --storage-account-name "<Data Lake Store account name>"
You must have a Data Lake Analytics account before you can create a job. For more information, see Manage Data Lake Analytics accounts.
az dla job list --account "<Data Lake Analytics account name>"
az dla job show --account "<Data Lake Analytics account name>" --job-identity "<Job Id>"
Note
The default priority of a job is 1000, and the default degree of parallelism for a job is 1.
az dla job submit --account "<Data Lake Analytics account name>" --job-name "<Name of your job>" --script "<Script to submit>"
Use the list command to find the job ID, and then use cancel to cancel the job.
az dla job cancel --account "<Data Lake Analytics account name>" --job-identity "<Job Id>"
Use the az dla job pipeline
commands to see the pipeline information previously submitted jobs.
az dla job pipeline list --account "<Data Lake Analytics Account Name>"
az dla job pipeline show --account "<Data Lake Analytics Account Name>" --pipeline-identity "<Pipeline ID>"
Use the az dla job recurrence
commands to see the recurrence information for previously submitted jobs.
az dla job recurrence list --account "<Data Lake Analytics Account Name>"
az dla job recurrence show --account "<Data Lake Analytics Account Name>" --recurrence-identity "<Recurrence ID>"