az stream-analytics job

Note

This reference is part of the stream-analytics extension for the Azure CLI (version 2.15.0 or higher). The extension will automatically install the first time you run an az stream-analytics job command. Learn more about extensions.

Manage streaming job with stream analytics.

Commands

az stream-analytics job create

Create a streaming job or replaces an already existing streaming job.

az stream-analytics job delete

Delete a streaming job.

az stream-analytics job list

List all of the streaming jobs in the specified resource group. And Lists all of the streaming jobs in the given subscription.

az stream-analytics job scale

Scale a streaming job when the job is running.

az stream-analytics job show

Get details about the specified streaming job.

az stream-analytics job start

Start a streaming job. Once a job is started it will start processing input events and produce output.

az stream-analytics job stop

Stop a running streaming job. This will cause a running streaming job to stop processing input events and producing output.

az stream-analytics job update

Update an existing streaming job. This can be used to partially update (ie. update one or two properties) a streaming job without affecting the rest the job definition.

az stream-analytics job wait

Place the CLI in a waiting state until a condition of the stream-analytics job is met.

az stream-analytics job create

Create a streaming job or replaces an already existing streaming job.

az stream-analytics job create --job-name
                               --resource-group
                               [--arrival-max-delay]
                               [--compatibility-level {1.0, 1.2}]
                               [--content-storage-policy {JobStorageAccount, SystemAccount}]
                               [--data-locale]
                               [--functions]
                               [--id]
                               [--identity]
                               [--if-match]
                               [--if-none-match]
                               [--inputs]
                               [--job-storage-account]
                               [--job-type {Cloud, Edge}]
                               [--location]
                               [--no-wait]
                               [--order-max-delay]
                               [--out-of-order-policy {Adjust, Drop}]
                               [--output-error-policy {Drop, Stop}]
                               [--output-start-mode {CustomTime, JobStartTime, LastOutputEventTime}]
                               [--output-start-time]
                               [--outputs]
                               [--tags]
                               [--transformation]

Examples

Create a complete streaming job (a streaming job with a transformation, at least 1 input and at least 1 output)

az stream-analytics job create --job-name "sj7804" --resource-group "sjrg3276" --location "West US" --compatibility-level "1.0" --data-locale "en-US" --arrival-max-delay 5 --order-max-delay 0 --out-of-order-policy "Drop" --functions "[]" --inputs "[{\"name\":\"inputtest\",\"properties\":{\"type\":\"Stream\",\"datasource\":{\"type\":\"Microsoft.Storage/Blob\",\"properties\":{\"container\":\"containerName\",\"pathPattern\":\"\",\"storageAccounts\":[{\"accountKey\":\"yourAccountKey==\",\"accountName\":\"yourAccountName\"}]}},\"serialization\":{\"type\":\"Json\",\"properties\":{\"encoding\":\"UTF8\"}}}}]" --output-error-policy "Drop" --outputs "[{\"name\":\"outputtest\",\"datasource\":{\"type\":\"Microsoft.Sql/Server/Database\",\"properties\":{\"database\":\"databaseName\",\"password\":\"userPassword\",\"server\":\"serverName\",\"table\":\"tableName\",\"user\":\"<user>\"}}}]" --transformation name="transformationtest" streaming-units=1 query="Select Id, Name from inputtest" --tags key1="value1" key3="value3" randomKey="randomValue"

Create a streaming job shell (a streaming job with no inputs, outputs, transformation, or functions)

az stream-analytics job create --job-name "sj59" --resource-group "sjrg6936" --location "West US" --compatibility-level "1.0" --data-locale "en-US" --arrival-max-delay 16 --order-max-delay 5 --out-of-order-policy "Drop" --functions "[]" --inputs "[]" --output-error-policy "Drop" --outputs "[]" --tags key1="value1" key3="value3" randomKey="randomValue"

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--arrival-max-delay

The maximum tolerable delay in seconds where events arriving late could be included. Supported range is -1 to 1814399 (20.23:59:59 days) and -1 is used to specify wait indefinitely. If the property is absent, it is interpreted to have a value of -1.

--compatibility-level

Controls certain runtime behaviors of the streaming job.

accepted values: 1.0, 1.2
--content-storage-policy

Valid values are JobStorageAccount and SystemAccount. If set to JobStorageAccount, this requires the user to also specify jobStorageAccount property. .

accepted values: JobStorageAccount, SystemAccount
--data-locale

The data locale of the stream analytics job. Value should be the name of a supported .NET Culture from the set https://msdn.microsoft.com/en-us/library/system.globalization.culturetypes(v=vs.110).aspx. Defaults to 'en-US' if none specified.

--functions

A list of one or more functions for the streaming job. The name property for each function is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual transformation. Expected value: json-string/json-file/@json-file.

--id

The resource id of cluster.

--identity

Describe the system-assigned managed identity assigned to this job that can be used to authenticate with inputs and outputs.

--if-match

The ETag of the streaming job. Omit this value to always overwrite the current record set. Specify the last-seen ETag value to prevent accidentally overwriting concurrent changes.

--if-none-match

Set to '*' to allow a new streaming job to be created, but to prevent updating an existing record set. Other values will result in a 412 Pre-condition Failed response.

--inputs

A list of one or more inputs to the streaming job. The name property for each input is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual input. Expected value: json-string/json-file/@json-file.

--job-storage-account

The properties that are associated with an Azure Storage account with MSI.

--job-type

Describes the type of the job. Valid modes are Cloud and 'Edge'.

accepted values: Cloud, Edge
--location -l

Location. Values from: az account list-locations. You can configure the default location using az configure --defaults location=<location>.

--no-wait

Do not wait for the long-running operation to finish.

--order-max-delay

The maximum tolerable delay in seconds where out-of-order events can be adjusted to be back in order.

--out-of-order-policy

Indicates the policy to apply to events that arrive out of order in the input event stream.

accepted values: Adjust, Drop
--output-error-policy

Indicates the policy to apply to events that arrive at the output and cannot be written to the external storage due to being malformed (missing column values, column values of wrong type or size).

accepted values: Drop, Stop
--output-start-mode

This property should only be utilized when it is desired that the job be started immediately upon creation. Value may be JobStartTime, CustomTime, or LastOutputEventTime to indicate whether the starting point of the output event stream should start whenever the job is started, start at a custom user time stamp specified via the outputStartTime property, or start from the last event output time.

accepted values: CustomTime, JobStartTime, LastOutputEventTime
--output-start-time

Value is either an ISO-8601 formatted time stamp that indicates the starting point of the output event stream, or null to indicate that the output event stream will start whenever the streaming job is started. This property must have a value if outputStartMode is set to CustomTime.

--outputs

A list of one or more outputs for the streaming job. The name property for each output is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual output. Expected value: json-string/json-file/@json-file.

--tags

Space-separated tags: key[=value] [key[=value] ...]. Use "" to clear existing tags.

--transformation

Indicate the query and the number of streaming units to use for the streaming job. The name property of the transformation is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual transformation.

az stream-analytics job delete

Delete a streaming job.

az stream-analytics job delete --job-name
                               --resource-group
                               [--no-wait]
                               [--yes]

Examples

Delete a streaming job

az stream-analytics job delete --job-name "sj59" --resource-group "sjrg6936"

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--yes -y

Do not prompt for confirmation.

az stream-analytics job list

List all of the streaming jobs in the specified resource group. And Lists all of the streaming jobs in the given subscription.

az stream-analytics job list [--expand]
                             [--resource-group]

Examples

List all streaming jobs in a resource group and do not use the $expand OData query parameter

az stream-analytics job list --resource-group "sjrg6936"

List all streaming jobs in a resource group and use the $expand OData query parameter to expand inputs, outputs, transformation, and functions

az stream-analytics job list --expand "inputs,outputs,transformation,functions" --resource-group "sjrg3276"

List all streaming jobs in a subscription and do not use the $expand OData query parameter

az stream-analytics job list

List all streaming jobs in a subscription and use the $expand OData query parameter to expand inputs, outputs, transformation, and functions

az stream-analytics job list --expand "inputs,outputs,transformation,functions"

Optional Parameters

--expand

The $expand OData query parameter. This is a comma-separated list of additional streaming job properties to include in the response, beyond the default set returned when this parameter is absent. The default set is all streaming job properties other than 'inputs', 'transformation', 'outputs', and 'functions'.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

az stream-analytics job scale

Scale a streaming job when the job is running.

az stream-analytics job scale --job-name
                              --resource-group
                              [--no-wait]
                              [--streaming-units]

Examples

Scale a streaming job

az stream-analytics job scale --job-name "sj59" --resource-group "sjrg6936" --streaming-units 36

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--streaming-units

Specifies the number of streaming units that the streaming job will scale to.

az stream-analytics job show

Get details about the specified streaming job.

az stream-analytics job show --job-name
                             --resource-group
                             [--expand]

Examples

Get a streaming job and do not use the $expand OData query parameter

az stream-analytics job show --job-name "sj59" --resource-group "sjrg6936"

Get a streaming job and use the $expand OData query parameter to expand inputs, outputs, transformation, and functions

az stream-analytics job show --expand "inputs,outputs,transformation,functions" --job-name "sj7804" --resource-group "sjrg3276"

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--expand

The $expand OData query parameter. This is a comma-separated list of additional streaming job properties to include in the response, beyond the default set returned when this parameter is absent. The default set is all streaming job properties other than 'inputs', 'transformation', 'outputs', and 'functions'.

az stream-analytics job start

Start a streaming job. Once a job is started it will start processing input events and produce output.

az stream-analytics job start --job-name
                              --resource-group
                              [--no-wait]
                              [--output-start-mode {CustomTime, JobStartTime, LastOutputEventTime}]
                              [--output-start-time]

Examples

Start a streaming job with CustomTime output start mode

az stream-analytics job start --job-name "sj59" --resource-group "sjrg6936" --output-start-mode "CustomTime" --output-start-time "2012-12-12T12:12:12Z"

Start a streaming job with JobStartTime output start mode

az stream-analytics job start --job-name "sj59" --resource-group "sjrg6936" --output-start-mode "JobStartTime"

Start a streaming job with LastOutputEventTime output start mode

az stream-analytics job start --job-name "sj59" --resource-group "sjrg6936" --output-start-mode "LastOutputEventTime"

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--output-start-mode

Value may be JobStartTime, CustomTime, or LastOutputEventTime to indicate whether the starting point of the output event stream should start whenever the job is started, start at a custom user time stamp specified via the outputStartTime property, or start from the last event output time.

accepted values: CustomTime, JobStartTime, LastOutputEventTime
--output-start-time

Value is either an ISO-8601 formatted time stamp that indicates the starting point of the output event stream, or null to indicate that the output event stream will start whenever the streaming job is started. This property must have a value if outputStartMode is set to CustomTime.

az stream-analytics job stop

Stop a running streaming job. This will cause a running streaming job to stop processing input events and producing output.

az stream-analytics job stop --job-name
                             --resource-group
                             [--no-wait]

Examples

Stop a streaming job

az stream-analytics job stop --job-name "sj59" --resource-group "sjrg6936"

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

az stream-analytics job update

Update an existing streaming job. This can be used to partially update (ie. update one or two properties) a streaming job without affecting the rest the job definition.

az stream-analytics job update --job-name
                               --resource-group
                               [--arrival-max-delay]
                               [--compatibility-level {1.0, 1.2}]
                               [--content-storage-policy {JobStorageAccount, SystemAccount}]
                               [--data-locale]
                               [--functions]
                               [--id]
                               [--identity]
                               [--if-match]
                               [--inputs]
                               [--job-storage-account]
                               [--job-type {Cloud, Edge}]
                               [--location]
                               [--order-max-delay]
                               [--out-of-order-policy {Adjust, Drop}]
                               [--output-error-policy {Drop, Stop}]
                               [--output-start-mode {CustomTime, JobStartTime, LastOutputEventTime}]
                               [--output-start-time]
                               [--outputs]
                               [--tags]
                               [--transformation]

Examples

Update a streaming job

az stream-analytics job update --job-name "sj59" --resource-group "sjrg6936" --arrival-max-delay 13 --order-max-delay 21

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--arrival-max-delay

The maximum tolerable delay in seconds where events arriving late could be included. Supported range is -1 to 1814399 (20.23:59:59 days) and -1 is used to specify wait indefinitely. If the property is absent, it is interpreted to have a value of -1.

--compatibility-level

Controls certain runtime behaviors of the streaming job.

accepted values: 1.0, 1.2
--content-storage-policy

Valid values are JobStorageAccount and SystemAccount. If set to JobStorageAccount, this requires the user to also specify jobStorageAccount property. .

accepted values: JobStorageAccount, SystemAccount
--data-locale

The data locale of the stream analytics job. Value should be the name of a supported .NET Culture from the set https://msdn.microsoft.com/en-us/library/system.globalization.culturetypes(v=vs.110).aspx. Defaults to 'en-US' if none specified.

--functions

A list of one or more functions for the streaming job. The name property for each function is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual transformation. Expected value: json-string/json-file/@json-file.

--id

The resource id of cluster.

--identity

Describe the system-assigned managed identity assigned to this job that can be used to authenticate with inputs and outputs.

--if-match

The ETag of the streaming job. Omit this value to always overwrite the current record set. Specify the last-seen ETag value to prevent accidentally overwriting concurrent changes.

--inputs

A list of one or more inputs to the streaming job. The name property for each input is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual input. Expected value: json-string/json-file/@json-file.

--job-storage-account

The properties that are associated with an Azure Storage account with MSI.

--job-type

Describes the type of the job. Valid modes are Cloud and 'Edge'.

accepted values: Cloud, Edge
--location -l

Location. Values from: az account list-locations. You can configure the default location using az configure --defaults location=<location>.

--order-max-delay

The maximum tolerable delay in seconds where out-of-order events can be adjusted to be back in order.

--out-of-order-policy

Indicates the policy to apply to events that arrive out of order in the input event stream.

accepted values: Adjust, Drop
--output-error-policy

Indicates the policy to apply to events that arrive at the output and cannot be written to the external storage due to being malformed (missing column values, column values of wrong type or size).

accepted values: Drop, Stop
--output-start-mode

This property should only be utilized when it is desired that the job be started immediately upon creation. Value may be JobStartTime, CustomTime, or LastOutputEventTime to indicate whether the starting point of the output event stream should start whenever the job is started, start at a custom user time stamp specified via the outputStartTime property, or start from the last event output time.

accepted values: CustomTime, JobStartTime, LastOutputEventTime
--output-start-time

Value is either an ISO-8601 formatted time stamp that indicates the starting point of the output event stream, or null to indicate that the output event stream will start whenever the streaming job is started. This property must have a value if outputStartMode is set to CustomTime.

--outputs

A list of one or more outputs for the streaming job. The name property for each output is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual output. Expected value: json-string/json-file/@json-file.

--tags

Space-separated tags: key[=value] [key[=value] ...]. Use "" to clear existing tags.

--transformation

Indicate the query and the number of streaming units to use for the streaming job. The name property of the transformation is required when specifying this property in a PUT request. This property cannot be modify via a PATCH operation. You must use the PATCH API available for the individual transformation.

az stream-analytics job wait

Place the CLI in a waiting state until a condition of the stream-analytics job is met.

az stream-analytics job wait --job-name
                             --resource-group
                             [--created]
                             [--custom]
                             [--deleted]
                             [--exists]
                             [--expand]
                             [--interval]
                             [--timeout]
                             [--updated]

Examples

Pause executing next line of CLI script until the stream-analytics job is successfully created.

az stream-analytics job wait --expand "inputs,outputs,transformation,functions" --job-name "sj7804" --resource-group "sjrg3276" --created

Pause executing next line of CLI script until the stream-analytics job is successfully deleted.

az stream-analytics job wait --expand "inputs,outputs,transformation,functions" --job-name "sj7804" --resource-group "sjrg3276" --deleted

Required Parameters

--job-name --name -n

The name of the streaming job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--created

Wait until created with 'provisioningState' at 'Succeeded'.

--custom

Wait until the condition satisfies a custom JMESPath query. E.g. provisioningState!='InProgress', instanceView.statuses[?code=='PowerState/running'].

--deleted

Wait until deleted.

--exists

Wait until the resource exists.

--expand

The $expand OData query parameter. This is a comma-separated list of additional streaming job properties to include in the response, beyond the default set returned when this parameter is absent. The default set is all streaming job properties other than 'inputs', 'transformation', 'outputs', and 'functions'.

--interval

Polling interval in seconds.

default value: 30
--timeout

Maximum wait in seconds.

default value: 3600
--updated

Wait until updated with provisioningState at 'Succeeded'.