How to securely use variables and parameters in your pipeline

Azure DevOps Services | Azure DevOps Server 2022 | Azure DevOps Server 2020

This article discusses how to securely use variables and parameters to gather input from pipeline users. If you'd like to learn more about using variables and parameters, see:

• Define variables
• Set secret variables
• Use predefined variables
• Use runtime parameters
• Template types & usage

Use caution with secret variables. The recommended ways to set secret variables are in the UI, in a variable group, and in a variable group from Azure Key Vault. For more information, see set secret variables.

Variables

Variables can be a convenient way to collect information from the user up front. You can also use variables to pass data from step to step within a pipeline.

But use variables with caution. Newly created variables, whether they're defined in YAML or written by a script, are read-write by default. A downstream step can change the value of a variable in a way that you don't expect.

For instance, your script reads:

msbuild.exe myproj.proj -property:Configuration=$(MyConfig)

A preceding step could set MyConfig to Debug & deltree /y c:. Although this example would only delete the contents of your build agent, you can imagine how this setting could easily become far more dangerous.

You can make variables read-only. System variables like Build.SourcesDirectory, task output variables, and queue-time variables are always read-only. Variables that are created in YAML or created at run time by a script can be designated as read-only. When a script or task creates a new variable, it can pass the isReadonly=true flag in its logging command to make the variable read-only.

In YAML, you can specify read-only variables by using a specific key:

variables:
- name: myReadOnlyVar
  value: myValue
  readonly: true

Queue-time variables

When defining a variable in the Pipelines UI editor, you can choose to let users override its value when running the pipeline. We call such a variable a queue-time variable. Queue-time variables are always defined in the Pipelines UI editor.

Queue-time variables are exposed to the end user when they manually run a pipeline, and they can change their values.

Users need Edit queue build configuration permission on the pipeline to specify variables set at queue time.

Limit variables that can be set at queue time

The UI and REST API used to run a pipeline provide means for users to define new variables at queue time.

In the early days of Azure Pipelines, this functionality had some issues:

• It allowed users to define new variables that aren't explicitly defined by the pipeline author in the definition.
• It allowed users to override system variables.

To correct these issues, we defined a setting to limit variables that can be set at queue time. With this setting enabled, only those variables that are explicitly marked as "Settable at queue time" can be set. In other words, you can set any variables at queue time unless this setting is enabled.

The setting is designed to work at organization level and at project level.

1. Organization level. When the setting is on, it enforces that, for all pipelines in all projects in the organization, only those variables that are explicitly marked as "Settable at queue time" can be set. When the setting is off, each project can choose whether to restrict variables set at queue time or not. The setting is a toggle under Organization Settings -> Pipelines -> Settings. Only Project Collection Administrators can enable or disable it.
2. Project level. When the setting is on, it enforces that, for all pipelines in the project, only those variables that are explicitly marked as "Settable at queue time" can be set. If the setting is on at the organization level, then it is on for all projects and can't be turned off. The setting is a toggle under Project Settings -> Pipelines -> Settings. Only Project Administrators can enable or disable it.

Lets look at an example. Say the setting is on and your pipeline defines a variable named my_variable that isn't settable at queue time.

Next, assume you wish to run the pipeline. The Variables panel doesn't show any variables, and the Add variable button is missing.

Using the Builds - Queue and the Runs - Run Pipeline REST API calls to queue a pipeline run and set the value of my_variable or of a new variable will fail with an error similar to the following.

{
  "$id": "1",
  "innerException": null,
  "message": "You can't set the following variables (my_variable). If you want to be able to set these variables, then edit the pipeline and select Settable at queue time on the variables tab of the pipeline editor.",
  "typeName": "Microsoft.Azure.Pipelines.WebApi.PipelineValidationException, Microsoft.Azure.Pipelines.WebApi",
  "typeKey": "PipelineValidationException",
  "errorCode": 0,
  "eventId": 3000
}

Parameters

Unlike variables, pipeline parameters can't be changed by a pipeline while it's running. Parameters have data types such as number and string, and they can be restricted to a subset of values. Restricting the parameters is useful when a user-configurable part of the pipeline should take a value only from a constrained list. The setup ensures that the pipeline won't take arbitrary data.

Enable shell tasks arguments parameter validation

Pipelines can reference tasks that are executed in the pipeline. Several tasks included in Azure DevOps have an arguments parameter that lets you specify more options for the task.

When the setting Enable shell tasks arguments parameter validation is enabled, the arguments parameter is reviewed for any characters that may not be executed correctly by the shell. Example characters include semi-colons, quotes, and parentheses.

Similar to the Limit variables that can be set at queue time option, Enable shell tasks arguments parameter validation may be configured at the organization-level at Settings > Pipelines > Settings or at the project-level at Settings > Pipelines > Settings.

When enabled, there's a detected validation issue, an error message like the following is logged: Detected characters in arguments that may not be executed correctly by the shell. Please escape special characters using backtick (`).

To resolve the issue, adjust the arguments by escaping special characters as indicated in the error message.

When Enable shell tasks arguments parameter validation is enabled, validation is applied to the arguments parameter in the following tasks.

• PowerShell
• BatchScript
• Bash
• Ssh
• AzureFileCopy
• WindowsMachineFileCopy

Next steps

After you secure your inputs, you also need to secure your shared infrastructure.