Environment - Kubernetes resource
Azure DevOps Services | Azure DevOps Server 2022 | Azure DevOps Server 2020
The Kubernetes resource view shows the status of objects within the namespace that are mapped to the resource. The resource view also overlays pipeline traceability so you can trace back from a Kubernetes object to the pipeline, and then back to the commit.
Use Kubernetes resources to target Kubernetes clusters in an environment for deployment. Use pipelines to deploy to Azure Kubernetes Service (AKS) and clusters from any other cloud provider.
You can use Kubernetes resources with public or private clusters. For more information about how resources work, see resources in YAML and security with resources.
Note
If you're using a private AKS cluster, make sure you're connected to the cluster's virtual network as the the API server endpoint is not exposed through a public IP address.
Azure Pipelines recommends setting up a self-hosted agent within a VNET that has access to the cluster's virtual network. See Options for connecting to the private cluster for details.
Overview
See the following advantages of using Kubernetes resource views within environments:
Pipeline traceability - The Kubernetes manifest task, used for deployments, adds more annotations to show pipeline traceability in resource views. Pipeline traceability helps to identify the originating Azure DevOps organization, project, and pipeline responsible for updates that were made to an object within the namespace.
Diagnose resource health - Workload status can be useful for quickly debugging mistakes or regressions that were introduced by a new deployment. For example, for unconfigured imagePullSecrets resulting in ImagePullBackOff errors, pod status information can help you identify the root cause for the issue.
Review App - Review App works by deploying every pull request from your Git repository to a dynamic Kubernetes resource under the environment. Reviewers can see how those changes look and work with other dependent services before they're merged into the target branch and deployed to production.
Use Azure Kubernetes Service
A ServiceAccount gets created in your chosen cluster and namespace when you use Azure Kubernetes Service (AKS). For a Kubernetes RBAC-enabled cluster, RoleBinding also gets created to limit the scope of the created service account to the chosen namespace. For a Kubernetes RBAC-disabled cluster, the ServiceAccount created has cluster-wide privileges (across namespaces).
Add an AKS Kubernetes resource
In the environment details page, select Add resource and choose Kubernetes.
Select Azure Kubernetes Service in the Provider dropdown.
Choose the Azure subscription, cluster, and namespace (new/existing).
Select Validate and create to create the Kubernetes resource.
Verify that you see a cluster for your environment. You'll see the text "Never deployed" if you haven't yet deployed code to your cluster.
Use an existing service account
The Azure Kubernetes Service maps a Kubernetes resource within your environment to a namespace.
For more information about setting up a Kubernetes service connection outside of an environment, see the Kubernetes service connection section in Service connections.
Tip
Use the generic provider (existing service account) to map a Kubernetes resource to a namespace from a non-AKS cluster.
Add a non-AKS Kubernetes resource
In the environment details page, select Add resource and choose Kubernetes.
Select Generic provider (existing service account) for your provider.
Add the cluster name and namespace values.
Add the server URL. You can get the URL with the following command:
kubectl config view --minify -o 'jsonpath={.clusters[0].cluster.server}'To get the secret object.
Kubernetes 1.22+
Replace
service-account-namewith your account name.kubectl get secret -n <namespace> -o jsonpath='{.items[?(@.metadata.annotations.kubernetes\.io/service-account\.name==\"service-account-name\")]}'If you get nothing, see Manually create a long-lived API token for a ServiceAccount.
Kubernetes 1.22 and below:
- Find the service account secret name
kubectl get serviceAccounts <service-account-name> -n <namespace> -o 'jsonpath={.secrets[*].name}'- replace
<service-account-secret-name>with the value in previous command in this command
kubectl get secret <service-account-secret-name> -n <namespace> -o jsonGet the secret object using the output of the previous step.
kubectl get secret <service-account-secret-name> -n <namespace> -o jsonCopy and paste the Secret object fetched in JSON form into the Secret field.
Select Validate and create to create the Kubernetes resource.
Reference your Kubernetes resources in a pipeline
If you're using Azure Kubernetes Service and building a YAML pipeline, the easiest way to configure your pipeline is to use a template. Connect to your repository and select one of the following two Kubernetes Service options:
- Deploy to Azure Kubernetes Services template
- Deploy to Kubernetes - Review App with Azure DevSpaces
The templates let you set up Review App without needing to write YAML code from scratch or manually create explicit role bindings.
Set up Review App
In the following example, the first deployment job is run for non-PR branches and does deployments against a regular Kubernetes resource under environments. The second job runs only for PR branches and deploys against Review App resources (namespaces inside Kubernetes cluster) generated on demand. Resources get labeled with "Review" in the resource listing view of the environment. Define variables to use in the pipeline. If you use the Deploy to Azure Kubernetes Services template, these variables get defined for you.
# Build and push image to Azure Container Registry; Deploy to Azure Kubernetes Service
trigger:
- main
resources:
- repo: self
variables:
# Container registry service connection established during pipeline creation
dockerRegistryServiceConnection: '12345' # Docker service connection identifier
envName: 'myEnv' # name of your environment
imageRepository: 'name-of-image-repository' # name of image repository
containerRegistry: 'mycontainer.azurecr.io' # path to container registry
dockerfilePath: '**/Dockerfile'
tag: '$(Build.BuildId)'
imagePullSecret: 'my-app-secret' # image pull secret
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Name of the new namespace being created to deploy the PR changes.
k8sNamespaceForPR: 'review-app-$(System.PullRequest.PullRequestId)'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: Docker@2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
- upload: manifests
artifact: manifests
- stage: Production
displayName: Deploy stage
dependsOn: Build
jobs:
- deployment: Production
condition: and(succeeded(), not(startsWith(variables['Build.SourceBranch'], 'refs/pull/')))
displayName: Production
pool:
vmImage: $(vmImageName)
environment:
name: $(envName).$(resourceName)
resourceType: Kubernetes
strategy:
runOnce:
deploy:
steps:
- task: KubernetesManifest@0
displayName: Create imagePullSecret
inputs:
action: createSecret
secretName: $(imagePullSecret)
dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
- task: KubernetesManifest@0
displayName: Deploy to Kubernetes cluster
inputs:
action: deploy
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yml
$(Pipeline.Workspace)/manifests/service.yml
imagePullSecrets: |
$(imagePullSecret)
containers: |
$(containerRegistry)/$(imageRepository):$(tag)
- deployment: DeployPullRequest
displayName: Deploy Pull request
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/pull/'))
pool:
vmImage: $(vmImageName)
environment:
name: $(envName).$(resourceName)
resourceType: Kubernetes
strategy:
runOnce:
deploy:
steps:
- reviewApp: default
- task: Kubernetes@1
displayName: 'Create a new namespace for the pull request'
inputs:
command: apply
useConfigurationFile: true
inline: '{ "kind": "Namespace", "apiVersion": "v1", "metadata": { "name": "$(k8sNamespaceForPR)" }}'
- task: KubernetesManifest@0
displayName: Create imagePullSecret
inputs:
action: createSecret
secretName: $(imagePullSecret)
namespace: $(k8sNamespaceForPR)
dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
- task: KubernetesManifest@0
displayName: Deploy to the new namespace in the Kubernetes cluster
inputs:
action: deploy
namespace: $(k8sNamespaceForPR)
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yml
$(Pipeline.Workspace)/manifests/service.yml
imagePullSecrets: |
$(imagePullSecret)
containers: |
$(containerRegistry)/$(imageRepository):$(tag)
- task: Kubernetes@1
name: get
displayName: 'Get services in the new namespace'
continueOnError: true
inputs:
command: get
namespace: $(k8sNamespaceForPR)
arguments: svc
outputFormat: jsonpath='http://{.items[0].status.loadBalancer.ingress[0].ip}:{.items[0].spec.ports[0].port}'
# Getting the IP of the deployed service and writing it to a variable for posting comment
- script: |
url="$(get.KubectlOutput)"
message="Your review app has been deployed"
if [ ! -z "$url" -a "$url" != "http://:" ]
then
message="${message} and is available at $url.<br><br>[Learn More](https://aka.ms/testwithreviewapps) about how to test and provide feedback for the app."
fi
echo "##vso[task.setvariable variable=GITHUB_COMMENT]$message"
To use this job in an existing pipeline, the service connection backing the regular Kubernetes environment resource must be modified to "Use cluster admin credentials". Otherwise, role bindings must be created for the underlying service account to the Review App namespace.