Sriram Balasubramaniam's blogs

Data source bindings for TFS/VSTS REST APIs

Sriram B — Tue, 20 Feb 2018 18:45:06 GMT

Overview

Data source bindings essentially bind a drop-down input field in UI (e.g. task input) which needs to be dynamically populated with values with the corresponding REST API that needs to be invoked to fetch the list of values. In case of REST APIs supported by external VSTS services (e.g. Azure, TeamCity, BitBucket etc.), data source binding takes the endpoint Id that needs to be used to query the values & details of REST API. For e.g. AzureRmWebAppDeployment task defines data source binding referring to the above data source: { "target": "SlotName", "endpointId": "$(ConnectedServiceName)", "dataSourceName": "AzureRMWebAppSlotsId", "parameters": { "WebAppName": "$(WebAppName)", "ResourceGroupName": "$(ResourceGroupName)" }, "resultTemplate": "{\"Value\":\"{{{ #extractResource slots}}}\",\"DisplayValue\":\"{{{ #extractResource slots}}}\"}" } The following blog describes the different fields of the data source binding: https://blogs.msdn.microsoft.com/sriramb/2016/09/15/service-endpoints-data-sources/ However, there are scenarios where inputs would need to be populated with values obtained from VSTS REST APIs. For e.g. getting a list of VSTS Build Definitions, VSTS feeds/packages etc. These queries do not require an endpoint to be created and specified in the data source binding. In order to support such queries, data source bindings support taking endpointId in the format: tfs:{service} For e.g. DownloadPackage task defines the following dataSourceBinding: { "target": "feed", "endpointId": "tfs:feed", "endpointUrl": "{{endpoint.url}}/_apis/packaging/feeds", "resultSelector": "jsonpath:$.value[*]", "resultTemplate": "{ \"Value\" : \"{{{id}}}\", \"DisplayValue\" : \"{{{name}}}\" }" },

Supported VSTS Services

Below are the set of VSTS services we currently support within dataSourceBindings: tfs:teamfoundation – Any micro service hosted within TFS (e.g. Build, Test etc.) tfs:packaging – Packaging service tfs:feed – Feed service tfs:rm – Release Management service Note that data source bindings using these services will work seamlessly in TFS as well as VSTS. Support for tfs:packaging & tfs:feed is added with TFS 2018 Release. Support for tfs:rm is added with TFS 2018 Update 2 Release. VSTS REST APIs are documented here: https://docs.microsoft.com/en-us/rest/api/vsts/

Creating a custom service endpoint

Sriram B — Fri, 27 Oct 2017 17:45:53 GMT

Overview

“Service endpoints - Customization” blog details how a custom endpoint can be registered with TFS/VSTS. This blog explains how a new endpoint for the custom endpoint type can be created taking Azure Classic endpoint type as an example. Azure Classic endpoint is an endpoint type contribution that is part of the Azure extension. Once an endpoint type is installed on a TFS collection or VSTS account, the endpoint type can be queried from the collection using a REST API. Here’s how Azure Classic endpoint type can be queried: https://.visualstudio.com/_apis/distributedtask/serviceendpointtypes?type=azure The response for the query is captured here.

Creating custom endpoint through UI

The endpoint creation UI is driven out of the service endpoint type response got by querying the above REST API. For e.g. when “Azure Classic” endpoint type needs to be created, the following UI is presented upon clicking “New Service Endpoint” -> “Azure Classic” from the endpoint UI menu:

UI elements:

Authentication schemes

The radio button group on the top indicates the authentication schemes supported by the endpoint type. Since Azure Classic endpoint supports Credentials & Azure Certificate authentication schemes, we see two options. UI uses the display names of the authentication scheme specified in the contribution:

… "authenticationSchemes": [ { … "scheme": "UsernamePassword", "displayName": "Credentials", }, { … "scheme": "Certificate", "displayName": "Certificate Based", } Service endpoints – Authentication schemes blog details the list of authentication schemes that are currently supported.

Connection name

Connection name is a mandatory input required for any endpoint type.

Endpoint data

Below are the additional data that are specific to Azure Classic endpoint:

Environment is defined as an input with inputMode=combo & with a given set of values.

{ "id": "environment", "name": "Environment", "description": "Microsoft Azure Environment for the subscription", "type": null, "properties": null, "inputMode": "combo", "isConfidential": false, "useInDefaultDescription": false, "groupName": null, "valueHint": null, "validation": { "dataType": "string", "maxLength": 300 }, "values": { "inputId": "environmentValues", "defaultValue": "AzureCloud", "possibleValues": [ { "value": "AzureCloud", "displayValue": "Azure Cloud" }, { "value": "AzureChinaCloud", "displayValue": "Azure China Cloud" }, { "value": "AzureUSGovernment", "displayValue": "Azure US Government" }, { "value": "AzureGermanCloud", "displayValue": "Azure German Cloud" } ] } The combo control shows the display value for each item in the values list. 2. Subscription Id is defined as input with inputMode=textbox with validation that it should be a GUID: { "id": "subscriptionId", "name": "Subscription Id", "description": "Subscription Id from the publish settings file", "type": null, "properties": null, "inputMode": "textBox", "isConfidential": false, "useInDefaultDescription": false, "groupName": null, "valueHint": null, "validation": { "dataType": "guid", "isRequired": true, "maxLength": 38 } }, Description of the input is used for showing help information on the input. The description can be a HTML mark-up. 3. Subscription name is a string with validation that it can be of maximum 255 characters length. "id": "subscriptionName", "name": "Subscription Name", "description": "Subscription Name from the publish settings file", "type": null, "properties": null, "inputMode": "textBox", "isConfidential": false, "useInDefaultDescription": false, "groupName": null, "valueHint": null, "validation": { "dataType": "string", "isRequired": true, "maxLength": 255 }

Authentication parameters

For “Credentials” authentication scheme, below are the parameters used:

Username input corresponds to input defined as part of the authentication scheme. "inputDescriptors": [ { "id": "username", "name": "Username", "description": "Specify a work or school account (for example @fabrikam.com). Microsoft accounts (for example @live or @hotmail) are not supported. Not recommended if Multi-Factored Authentication is enabled.", "type": null, "properties": null, "inputMode": "textBox", "isConfidential": false, "useInDefaultDescription": false, "groupName": "AuthenticationParameter", "valueHint": null, "validation": { "dataType": "string", "isRequired": true, "maxLength": 300 } },

Since “isConfidential” is set to “false”, the value of this input does not get masked in the UI. 2. Password input also corresponds to the input defined as part of the authentication scheme. { "id": "password", "name": "Password", "description": "Password for connecting to the endpoint", "type": null, "properties": null, "inputMode": "passwordBox", "isConfidential": true, "useInDefaultDescription": false, "groupName": "AuthenticationParameter", "valueHint": null, "validation": { "dataType": "string", "isRequired": true, "maxLength": 300 } } Since “isConfidential” is set to “true” the value of this field gets masked in the UI. In case “Certificate” authentication scheme is chosen, below are the inputs seen in the UI: For “Certificate” authentication scheme, the only input required is the “Management Certificate”: { "id": "certificate", "name": "Management Certificate", "description": "Management Certificate from the publish settings file", "type": null, "properties": null, "inputMode": "textArea", "isConfidential": true, "useInDefaultDescription": false, "groupName": "AuthenticationParameter", "valueHint": null, "validation": { "dataType": "string", "isRequired": true } } For the certificate, the inputMode is set to textArea to accommodate larger text to be entered.

Help link

The markdown specified in the endpoint type is used to populate the help link in the endpoint UI: "helpMarkDown": "For certificate: download publish settings file. Learn More",

Icon

The icon for the service endpoint type specified in the contribution shows up next to the created endpoint in the list view on the left pane of endpoint UI. "iconUrl": https://.visualstudio.com/_apis/public/Extensions/ms.vss-services-azure/16.125.0.938080564/Assets/icons/azure-endpoint-icon.png

Verify connection

Once the details for the endpoint are entered, we allow verifying the details if the endpoint type supports a data source named “TestConnection”. In case of Azure RM, here is how the data source looks like: "dataSources": [ { "name": "TestConnection", "endpointUrl": "$(endpoint.url)/$(endpoint.subscriptionId)/services/WebSpaces?properties=georegions", "resourceUrl": "", "resultSelector": "xpath://Name", "headers": [] }, Verify connection will succeed if the details entered are valid. In case any error is encountered when validating, the error will be displayed. But failure in verifying connection will not block creation of the endpoint.

Creating endpoint through REST API

In case there is a need to create endpoint of a type directly using API, it can be done by constructing the endpoint request body corresponding to the inputs specified by the endpoint type. For example, here is the request/response for creating Azure Classic service endpoint type using Certificate authentication scheme: REST API: https:////_apis/distributedtask/serviceendpoints Method: POST Request body: { "id": “<>”, "description": "", "administratorsGroup": null, "authorization": { "parameters": { "certificate": “<>” }, "scheme": "Certificate" }, "createdBy": null, "data": { "environment": "<>", "subscriptionId": "<>", "subscriptionName": "<>" }, "name": "<>", "type": "azure", "url": "https://management.core.windows.net/", "readersGroup": null, "groupScopeId": null, "isReady": false, "operationStatus": null } Response body: { "data": { "environment": "<>", "subscriptionId": "<>", "subscriptionName": "<>" }, "id": "c0407398-23ea-465b-9857-a7647f924699", "name": "<>", "type": "azure", "url": "https://management.core.windows.net/", "createdBy": { "id": "<>", "displayName": "<>", "uniqueName": "<>", "url": "<>", "imageUrl": "<>"}, "description": "", "authorization": { "parameters": { "certificate": null }, "scheme": "Certificate" }, "isReady": true }

Service endpoints - Data sources

Sriram B — Thu, 15 Sep 2016 12:59:30 GMT

Specifying data sources

Overview

Service endpoints support querying data from external services through REST API. The data queried can be used to populate task input dropdowns. Endpoints can define data sources which are essentially external service’s REST API references. For e.g. Azure RM endpoint type defines the following data source: { "name": "AzureRMWebAppSlotsId", "endpointUrl": "https://management.azure.com/subscriptions/$(endpoint.subscriptionId)/resourceGroups/$(ResourceGroupName)/providers/Microsoft.Web/sites/$(WebAppName)/slots?api-version=2015-08-01", "resultSelector": "jsonpath:$.value[*].id" } URL for the data source can include endpoint.* variables. These correspond to the inputs in the endpoint data. For e.g. subscriptionId is an additional input defined in the Azure RM endpoint type: { "id": "subscriptionId", "name": "i18n:Subscription Id", "description": "i18n:Subscription Id from the publish settings file", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "guid", "maxLength": 38 } }, In addition to the endpoint inputs, endpoint.Url is also an available variable. URL can also include variables that correspond to other task inputs. Variable value resolves to the value of the corresponding task input. For e.g. ResourceGroupName is an input defined in the AzureRMWebAppDeployment task: { "name": "ResourceGroupName", "type": "pickList", "label": "Resource Group", "defaultValue": "", "required": true, "properties": { "EditableOptions": "True" }, "helpMarkDown": "Enter or Select the Azure Resource Group that contains the AzureRM Web App specified above.", "visibleRule": "DeployToSlotFlag = true" }, In addition to defining the URL for the data source, a result selector can also be defined. In the example above, result selector is a JPATH query which gets applied on each of the elements in the the REST API response data. Result selector can also be an XPATH query. For e.g. Azure Classic endpoint type supports data sources that use XPATH query. { "name": "AzureWebSiteNames", "endpointUrl": "$(endpoint.url)/$(endpoint.subscriptionId)/services/webspaces/$(WebSiteLocation)Webspace/sites", "resultSelector": "xpath://Site/Name" },

Data source bindings

In order to refer to data sources defined by endpoint type in tasks, data source bindings are used. For e.g. AzureRmWebAppDeployment task defines data source binding referring to the above data source: { "target": "SlotName", "endpointId": "$(ConnectedServiceName)", "dataSourceName": "AzureRMWebAppSlotsId", "parameters": { "WebAppName": "$(WebAppName)", "ResourceGroupName": "$(ResourceGroupName)" }, "resultTemplate": "{\"Value\":\"{{{ #extractResource slots}}}\",\"DisplayValue\":\"{{{ #extractResource slots}}}\"}" } Target indicates the task input that this data source binding will bind to. For e.g. AzureRmWebAppDeployment task takes the following task input that corresponds to the slot name drop down: { "name": "SlotName", "type": "pickList", "label": "Slot", "defaultValue": "", "required": true, "properties": { "EditableOptions": "True" }, "helpMarkDown": "Enter or Select an existing Slot other than the Production slot.", "visibleRule": "DeployToSlotFlag = true" }, EndpointId contains the service endpoint ID to be used in the data source binding. It typically corresponds to the service endpoint task input defined in the task. For e.g. this is the input that corresponds to the service endpoint in case of AzureRmWebAppDeployment task: { "name": "ConnectedServiceName", "type": "connectedService:AzureRM", "label": "AzureRM Subscription", "defaultValue": "", "required": true, "helpMarkDown": "Select the Azure Resource Manager subscription for the deployment." }, Here the task defines endpoint input of type Azure RM. If endpoint type supports multiple authentication schemes, we can even scope down the endpoint selection to specific authentication schemes. For e.g. this is the input that corresponds to the service endpoint in case of AzureRmWebAppDeployment task: { "name": "ConnectedServiceName", "type": "connectedService:Azure:Certificate,UsernamePassword", "label": "Azure Subscription (Classic)", "defaultValue": "", "required": true, "helpMarkDown": "Azure Classic subscription to target for deployment." }, DatasourceName in the data source binding corresponds to the data source in the endpoint type. In case the endpoint type does not define data source that the task is interested in, then there is provision to define the details of data source inline as part of the data source binding. Here is how the data source corresponding to AzureRMWebAppSlotsId looks like: { "name": "AzureRMWebAppSlotsId", "endpointUrl": "https://management.azure.com/subscriptions/$(endpoint.subscriptionId)/resourceGroups/$(ResourceGroupName)/providers/Microsoft.Web/sites/$(WebAppName)/slots?api-version=2015-08-01", "resultSelector": "jsonpath:$.value[*].id" } Parameters represent the key/value pairs used when resolving the data source URL. In addition to endpoint.* variables, there could be variables used in the URL that correspond to the other task inputs. The name/value for these variables are provided by the parameters field in the data source binding. For e.g. WebAppName & ResourceGroupName parameters correspond to the task inputs for AzureRmWebAppDeployment task: { "name": "WebAppName", "type": "pickList", "label": "Web App Name", "defaultValue": "", "required": true, "properties": { "EditableOptions": "True" }, "helpMarkDown": "Enter or Select the name of an existing AzureRM Web Application." }, … { "name": "ResourceGroupName", "type": "pickList", "label": "Resource Group", "defaultValue": "", "required": true, "properties": { "EditableOptions": "True" }, "helpMarkDown": "Enter or Select the Azure Resource Group that contains the AzureRM Web App specified above.", "visibleRule": "DeployToSlotFlag = true" }, ResultTemplate defines a template that represents how the data has to be transformed before returning back. The template essentially is a Mustache template & can include Mustache handlers natively supported. Here’s sample response from AzureRMWebAppSlotsId data source: { "value": [ { "id": "/subscriptions/c94bda7a-0577-4374-9c53-0e46a9fb0f70/resourceGroups/webapprgpt/providers/Microsoft.Web/sites/nodewebapp123/slots/Slot1", "name": "nodewebapp123/Slot1", "type": "Microsoft.Web/sites/slots", "location": "South Central US", "tags": null, "properties": { … } }, { "id": "/subscriptions/c94bda7a-0577-4374-9c53-0e46a9fb0f70/resourceGroups/webapprgpt/providers/Microsoft.Web/sites/nodewebapp123/slots/Slot2", "name": "nodewebapp123/Slot2", "type": "Microsoft.Web/sites/slots", "location": "South Central US", "tags": null, "properties": { … } }, { "id": "/subscriptions/c94bda7a-0577-4374-9c53-0e46a9fb0f70/resourceGroups/webapprgpt/providers/Microsoft.Web/sites/nodewebapp123/slots/Slot3", "name": "nodewebapp123/Slot3", "type": "Microsoft.Web/sites/slots", "location": "South Central US", "tags": null, "properties": { … } }, ], "nextLink": null, "id": null } Here’s the result after applying the data source binding corresponding to Slot Name: { "count":3, "value":[ "{\"Value\":\"Slot1\",\"DisplayValue\":\"Slot1\"}", "{\"Value\":\"Slot2\",\"DisplayValue\":\"Slot2\"}", "{\"Value\":\"Slot3\",\"DisplayValue\":\"Slot3\"}" ] }

Test service endpoint

To avoid creating or updating a service endpoint with incorrect values for the inputs, we support a “Test” action in UI. Upon choosing to “Test” endpoint, we internally invoke query on a data source with a specific name – TestConnection. For e.g. Azure RM endpoint type defines the following TestConnection data source: { "name": "TestConnection", "endpointUrl": "https://management.azure.com/subscriptions/$(endpoint.subscriptionId)/providers/Microsoft.Web/sites?api-version=2015-08-01", "resultSelector": "jsonpath:$.value[*].name" }, If this data source isn’t present in the endpoint type, then testing the connection isn’t supported. resultSelector in TestConnection is optional and is not used to determine if the test succeeded or failed. When testing the connection using this datasource, we only check for the HTTP status code of the underlying REST API call. If HTTP status code == OK then the test will succeed.

Further References

Service endpoints - Overview Service endpoints – Authentication schemes Service endpoints - Customization MD version of this blog

Service endpoints - Customization

Sriram B — Thu, 15 Sep 2016 12:06:57 GMT

Customizing service endpoints

Overview

Basic goal for customizing service endpoint is to allow extensibility of endpoint type. This extensibility serves the following purposes:

Choosing appropriate authentication scheme(s) for the endpoint type.
Inputs/URL for the endpoint type can be customized to help user when creating endpoint.
Custom icon for the endpoint type can be provided to help differentiate the type among various other types.
Tasks can refer to appropriate endpoint type and help user in selecting endpoint relevant to the task.
Custom authorization headers can be specified when using the endpoint from TFS/VSTS to query the external service.
Data sources can be specified within the endpoint type so that any task can refer to it in order to populate drop downs.

The following write-up captures the steps involved in creating a custom endpoint type contribution and referring it within a custom task: https://www.visualstudio.com/en-us/docs/integrate/extensions/develop/service-endpoints

Choosing authentication scheme(s)

Endpoint type can choose to specify one or more authentication schemes that we currently support. Service endpoints – Authentication schemes blog details the list of authentication schemes that are currently supported. For e.g. External TFS endpoint type specifies the following authentication schemes: "authenticationSchemes": [ { "type": "ms.vss-endpoint.endpoint-auth-scheme-basic" }, { "type": "ms.vss-endpoint.endpoint-auth-scheme-token", ...

Customizing inputs

Additional inputs

Endpoint type can include inputs in addition to name, URL and the auth. parameters. For e.g. Azure Classic endpoint type includes subscription ID and subscription name inputs: { "id": "subscriptionId", "name": "i18n:Subscription Id", "description": "i18n:Subscription Id from the publish settings file", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "guid", "maxLength": 38 } }, { "id": "subscriptionName", "name": "i18n:Subscription Name", "description": "i18n:Subscription Name from the publish settings file", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string", "maxLength": 255 } }

Supported input types

Text box, Text area, Combo inputs are supported. When specifying combo input, the set of possible values & default value can be provided. For e.g. Azure Classic endpoint uses environment combo input: { "id": "environment", "name": "i18n:Environment", "description": "i18n:Windows Azure Environment for the subscription", "inputMode": "combo", "isConfidential": false, "validation": { "isRequired": false, "dataType": "string", "maxLength": 300 }, "values": { "inputId": "environmentValues", "defaultValue": "AzureCloud", "possibleValues": [ { "value": "AzureCloud", "displayValue": "Azure Cloud" }, { "value": "AzureChinaCloud", "displayValue": "Azure China Cloud" }, { "value": "AzureUSGovernment", "displayValue": "Azure US Government" }, { "value": "AzureGermanCloud", "displayValue": "Azure German Cloud" } ] } }

Hiding auth. parameter input

Auth. parameter input can be hidden if required. For e.g. HockeyApp endpoint type hides user name field although it uses basic authentication scheme. { "id": "username", "name": "Username", "description": "Username", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": false, "dataType": "string" }, "values": { "inputId": "usernameInput", "defaultValue": "", "isDisabled": true } },

Overriding auth. parameter input field name

Input field name of the auth. parameter input can be overridden if required. For e.g. Chef endpoint overrides the input field names for basic auth. scheme: "authenticationSchemes": [ { "type": "ms.vss-endpoint.endpoint-auth-scheme-basic", "inputDescriptors": [ { "id": "username", "name": "i18n:Node Name (Username)", "description": "i18n:Chef username", "_description.comment": "Don't Localize the word 'Chef'", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string" } }, { "id": "password", "name": "i18n:Client Key", "description": "i18n:Client key specified in the .pem file", "inputMode": "textarea", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string" } } ] }

Help text for input

Help text can be provided for inputs & they can include hyperlinks as well. For e.g. subscription name input of Azure Classic endpoint type has customized help: { "id": "subscriptionName", "name": "i18n:Subscription Name", "description": "i18n:Subscription Name from the publish settings file", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string", "maxLength": 255 } }

Customizing URL

Default value for URL

URL is a mandatory field to be provided when creating a service endpoint. In some cases, it would be redundant/unnecessary to expect user to provide value for URL. For e.g. URL for Azure Classic endpoint when using Azure public cloud would always be https://management.core.windows.net/. In such cases, the endpoint type can define it as default value. "url": { "displayName": "i18n:Server Url", "value": "https://management.core.windows.net/", "isVisible": "true",

Hiding URL field

In cases where there is no requirement for user to override default value of URL provided in the endpoint type, the URL field can be completely hidden. This can be achieved by defining “isVisible” property of the Url as “false”. NOTE: To support backward compatibility, if isVisible property is not specified, it is assumed to be false & the URL is hidden. For e.g. HockeyApp endpoint type hides the URL value when creating service endpoint: "url": "https://rink.hockeyapp.net/api/2/apps/",

Help text for URL

Help text can be provided for inputs & they can include hyperlinks as well. For e.g. Azure service fabric endpoint type defines help for the URL: "url": { "displayName": "i18n:Cluster Endpoint", "required": true, "helpText": "i18n:Client connection endpoint for the cluster. Prefix the value with "https://"." }

Dependency on another input

URL can be dependent on value of another input field in the endpoint type. As and when the value of the input changes, URL value will also get updated in the endpoint creation UI. For e.g. Azure Classic endpoint type’s URL depends on “environment” input. A map of the “environment” to “URL” values is defined in the endpoint type & based on this, the value of URL gets populated: "url": { "displayName": "i18n:Server Url", "value": "https://management.core.windows.net/", "isVisible": "true", "helpText": "", "dependsOn": { "input": "environment", "map": [ { "key": "AzureCloud", "value": "https://management.core.windows.net/" }, { "key": "AzureChinaCloud", "value": "https://management.core.chinacloudapi.cn/" }, { "key": "AzureUSGovernment", "value": "https://management.core.usgovcloudapi.net/" }, { "key": "AzureGermanCloud", "value": "https://management.core.cloudapi.de/ " } ] }

Customizing help text for the endpoint type

Help text including help links can be specified for the endpoint type that appears in the bottom of the create/update endpoint UI For e.g. Azure Classic endpoint type provides custom help: { "id": "azure-endpoint-type", … "properties": { "name": "azure", … "helpMarkDown": "i18n:For certificate: download publish settings file. For Service Principal: refer to link. Learn More",

Customizing endpoint type icon

Endpoint type will use the icon of the extension that it is part of by default. For e.g. HockeyApp endpoint type uses the extension’s icon: "icons": { "default": "images/hockeyapp-logo.png", "large": "images/hockeyapp-logo-large.png" }, If required, an icon specific to the endpoint type can be defined & this will override the extension’s icon when displayed. For e.g. Azure extension supports an icon but Azure Service Fabric endpoint overrides the icon: "icons": { "default": "icons/vss_default.png", "wide": "icons/vss_wide.png" }, "files": [ { "path": "icons/azure-service-fabric-icon.png", "addressable": true }, { "path": "icons/azure-endpoint-icon.png", "addressable": true } ], … { "id": "servicefabric-endpoint-type", … "properties": { "name": "servicefabric", … "icon": "icons/azure-service-fabric-icon.png"

Customizing auth. header

Different external services expect the auth. scheme parameters to be passed differently in the HTTP requests. For TFS/VSTS to be able to communicate with the external service to query values to populate in task input drop downs, auth. header has to be constructed as per how the external service expects. Auth. header to specify in the HTTP request can be customized when defining the endpoint type contribution. For e.g. Bitbucket endpoint type specifies the following custom auth. header: "authenticationSchemes": [ { "type": "ms.vss-endpoint.endpoint-auth-scheme-token", "headers": [{"name": "Authorization", "value": "Basic {{ #base64 endpoint.username \":\" endpoint.apitoken }}"}], Mustache evaluation is supported when constructing the auth. header. #base64 is a supported Mustache handler that computes Base64 encoded value of the (username:apitoken) combination. Auth. parameters in the endpoint can be referred in the expression using endpoint.* notation & these get resolved with the values in the endpoint used in the task when evaluating the expression.

Further References

Service endpoints - Overview Service endpoints – Authentication schemes Service endpoints – Data sources MD version of this blog

Service endpoints - Authentication schemes

Sriram B — Wed, 14 Sep 2016 17:18:58 GMT

Authentication scheme in a service endpoint determines the credentials that would be used to connect to the external service. In order to populate task drop downs, TFS/VSTS connects to the external service using the credentials provided as part of the endpoint. TFS/VSTS effectively becomes a client of the external service querying for details pertaining to the task input. For TFS/VSTS to be able to connect to the external service, in addition to using the credentials, there is also need to know how to set the credentials in the HTTP request header when calling the external endpoint. TFS/VSTS supports a closed set of authentication schemes that can be utilized by a custom service endpoint type. This set is closed so that VSTS/TFS would be able to interpret the auth. scheme used injso any custom endpoint & support connecting to the external service. Following are the authentication schemes that are part of the closed set:

Basic authentication

"id": "endpoint-auth-scheme-basic", "description": "Basic Authentication based endpoint authentication scheme", "type": "ms.vss-endpoint.service-endpoint-auth-scheme", "targets": [ "ms.vss-endpoint.endpoint-auth-schemes" ], "properties": { "name": "UsernamePassword", "displayName": "i18n:Basic Authentication", "headers": [ { "name": "Authorization", "value": "Basic {{ #base64 endpoint.username \":\" endpoint.password }}" } ], "inputDescriptors": [ { "id": "username", "name": "i18n:Username", "description": "i18n:Username for connecting to the endpoint", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string", "maxLength": 300 } }, { "id": "password", "name": "i18n:Password", "description": "i18n:Password for connecting to the endpoint", "inputMode": "passwordbox", "isConfidential": true, "validation": { "isRequired": true, "dataType": "string", "maxLength": 300 } } ] } This scheme takes 2 inputs – Username & Password (confidential) Default auth. header used is: "Basic {{ #base64 endpoint.username \":\" endpoint.password }}"

Token based authentication

"id": "endpoint-auth-scheme-token", "description": "i18n:Token based endpoint authentication scheme", "type": "ms.vss-endpoint.service-endpoint-auth-scheme", "targets": [ "ms.vss-endpoint.endpoint-auth-schemes" ], "properties": { "name": "Token", "displayName": "i18n:Token Based Authentication", "headers": [ { "name": "Authorization", "value": "{{endpoint.apitoken}}" } ], "inputDescriptors": [ { "id": "apitoken", "name": "i18n:API Token", "description": "i18n:API Token for connection to endpoint", "inputMode": "textbox", "isConfidential": true, "validation": { "isRequired": true, "dataType": "string", "maxLength": 300 } } ] } This scheme takes 1 input – API Token (confidential) Default auth header used is: {{endpoint.apitoken}}

Certificate based authentication

"id": "endpoint-auth-scheme-cert", "description": "i18n:Creates a certificate-based endpoint authentication scheme", "type": "ms.vss-endpoint.service-endpoint-auth-scheme", "targets": [ "ms.vss-endpoint.endpoint-auth-schemes" ], "properties": { "name": "Certificate", "displayName": "i18n:Certificate Based", "inputDescriptors": [ { "id": "certificate", "name": "i18n:Certificate", "description": "Content of the certificate", "inputMode": "TextArea", "isConfidential": true, "validation": { "isRequired": true, "dataType": "string" } } ] } This scheme takes 1 input – Certificate (confidential) The value of certificate has to be provided in the text area.

No authentication

"id": "endpoint-auth-scheme-none", "description": "i18n:Creates an endpoint authentication scheme with no authentication.", "type": "ms.vss-endpoint.service-endpoint-auth-scheme", "targets": [ "ms.vss-endpoint.endpoint-auth-schemes" ], "properties": { "name": "None", "displayName": "i18n:No Authentication" } This scheme is used when an endpoint type does not require to take any input. For e.g. external services that support anonymous access to its resources.

JSON web token based OAUTH authentication

"id": "endpoint-auth-scheme-JWT", "description": "i18n:Endpoint authentication scheme to support OAUTH using JSON Web token", "type": "ms.vss-endpoint.service-endpoint-auth-scheme", "targets": ["ms.vss-endpoint.endpoint-auth-schemes"], "properties": { "name": "JWT", "displayName": "i18n:JSON Web Token based authentication", "inputDescriptors": [ { "id": "Issuer", "name": "i18n:Issuer", "description": "i18n:Issuer for creating JWT", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": false, "dataType": "string", "maxLength": 300 } }, { "id": "Audience", "name": "i18n:Audience", "description": "i18n:Audience for creating JWT", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": true, "dataType": "string", "maxLength": 300 } }, { "id": "Scope", "name": "i18n:Scope", "description": "i18n:Scope to be provided", "inputMode": "textbox", "isConfidential": false, "validation": { "isRequired": false, "dataType": "string", "maxLength": 300 } }, { "id": "PrivateKey", "name": "i18n:Private Key", "description": "i18n:Private Key for connecting to the endpoint", "inputMode": "textbox", "isConfidential": true, "validation": { "isRequired": true, "dataType": "string", "maxLength": 2000 } } ] } This authentication scheme takes 4 inputs – Issuer, Audience, Scope, PrivateKey. The following processing is done in order to generate auth. header for this authentication scheme:

Create JSON web token using the issuer, audience & scope provided. Scope is added as additional claim in the token.
PrivateKey is used to populate the signature in the token. It is expected to be in PEM format.
POST call is made to the audience with the generated token as content & the response of the call is taken as the bearer’s access token in the auth. header. (Bearer )

Azure specific authentication schemes

In addition to the above schemes, Azure certificate & Azure service principal based authentication schemes are supported which are used in the Azure Classic and Azure RM endpoint types respectively.

Further References

Service endpoints - Overview Service endpoints - Customization Service endpoints – Data sources MD version of this blog

Service endpoints - Overview

Sriram B — Wed, 14 Sep 2016 17:03:56 GMT

Purpose

TFS/VSTS supports Continuous Integration (CI) & Continuous Deployment (CD) scenarios involving external services:

Building source code from various external source repo providers like GitHub, SVN etc.
Consuming build artifacts from various external services like Jenkins, TeamCity etc. within release.
Deploying applications to various external services like Azure, Chef, SCVMM, Docker etc. as part of release.

Service endpoints is used to light up CI/CD scenarios involving external services. Service endpoints serves the following purposes:

Stores details related to external service that can include secrets as well.
Allows querying the external service in a secure manner and use the results to enhance CI/CD user experience.

Basic structure of service endpoint

Details that a service endpoint typically captures include the following:

Name of the endpoint using which it can be referred to later
Type of the endpoint
URL to connect to the external service
Authentication scheme to use (e.g. Basic authentication, Certificate based authentication etc.)
Authentication scheme parameters (User name, Password, Certificate etc.)

Single endpoint type can support multiple authentication schemes. There is support for providing additional details specific to endpoint type beyond the above. How additional details can be included is covered in section below. Authentication parameters provided as part of endpoint are stored securely and is never returned back from the service. Service endpoints are stored within individual team projects.

Creating service endpoint

Service endpoint can be created through the following UI:

Managing security on service endpoints

Security can be applied on endpoints to control access to them. Endpoints support security 2 role types: Administrator or User. Administrator of endpoint can essentially manage the endpoint (update/delete, set permissions) and also be able to use the endpoint within build definitions or release definitions. User of endpoint can only use the endpoint within build definitions or release definitions. Each project has 2 groups – Endpoint Administrators & Endpoint Creators. Endpoint Administrators group is added as an Administrator role on each endpoint by default. Project Administrators are added to the Endpoint Administrators group in every project by default. Endpoint Creators group control who can create endpoint within the project. Contributors are added to the Endpoint Creators group in every project by default.

REST APIs

Service endpoints support REST APIs to be able to work with them. The set of supported REST APIs is documented here: https://www.visualstudio.com/en-us/docs/integrate/api/endpoints/overview

Using service endpoints in tasks

Populating task drop downs

Endpoints created within the project can later be referenced in tasks within Build or Release definition. Tasks can define an endpoint input & additional inputs that are essentially drop-downs that get populated by querying external services using the endpoint input. Here are references to some tasks that use endpoints: https://github.com/Microsoft/vsts-tasks/blob/ed7d8ec751176d75c83132f79e9ad6eae1109377/Tasks/SqlAzureDacpacDeployment/task.json https://github.com/Microsoft/vsts-tasks/blob/2cb2a67ee75be8ecc75e155d99d55d33aa364b40/Tasks/AzureCLI/task.json

Using endpoints in task implementation

During runtime, TFS/VSTS passes down the endpoint information down to the agent and makes it available to tasks that use them. vsts-task-lib contains helper functions to let tasks query endpoint information:

For PS based tasks, Get-VstsEndpoint commandlet can be used to retrieve endpoint details.
For Node based tasks, task.getEndpointUrl & task.getEndpointAuthorization helpers can be used to retrieve endpoint details.

Here are references to task implementations that consume endpoints: https://github.com/Microsoft/vsts-tasks/blob/ed7d8ec751176d75c83132f79e9ad6eae1109377/Tasks/SqlAzureDacpacDeployment/Utility.ps1 https://github.com/Microsoft/vsts-tasks/blob/2cb2a67ee75be8ecc75e155d99d55d33aa364b40/Tasks/AzureCLI/azureclitask.ts

Further references

Service endpoints – Authentication schemes Service endpoints - Customization Service endpoints – Data sources

Hello world!

Sri Divya — Fri, 09 Sep 2016 12:44:54 GMT

Welcome to Developer Network. This is your first post. Edit or delete it, then start blogging!