Neeley Ops

Create Custom Endpoint in Azure Devops with VSTeam

I’m very happy to have found the vsteam Powershell module on GitHub. At work we have a custom application to provision resource groups in Azure, but for my home projects it’s powershell all the way. I’ve seen some other modules, but vsteam has a very clean implementation. It’s great stuff.

I’ve been bootstrapping my home Azure DevOps projects and their Azure resources from within Azure DevOps. That is, I use one Azure DevOps project and infrastructure as codee to create more Azure DevOps projects within my account. It’s a work in progress, but it has helped me to better understand the permissions model of Azure DevOps and Azure too. This can be challenging when trying to enforce the principle of least priviledge.

Here’s my script for creating an Azure Resource Manager (arm or azurerm) connection that is scoped to a particular resource group. Save this file as azure/vsteamendpoint.ps1.

    .Synopsis Create an Azure App Registration and Azure Devops Service Endpoint

    .Parameter $account
    The Azure DevOps account name (also called the project collection name in some docs)

    .Parameter $project
    The Azure DevOps project name within the project collection

    .Parameter $resourceGroupName
    The name of the Azure Resource Group to which the endpoint will be scoped

    .Parameter $endpointType
    The string 'id' of the new endpoint's type in Azure Devops, see the list with Get-VsTeamEndpointTypes

    .Parameter $scope
    A custom Azure Graph scope if overriding the default resource group scope

    .Parameter $role
    A custom role given to the service principle during creation only, default is Contributor

    .Parameter $data
    A hashtable of additional Azure Devops Service Endpoint settings that will be merged in with the default set of endpoint settings. Defaults are overridden by passed in data

    .Parameter $appName
    Override the generated but readable name of the Azure App Registration as seen in Azure Portal

    .Parameter $endpointName
    Override the generated but readable name of the endpoint as seen in Azure Devops

    .Parameter $spName
    Override the generated but readable name of the Service Principal created with the App Regisration as seen in Azure Portal

    .Parameter $token
    Override the Azure DevOps Bearer token with a personal access token, useful when running outside an Azure Pipeline build

    .Parameter $ignoreEndPointCreationErrors
    Assume the endpoint was created even if it told you the build service can't access it later
    The Azure Devops REST API will not display endpoints created by a build service account when the list is requested by the build service account

param (
  $role = "Contributor",
  $data = @{},
  $appname = "$account-$endpointType-$project-application",
  $endpointName = "$endpointType-$project",
  $spname = "$appname-sp",
  $token = $env:SYSTEM_ACCESSTOKEN,
  $ignoreEndPointCreationErrors = $true

###################### Functions

function New-VSTeamInstance {
  param (
  if (!(Get-Command -module VSTeam)) {
    write-host installing VSTeam
    Install-Module VSTeam -Force
    Import-Module VSTeam -Force -NoClobber
  $accountParams = @{account=$account;token=$token}
  if ($env:SYSTEM_ACCESSTOKEN -and $token -eq $env:SYSTEM_ACCESSTOKEN) {
    write-verbose "using bearer token"
    $accountParams.UseBearerToken = $true
  Set-VSTeamAccount @accountParams
  Set-VSTeamDefaultProject $project
  $team = Get-VSTeamProject -Name $project
  if (!$team) {
    throw "Please add the project '$project' in azure devops first"

function New-VSTeamServiceEndpointSettings {
  param (
  $settings = @{
    data = @{
      SubscriptionId = $azContext.Subscription.Id
      SubscriptionName = $azContext.Subscription.Name
      environment = $azContext.Environment.Name
    authorization = @{
      parameters = @{
        tenantId = $azcontext.Tenant.Id
        servicePrincipalId = $app.ApplicationId
        servicePrincipalKey = $pass
        authenticationType = "spnKey"
        scope = $scope
      scheme = "ServicePrincipal"
    isShared = $true
  $settings, $data | Merge-Object -Strategy Override

function Get-RandomAlphanumericString {
  Param (
    [int] $length = 8
  return ( -join ((0x30..0x39) + ( 0x41..0x5A) + ( 0x61..0x7A) | Get-Random -Count $length | % {[char]$_}) )


#used for Merge-Object
if (!(Get-Command -module Functional)) {
  Install-Module Functional -Force
  Import-Module Functional -Force
#used for azure commands
if (!(Get-Command -module 'Az.*')) {
  Install-Module Az -Force
  Import-Module Az -Force
New-VSTeamInstance -account $account -project $project -token $token
$team = Get-VSTeamProject -Name $project
if (!$team) {
  throw "Please add the project '$project' in azure devops first"
$azcontext = Get-AzContext
$app = Get-AzAdApplication -DisplayName $appname
if (!$app) {
  write-host creating application $appname
  $app = New-AzADApplication -DisplayName $appname -HomePage "https://VisualStudio/SPN" -IdentifierUris "https://Visualstudio/SPN/$appname"
if (!$scope) {
  $scope = "/subscriptions/$($azcontext.Subscription.Id)/resourcegroups/$resourceGroupName"
$sp = $app | Get-AzAdServicePrincipal
if (!$sp) {
  write-host creating service principal $spname
  $sp = New-AzAdServicePrincipal -ApplicationId $app.ApplicationId -Displayname $spname -Scope $scope -Role $role
$ep = Get-VSTeamServiceEndpoint
$ep = $ep | Where-Object {$ -eq $endpointName}
if (!$ep) {
  write-host creating endpoint $endpointName
  # if we don't already have a connection, we need to create a new password for the app registration
  $passString = Get-RandomAlphanumericString -length 22
  $securepass = ConvertTo-SecureString $passString -AsPlainText -Force
  $void = $app | New-AzADAppCredential -Password $securepass -StartDate $(Get-Date) -EndDate $((Get-Date).AddDays(365))

  $epSettings = New-VSTeamServiceEndpointSettings -azcontext $azcontext -app $app -pass $passString -scope $scope -data $data

    $ep = Add-VSTeamServiceEndpoint -endpointName $endpointName -endpointType $endpointType -object $epSettings
  } catch {
  if ($_ -notmatch "already exists") {
  write-warning $_
  write-output $($_ | get-member)
  $ep = Get-VSTeamServiceEndpoint | Where-Object {$ -eq $endpointName}
  if (!$ep -and !$ignoreEndPointCreationErrors) {
      #See note in parameter description to understand why this is necessary
      throw "could not create endpoint"

Save the script below as templates/resource-group.yml. Since I use this for multiple projects, it’s in a yaml template file. I’ve left out the step that creates the resource group.

  connection: '' #name of arm connection in bootstrap project
  project: '' #name of project within azure devops collection
  region: '' #region where resources will be located
  account: '' #name of azure devops collection (<account>/)

- job:
  - task: AzurePowershell@4
    displayName: 'arm connection'
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)
      azureSubscription: ${{ parameters.connection }}
      scriptType: FilePath
      scriptPath: azure/vsteamendpoint.ps1
        -account ${{ parameters.account }}
        -project ${{ parameters.project }}
        -resourceGroupName ${{ parameters.project }}-${{ parameters.region }}
        -endpointType azurerm
      azurePowershellVersion: latestVersion

Save this file as azure-pipelines.yml. After you push this code, start a new pipeline from an existing script in Azure Devops and use this file.

- master

  vmImage: 'ubuntu-latest'

- stage: my_resource_group_and_region
  - template: templates/resource-group.yml
      connection: <put your bootstrap project's connection name here>
      project: <put the new project name here>
      region: <put the region where resources will be deployed here>
      account: <put the name of the azure devops account (project collection) here>

I’ve read various github issues, azure devops uservoice tickets, API and Graph docs to find the right set of permissions. I think I’ve got it mostly right. I’ve been keeping these notes in the README of my project. I’ve tried to update the readme every time I manually tweaked something in Azure Devops or Azure portal. Some of the steps at the end of the list are hack attempts at the issue of the azure devops build service account being unable to view endpoints that it created. That’s still unresolved.

Login to Azure Devops
Create Subscription in Azure Portal
Add Azure Active Directory to Azure Devops (org settings -> general -> azure active directory -> connect)
NOTE: This will invalidate all of your pre-existing personal access tokens...
Add bootstrap project in Azure Devops
Build a pipeline to get bootstrap build service created in Azure Devops
Add bootstrap build service to Project Collection Service Accounts in Azure Devops (it's vsts only)
Create my-bootstrap-service-connection in Azure Devops bootstrap project, it's an Azure Resource Manager connection
Find app registration for my-bootstrap-service-connection in azure portal
Rename to find easier later my-bootstrap-app
Grant 'Application Manager' to my-bootstrap-app in azure portal

#notes get fuzzier at this point, use at your own risk!
Grant 'Application.ReadWrite.OwnedBy', 'Application.Read.All', 'Application.ReadWrite.All', 'Directory.Read.All', 'Directory.ReadWrite.All' to my-bootstrap-app in azure portal (directory -> select user -> api permissions -> add permission -> microsoft graph -> 'application permissions' -> search/select 'Application.ReadWrite.OwnedBy' -> Add Permissions
Directory.AccessAsUser.All is also required, according to help docs for New-AzAdApplication. This is found under 'delegated permissions' instead of 'application permissions' in the above sequence.
Different links say different things about what is and isn't required - It would be nice if it was only 'Application.ReadWrite.OwnedBy'...
Approve admin permission grant
Grant 'User Access Administrator' to my-bootstrap-app in Subscriptions -> (subscription) -> Access control (IAM) in azure portal
very helpful:
Add project collection build service to Project Collection Service Accounts in Azure Devops (it's vsts only)
Grant Azure Active Directory (legacy) 'Application.ReadWrite.OwnedBy' api permission to my-bootstrap-app in azure portal
covers "Resource not found for the segment 'me'"
Grant Azure Active Directory (legacy) 'Directory.Read.All' api permission to my-bootstrap-app in azure portal