Terraform Onboarding My Shiny New Config

So my first blog post here was mainly there as the first idea I had had for a blog. Logically I probably should have started with this one which details the journey I have been on to onboard my shiny new domain, ADO env and azure account in to Terraform. My aim here is to not let it all descend in to a GUI nightmare and to codify all the things because, well, IaC and automation are bloody brilliant.

Tricky Parts

Much as I love Azure I have a slight frustration with it in terms of being heavily GUI driven. Overall it seems much more forgiving than AWS but it does sometimes get in the way a bit. The prime example of this is with Subscriptions. When you onboard a shiny new trial you automagically have a subscription spat out for you. This is nice but it has a horrible name - which needs 10 minutes to sync changing - and it also does not have an alias which is a key bit of metadata for terraform to handle managing this subscription.

Service Principals also suffer a similar "hand holdy" approach. I need a service account to run my terraform and to use in automation. Hit up the docs and find the powershell commands like so:

$sp = New-AzADServicePrincipal -DisplayName <service_principal_name> -Role "Contributor"

Simples. Alas this has actually created several things - a service principal, an Enterprise Application and an AzureRM Role Assignment scoped to the subscription. These are all things that will need to be proted to code and now they exist imported in to terraform state. Still it could be worse. Somehow.

Learn to embrace change and roll with it

Despite my misgivings we have to persevere :) Knowing how terraform works there is nothing stopping us importing these resources and as long as what we write as desired state matches what is imported to state then we can carry on our way. To that end I fashioned a terraform module to handle my subscription. Initially this is as barre bones as it gets and contains the following:

the subscription entity
the Enterprise Application and the Service Principal
a password rotation mechanism for this
a Service Connection within ADO (where all my code will be run from eventually).

It may seem like a lot of effort to make this a module but I feel this will server as the cornerstone of the environemnt going forward. When I need tags or a budget definining I can add it in to the module code base and if there are multiple subscriptons then they will all benefit form this. They will all end up the same which is good (and the overal dream of running infrastructure as code).

Problem 1 - the alias problem

As mention when you create a subscription in Azure non-promgramatically it ends up without an alias. Terraform requires this to operate against (or more specifically the Azure Subscription API design requires this). The Azure Powershell module can handle this thankfully so I can keep all the onboarding in a single entity:

new-azsubscriptionAlias -AliasName "<Sub_Aias_Name>" -SubscriptionId "00000000-0000-0000-0000-000000000000"

Or more specifically:

$azSubName = "CalFinnIO"

Connect-AzAccount

$azCtx = Set-AzContext -Subscription $azSubName
$azSubObj = Get-AzSubscription -SubscriptionName $azSubName

$azSubAlias = new-azsubscriptionAlias -AliasName $($azSubName.ToLower()) -SubscriptionId $azSubObj.SubscriptionId

Problem 2 - lots of objects form a single command

The cmdlet New-AzADServicePrincipal seems straightforward but is actually doing several different things behind the scenes. We end up with the following resources:

An Azure Service Principal
An Azure AD Registered App
An Azure Role Assignment scoped to our context.

To match up with the terraform code therefore we need to import these entities in to teh state. Luckily this is all available to us via the powershell cmdlets. The return of New-AzADServicePrincipal provides us with the AAD Registered App Id so we can shovel that in to Get-AzADApplication to find the ObjectId needed to import that in to state as an "azuread_application". We can then filter the Role Assignments by the Service Principal Id and return the correct assignment data needed to tell terraform what to read. This looks like this:

$sp = New-AzADServicePrincipal -DisplayName "sp_sub_$($azSubName.ToLower()) -Role "Contributor"
$adApp = Get-AzADApplication -ApplicationId $sp.AppId
$azRoleAssignment = Get-AZRoleAssignment -ObjectId $sp.Id

Problem 3 - ADO

In my haste to get things running I had gotten a bit eager and manually created the Service Connection within my ADO Project. In the end this code is all running locally to start with and the next steps will be to shovel the state up in to a storage account and run it from an ADO pipeline sourcing the module from a git repo in there. But for now I need to import what I have created so that we have a zero delta plan at the end.

Annoyingly, Powershell and ADO are not the best friends - there is no out the box cmdlet set published. There are various projects out there providing the functionality but I have always found them to be missing the bits I want/need. Either way there is a fairly decent REST API present.

All we need to do is plumb in a PAT and starting hitting the API:

$ADOOrgName = "ADO_Org_Name"
$AzureDevOpsPAT = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
$AzureDevOpsAuthenicationHeader = @{Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($AzureDevOpsPAT)")) }

$UriOrga = "https://dev.azure.com/$($ADOOrgName)/" 
$projectUri = $UriOrga + "_apis/projects?api-version=7.0"
$project = Invoke-RestMethod -Uri $uriAccount -Method get -Headers $AzureDevOpsAuthenicationHeader

$endpointURI = $UriOrga + "$($project.value.id)/_apis/serviceendpoint/endpoints?api-version=7.0"
$endpoint = Invoke-RestMethod -Uri $endpointURI -Method get -Headers $AzureDevOpsAuthenicationHeader

This code returns us the projectID and the Service Connection endpoint ID that we can then pass to terraform.

Problem 4 - Return of the Alias

Configuring the alias and getting it imported nicely in to terraform state has gone well so far but I started getting errors in the plan. I had been using the id attribute form the subscription in various places - azurerm_subscription.this.id - and had assumed that this was the subscriptionId. It is not and is actually the alias ID. This led to various errors and a bit of headscratching on my part. The azurerm_role_assignment and the azuredevops_serviceendpoint_azurerm were both generating errors - the latter about string length and the former about scoping. A few tweaks to the module code though and sanity returns:

resource "azuredevops_serviceendpoint_azurerm" "this" {  
    project_id            = data.azuredevops_project.this.project_id  
    service_endpoint_name = local.service_connection_name  
    description           = "Managed by Terraform"  
    credentials {  
    serviceprincipalid  = azuread_service_principal.this.id  
    serviceprincipalkey = local.credential_selector ? azuread_service_principal_password.even.value : azuread_service_principal_password.odd.value  
    }  
    azurerm_spn_tenantid      = azurerm_subscription.this.tenant_id  
    **azurerm_subscription_id   = azurerm_subscription.this.subscription_id**  
    azurerm_subscription_name = var.subscription_name  
}   

And likewise the scope for the azurerm_role_assignment needed a tweak:

resource "azurerm_role_assignment" "this" {
  scope                = "/subscriptions/${azurerm_subscription.this.subscription_id}"
  role_definition_name = "Contributor"
  principal_id         = azuread_service_principal.this.id
}

The terraform code

With all the legwork in place I just needed to write the code to call the module, complete a terraform init and then run the powershell to import everything discussed above. The module block and terraform config look like this (note the local path to the terraform module for now - YMMV):

module "subscription_az_sub_name" {
    source = "../../../../tf_module_subscription"
    date = var.date
    ado_project = "ADO_PROJECT_NAME"
    subscription_name = "AZ_SUB_NAME"
    subscription_id = "XXXXXXX-XXX-XXXX-XXXX-XXXXXXXXXXXX"
    owner = "Tim.Alexander@calfinn.io"
    available_to_other_tenants = false
}

terraform {

  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.45.0"
    }
    azuread = {
      source = "hashicorp/azuread"
      version = "2.35.0"
    }
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "0.3.0"
    }
  }
}

provider "azuread" {
  # Configuration options
}

provider "azurerm" {
  # Configuration options
  features {}
}

provider "azuredevops" {
  # Configuration options
}

variable "date" {
    type = string
}

We then run a terraform init in that directory to get terraform configured. Now we can run the powershell (populating the correct variables):

$azSubName = "SubName"
$ADOOrgName = "ADoOrgName"
$AzureDevOpsPAT = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
$tfProjectPath = "<PATH TO TF PROJECT>"
Connect-AzAccount

$azCtx = Set-AzContext -Subscription $azSubName
$azSubObj = Get-AzSubscription -SubscriptionName $azSubName

$azSubAlias = new-azsubscriptionAlias -AliasName $($azSubName.ToLower()) -SubscriptionId $azSubObj.SubscriptionId

$sp = New-AzADServicePrincipal -DisplayName "sp_sub_$($azSubName.ToLower()) -Role "Contributor"
$adApp = Get-AzADApplication -ApplicationId $sp.AppId
$azRoleAssignment = Get-AZRoleAssignment -ObjectId $sp.Id


$AzureDevOpsAuthenicationHeader = @{Authorization = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$($AzureDevOpsPAT)")) }

$UriOrga = "https://dev.azure.com/$($ADOOrgName)/" 
$projectUri = $UriOrga + "_apis/projects?api-version=7.0"
$project = Invoke-RestMethod -Uri $uriAccount -Method get -Headers $AzureDevOpsAuthenicationHeader

$endpointURI = $UriOrga + "$($project.value.id)/_apis/serviceendpoint/endpoints?api-version=7.0"
$endpoint = Invoke-RestMethod -Uri $endpointURI -Method get -Headers $AzureDevOpsAuthenicationHeader


Write-Output "Importing resources to terraform state..."
Set-Location $tfProjectPath
terraform import module.subscription_az_sub_name.azuread_application.this $adApp.Id
terraform import module.subscription_az_sub_name.azuread_service_principal.this $sp.Id
terraform import module.subscription_az_sub_name.azuredevops_serviceendpoint_azurerm.this $($project.value.id)/$($endpoint.value.id)
terraform import module.subscription_az_sub_name.azurerm_subscription.this $azSubObj.Id
terraform import module.subscription_az_sub_name.azurerm_role_assignment.this $azRoleAssignment.RoleAssignmentId

This will import all the resources in to the state. We can then run a terraform plan and should see changes only to the credentials (see previous blog post).

Where next

Well the next steps are to start shunting all this code and now critical TF state up in to Azure. We can leverage the terraform module to spin out a storage account and push the state up to that. We can also push the module and the calling code up to ADO and get it pipelined to run from there.

Code

The module I referred to and an example calling it can be found on github

Tricky Parts​

Learn to embrace change and roll with it​

Problem 1 - the alias problem​

Problem 2 - lots of objects form a single command​

Problem 3 - ADO​

Problem 4 - Return of the Alias​

The terraform code​

Where next​

Code​