Terraform Output Readability

Using Terraform in a pipeline or using a script locally can be a bit overwhelming with information. A couple of filters I use in Bash are helpful to make it more readable for changes. 

Compact:

terraform plan | sed -E '/Refreshing state|Reading|Read complete/Id'"

Very compact:

terraform plan | awk 'length' | sed -E '/Refreshing state|Reading|Read complete|Plan:|\.plan|Saved the plan to|To perform exactly|Terraform has compared|found no differences|You can apply this plan to save these new output values|without changing any real infrastructure/Id'"

Azure App Service Domain Renewals

Wasn't the easiest thing to export the renewal status for Azure App Service Domains, for use in a report. So I used a bit of Powershell to extract and concatenate it into one variable. The results are combined in variable $ResultsTableArray, which can then be used for reporting. I only need the two fields of name and renewal status, you can add more if you need it!

$ResourceGroup= "<insert your resource group for domains here>"

$JSON_PATH="<insert the full path to a folder you want to use as a working area>"

Get-AzResource -ResourceGroupName "$ResourceGroup" -ResourceType "Microsoft.DomainRegistration/domains" | ForEach-Object {
    $JSON_FILE_PATH="$JSON_PATH/$($_.Name).json"
    Export-AzResourceGroup -ResourceGroupName $_.ResourceGroupName -Resource $_.ResourceId -Path "$JSON_FILE_PATH" | Out-Null
    $DNS_JSON = Get-Content "$JSON_FILE_PATH" | ConvertFrom-Json
    [void]$ResultsTableArray.Add("$($_.Name),$($DNS_JSON.resources.properties.autoRenew)")
    Remove-Item "$JSON_FILE_PATH" -Force | Out-Null
}

Updating AzureRM from 3.90.0 to anything later than 3.95.0

I had a troublesome few hours upgrading the AzureRM Terraform provider recently. It refused to plan or apply one specifc storage account container.  To really confuse things, it was only doing this in two environments out of three, that have this particular container. I removed the object from state and tried to import it again, still the same error. I forgot to write the error down, it was something like invalid domain. I used the console to view the objects details. I was almost all the way through completing a bug report in Github against the provider when I finally saw something slightly odd between the working and a non working environment. Can you see it?

It looks lke the the two broken environments were missing a subdomain from the ID attribute. I presume that when these two environments were built, Azure used a different URL nomenclature and the latest providers now validate the domain is correct! Thankfully, the fix was easy.  Remove the object from state and import it back, but adding the subdomain blob that was missing from the ID in the broken environments. This ID is what you use to import the resource and it worked fine by using the new URL. Phew, provider now up to the latest and planning cleanly with no changes again!

> azurerm_storage_container.storage_container["document/582c3272-97aa-yyyy-xxxx-redactedguid"]

{

  "container_access_type" = "private"

  "has_immutability_policy" = false

  "has_legal_hold" = false

  "id" = "https://redacted-acc.core.windows.net/582c3272-97aa-yyyy-xxxx-redactedguid"

  "metadata" = tomap({})

  "name" = "582c3272-97aa-yyyy-xxxx-redactedguid"

  "resource_manager_id" = "/subscriptions/5acc90e0-7015-yyyy-xxxx-redactedguid/resourceGroups/stg-storage-rg/providers/Microsoft.Storage/storageAccounts/redacted-acc/blobServices/default/containers/582c3272-97aa-yyyy-xxxx-redactedguid"

  "storage_account_name" = "redacted-acc"

  "timeouts" = {

    "create" = tostring(null)

    "delete" = tostring(null)

    "read" = tostring(null)

    "update" = tostring(null)

  }

}

 

> azurerm_storage_container.storage_container["document/582c3272-97aa-yyyy-xxxx-redactedguid"]

{

  "container_access_type" = "private"

  "has_immutability_policy" = false

  "has_legal_hold" = false

  "id" = "https://redacted-acc.blob.core.windows.net/582c3272-97aa-yyyy-xxxx-redactedguid"

  "metadata" = tomap({})

  "name" = "582c3272-97aa-yyyy-xxxx-redactedguid"

  "resource_manager_id" = "/subscriptions/974dd827-dc49-yyyy-xxxx-redactedguid/resourceGroups/dh1-storage-rg/providers/Microsoft.Storage/storageAccounts/redacted-acc/blobServices/default/containers/582c3272-97aa-yyyy-xxxx-redactedguid"

  "storage_account_name" = "redacted-acc"

  "timeouts" = null /* object */

}

Updating The Terraform Provider For AzureAD From v1.0 to v2.0

 Updating The Terraform Provider For AzureAD From v1.0 to v2.0

This entry is to briefly describe some of the steps I had to go to to be able to upgrade the Terraform AzureAD provider from v1.0 to v2.0.  Specific versions in my case were v1.6.0 to v2.13.0.

The primary issue blocking the upgrade was the change in Azure AD applications that means that the UUID for Oauth2 permission IDs is no longer calculated by the Azure backend, presumably an effect of the provider migrating to Microsoft Graph instead of Azure AD Graph.
This means that you must now create the UUIDs in the terraform code and set them in the azuread_application resource.  However if your applications already exist this will unexpectedly destroy and create parts of the AzureAD application, which in my case was not comfortable.  I always prefer upgrades to be able to run through with no changes.

After some experimenting in a test environment my strategy to upgrade become a two step process along these lines.

Step A : Prep for Azure AD applications by importing the UUIDs whilst using AzureAD v1
Step B : Update Azure AD applications to AzureAD v2 including some tweaks to maintain the current configuration

Let's go into a little more detail.  I use a git repo for my terraform and found it useful to make a branch for each step so that I could switch between them as needed.

Step A

1. Ascertain all the UUIDs required for your Azure AD applications.  This can be by going into the Azure AD GUI or using terraform, review outputs if you have them for your applications (or add them temporarily.)  You can also plan using Azure AD v2.0 to see the changes which will include the UUIDs and other useful changes, ensure you DO NOT APPLY or you may make unexpected changes.
2. Ensure you are pinned with the provider AzureAD at v1 or changes to the behviour may cause issues.
3. Add the random_uuid resources to your code.
4. Import the current UUIDs to the new random_uuid resources with a command like `terraform import random_uuid.azuread_application_client_apin 1234ec9f-1234-123f-abc2-55ad558f4098`
5. Now your state is ready for Step B.

    
Step B

1. In this step you now force the provider AzureAD to use v2.
2. Tweak any attributes in the resource "azuread_application" and ensure the id attributes are now using the relevant random_uuid value imported in step A. Here you plan and tweak attributes until you have minimised or eliminated any changes.
3. Plan and Apply the latest code and ideally you should see no change.

Example of code blocks :

locals {
  azuread_application_name_client  = "sfabric-app-client"
}

resource "random_uuid" "azuread_application_client_api" {
}

resource "azuread_application" "client" {
  display_name                   = local.azuread_application_name_client
  sign_in_audience               = "AzureADMyOrg"
  fallback_public_client_enabled = true

  api {
    known_client_applications      = []
    mapped_claims_enabled          = false
    requested_access_token_version = 1

    oauth2_permission_scope {
      admin_consent_description  = format("Allow the application to access %s on behalf of the signed-in user.", local.azuread_application_name_client)
      admin_consent_display_name = format("Access %s", local.azuread_application_name_client)
      enabled                    = true
      id                         = random_uuid.azuread_application_client_api.result
      type                       = "User"
      user_consent_description   = format("Allow the application to access %s on your behalf.", local.azuread_application_name_client)
      user_consent_display_name  = format("Access %s", local.azuread_application_name_client)
      value                      = "user_impersonation"
    }
  }
}

Azure Storage Account /.well-known/

I was really struggling to figure out a way of creating the path /.well-known/ in a standard Azure storage account. This was to allow a simple set of files for an iOS and Android app to allow them to be verified. I could neither create a container with this name or name the blob with .Well-Known/ at the start of it.

As we use Azure Application Gateway I worked out that the /.well-known/ path can be emulated with path based routing and aliasing it to the root of the storage account.

There may be simpler methods but this worked for me. I created two paths to the same root with some variables similar to these :

# appmanifest is a template for routing to a storage account, storage account name is defined elsewhere.
appmanifest = {
  waf_hostname                  = "app"
  waf_probe_path                = "/hello.txt"
  waf_backend_path              = "/"
  waf_request_routing_rule_type = "PathBasedRouting"
  waf_url_path_map = {
    appmanifest-default   = "/*"
    appmanifest-wellknown = "/.well-known/*"
  }
}

Github

Recently I had to sign up to Github for a training course so it seemed like a good use to start putting some snippets of code there. PowerShell goodies will be here:

https://github.com/dhmoto17/PowerShell

I have been interested in what can be achieved with scripting since my first forays into the IT world, being able to repeat tasks easily is so powerful.  With devops being a buzzword and current hot topic it's amazing to see what can be done with new tools and a bit of coding skill.

VMware convertor

Always remember to completely disable any AV before starting a conversion.