Boost your Infrastructure as Code with Bicep

At tech agency Q42 we start several new web projects every year. Each of these projects need a cloud infrastructure for hosting. In the majority of our projects, we use Infrastructure as Code (IaC) to automate the provisioning of our cloud resources.

It is great to be able to provision a whole new environment (for example for dev/test or acceptance) with just one click. As an extra bonus you are certain that all the resources are configured and created the same way. Finally, it enables you to apply version control to your infrastructure as well, with all the benefits that follow from that (e.g. pull requests, code reviews, et cetera).

While the first setup always takes a bit more time than just using the cloud management portal, we’ve learned scripting your IaC always pays off in the long run. Luckily, there are some great tools to ease the set up and management of your Infrastructure as Code, like Terraform or Pulumi.

Until recently our way to manage resources in Microsoft Azure was using the Azure Resource Manager (ARM) script. The syntax of ARM is not very developer friendly, poorly documented and there is no good way to validate your code without deploying to Azure.

Microsoft has introduced Bicep as an alternative. The first public version has been released late 2020. This makes writing IaC for Azure a lot easier.

In the first part of this blog I will introduce you to some basics around the new language. In the second part I give some more details on how we use it in our day to day DevOps practices. I will conclude with some ideas about improvements and (hopefully) future developments.

Let's start!

Some basic introduction

So Bicep is the new kid on the block. In which way is it easier to use? Well, it starts with the fact that Bicep files compile (or actually transpile) into an ARM template. So you can actually compare it with Typescript transpiling into Javascript.

The transpile step provides some nice advantages, like warnings and errors now happen on build rather than during deployment. Next, it adds the possibility to apply linting rules. Furthermore, the syntax is way easier.

Bicep sample

All of this can be demonstrated with a small sample for deploying a web application. Make sure you install Bicep.

Create a new file in your favorite editor called sample.bicep.

Parameter declaration

We start with some parameter declarations with a default initial value:

param location string = 'West-Europe'
param appServicePlanName string = 'MyAppservicePlan'
param skuName string = 'S1'
param skuCapacity int = 1
param webAppName string = 'MyWebApp'

Notice that parameters are typed. So when I provide a string value for the skuCapacity, the compiler will tell me I’m doing something wrong.

App service plan and Web app

In order to deploy a web application we need two resources, an App service plan and a Web app. Let's first describe our App service plan. It starts with the resource keyword and the name of the resource including the resource definition.

After that, we fill the required properties with our parameters and the App service plan is done.

resource AppServicePlan 'Microsoft.Web/serverfarms@2021-02-01' = {
  name: appServicePlanName
  location: location
  sku: {
    size: skuName
    capacity: skuCapacity
  }
}

Next we define our Web app in the same way:

resource WebApp 'Microsoft.Web/sites@2021-02-01' = {
  name: webAppName
  location: location
  properties: {
    serverFarmId: AppServicePlan.id
  }
}

Notice the new feature of Bicep here. We reference our previously defined AppServicePlan and get its Id value AppServicePlan.id to set it as the serverFarmId. This is way easier than the ARM equivalent.

"[resourceId('Microsoft.Web/serverfarms',variables('appServicePlanName'))]",

The full sample to deploy an App service plan and Web app:

param location string = 'West-Europe'
param appServicePlanName string = 'MyAppservicePlan'
param skuName string = 'S1'
param skuCapacity int = 1
param webAppName string = 'MyWebApp'
 
resource AppServicePlan 'Microsoft.Web/serverfarms@2021-02-01' = {
  name: appServicePlanName
  location: location
  sku: {
    size: skuName
    capacity: skuCapacity
  }
}
 
resource WebApp 'Microsoft.Web/sites@2021-02-01' = {
  name: webAppName
  location: location
  properties: {
    serverFarmId: AppServicePlan.id
  }
}

Modules

One of the major improvements in Bicep is the use of modules. In a lot of our IaC files we deploy multiple Azure Web apps for a specific project, for example a WebApp, CmsApp, ApiApp and AdminApp.

You can simply define all these resources explicitly, however this leads to a lot of duplicate code and changes become cumbersome. The use of Bicep modules provides a nice and maintainable solution.

The definition of a module is done in a separate Bicep file. So let's create app-service.bicep. In this file we define a generic AppService module with a few predefined values. We prefer alwaysOn, .NET 6 framework, HTTPS Only and a Managed Identity which we will use later for a key vault connection.

param location string = resourceGroup().location
param appName string
param appServicePlanId string
 
resource AppService 'Microsoft.Web/sites@2020-06-01' = {
  name: appName
  location: location
  properties: {
    serverFarmId: appServicePlanId
    siteConfig: {
      appSettings: []
      alwaysOn: true
      http20Enabled: true
      netFrameworkVersion: 'v6.0'
    }
    httpsOnly: true
  }
  identity: {
    type: 'SystemAssigned'
  }
}

The module has a few input parameters as well (location, appName and appServicePlanId), which need to be provided when using the module. You could give them a default value to make them optional.

Now let's use this module in our previous example. Instead of starting with the resource keyword we use the module keyword and reference our app-service.bicep file.

module appServiceWeb 'app-service.bicep' = {
  name: 'appServiceWeb'
  params: {
    appName: webAppName
    appServicePlanId: AppServicePlan.id
  }
}

While it looks not very different from the previous sample, this module has much more defaults defined than the previous sample, using fewer lines of code! Additionally, we now have the benefit of reusing this module for other apps as well.

Loops

So we want to deploy several Web apps without copy-pasting the resource definition. For this we can use loops.

First, let's define an array with the names of our Web apps we want to deploy:

// array of website names
param Websites array = [
  'MyApplication-Web'
  'MyApplication-Cms'
  'MyApplication-Api'
  'MyApplication-Admin'
]

Second, we can loop through this array and create a Web app for each entry in the array with just a few simple lines lines of code:

module WebAppResources 'app-service.bicep' = [for webAppName in Websites: {
  name: webAppName
  params: {
    appName: webAppName
    appServicePlanId: AppServicePlan.id  
  }
}]

If we ever need to create an extra web app, we can just add the name to the array and it will be deployed.

You made it halfway through, good job! Hopefully you now have some basic knowledge of how to use Bicep and how modules and loops can be very powerful to deploy multiple resources.

🇳🇱 👋 Hey Dutchies!
Even tussendoor... we zoeken nieuwe Q'ers!

In the next part of this blog we dive into some more advanced use cases. We will explain how we use registries to centralize our modules and automate our deployment using Azure Devops.

Staging slot with conditional deployment

To support new deployments without any downtime of your website the common practice is to use a staging slot.

The idea is simple, you deploy your new code to a staging environment, warm up the application and check if everything works as expected. Then you swap your production and staging slot.

Sounds good right? However, defining a staging slot for each Web app is not as simple as it sounds. It is quite repetitive and error-prone to keep the configuration of the production slot in sync with the staging slot. Furthermore, deployment slots are sensible to use in production environments but for development it might be skipped.

In Bicep we solve this by implementing an app service module with conditional deployment.

Let's extend the previously created appservice module. We start by defining some new parameters. First, a boolean deployStaging which defines if you want to deploy a staging slot with your Web app. Furthermore, we define the siteConfig object which can be used in our production slot and staging slot, so we have the same configs.

param deployStaging bool = false
 
param siteConfig object = {
  appSettings: []
  alwaysOn: true
  http20Enabled: true
  netFrameworkVersion: 'v6.0'
}

Next, we define our staging slot resource which is quite similar to the definition of the webApp. Note the new syntax if (boolean) { } after the resource definition. This generates the conditional deployment of the staging slot.

resource AppService_staging 'Microsoft.Web/sites/slots@2021-01-01' = if (deployStaging) { 
  name: '${AppService.name}/staging'
  location: location
  properties: {
    serverFarmId: appServicePlanId
    siteConfig: siteConfig
    httpsOnly: true
  }
  identity: {
    type: 'SystemAssigned'
  }

Now it is super easy to define in our main Bicep file that we would like to deploy a staging slot by adding the property:

deployStaging: true

Key vault connection

In every project there is sensitive information, for example database connection strings, API keys, etc. Storing them in plain text in your configuration files is not a great idea. At Q42 it is good practice to always deploy a key vault in which we store these secrets (especially for production environments).

Your Web Apps need a secure way to access those secrets. Luckily Microsoft has introduced managed identities. These are application identities which can be used to access other (Azure) resources. We already enabled managed identity on the Web apps by adding this to the definition.

identity: {
    type: 'SystemAssigned'
  }

However, this only creates an application identity. This identity also requires read access to the key vault. To make it even more complex, a staging slot also has a separate managed identity which also requires read access.

Forgetting to give a web app access to a key vault, was one of the most common mistakes we made when adding a new Web app or staging slot to our ARM script.
By automatically adding this to our Infrastructure as Code setup, we now have solved this issue. So, how to achieve this?

First, we create a module for the key vault access policy. Kalle Marjokorpi has written a nice blog post about this, which contains a great example of a key vault policy module.

resource keyVault 'Microsoft.KeyVault/vaults@2019-09-01' existing = {
  name: keyVaultResourceName
  resource keyVaultPolicies 'accessPolicies' = {
    dependsOn: [
      keyVault
    ]
    name: policyAction
    properties: {
      accessPolicies: [
        {
          objectId: principalId
          permissions: keyVaultPermissions
          tenantId: subscription().tenantId
        }
      ]
    }
  }
}

Note that two new concepts are used here. The existing keyword is added to make a reference to the already created key vault based on the key vault name. Furthermore the keyVaultPolicies resource is nested inside the keyVault resource which automatically implies a child resource of the parent resource (keyVault in this case).

The second step is to add this keyVaultAccessPolicy module to our app-service module.

param keyVaultName string = ''

var keyVaultPermissions = {
  secrets: [
    'get'
    'list'
  ]
}
 
module keyVaultAccessPolicy './key-vault-access-policy.bicep' = if (!empty(keyVaultName)) {
  name: 'AppService_KeyVault_AccessPolicy'
  params: {
    keyVaultResourceName: keyVaultName
    principalId: AppService.identity.principalId
    keyVaultPermissions: keyVaultPermissions
    policyAction: 'add'
  }
}

If you supply the key vault name it automatically creates an access policy for your Web app (and staging slot if specified!) to the key vault. In this way we will never forget to add this anymore.

We’ve already used this key vault module in several of our own projects, and are happy about the results!

Using modules with a private registry

Having a set of custom modules which can be reused in several projects is great. However, at first the only way to reuse the modules was by copy-paste or by using Git submodules. We all know copy-paste isn't the best way to manage code, since each time an improvement is made to one of our modules, we need to propagate this change to all of our projects. The Bicep team recognized this problem and came up with a solution. A central (private) registry in which you could store your modules and that also supports version control.

Because Azure already has such a system in place to put your container images in, called Azure Container Registry, it was fairly easy to reuse this for Bicep Modules.

The idea is as follows:

• You publish your Bicep modules(s) into the Azure container registry.
• From your Bicep template you make a module reference.
• When building the template, the referenced module is retrieved from the registry and used.

Let's implement this for our app-service example. To set up the Azure container registry, you can follow the instructions from Microsoft.

Then we publish our app-service module with the Azure CLI by navigating to the folder in which the module is saved and run the following command. Replace the {ACRname} with the name of your Azure Container registry.

az bicep publish --file app-service.bicep --target br:{ACRname}.azurecr.io/bicep/app-service:1.0

Finally, we can make a reference to the module by changing the reference in our sample.bicep.

module appServiceWeb 'br:{ACRname}.azurecr.io/bicep/app-service:1.0' = {
  name: 'appServiceWeb'
  params: {
    appName: webAppName
    appServicePlanId: AppServicePlan.id
  }
}

Now we have a reference to the centrally stored app-service module. When improvements to this module are made, a new version of the module can be published. If the version number of the module hasn't changed, the changes are automatically applied. By bumping the reference's version number we can easily propagate the changes to all our projects.

Registry alias

If you add a bicep.config file to your repository you can specify an alias for your registry. This is useful so when your container registry changes, you only need to change the alias. Furthermore, it is a bit cleaner in your main file.

 "moduleAliases": {
    "ts": {},
    "br": {
      "q42": {
        "registry": "{ACRname}.azurecr.io",
        "modulePath": "bicep"
      }
    }
  }

Now you can use the alias in your reference.

module appServiceWeb 'br/q42:app-service:1.0' =

Note the slight change in syntax using a / between br and the alias name.

Deployment of your infrastructure

Up until now we only have explained how to create the template files which describe the deployment of your resources. Now let's look at actually performing the deploy, which is fairly straightforward.

You can deploy the resources defined in the sample.bicep by running the following command locally from your Azure CLI. This will build the Bicep file and deploy the resources in one go.

az deployment group create --resource-group <resource-group-name> --template-file sample.bicep

We find it more convenient to use Azure DevOps or Github Action to do the work for us.

At the moment of writing there is no official Bicep task in Azure DevOps so we use the Azure CLI. In Azure DevOps we create a yaml pipeline and add the following:

variables:
  azureServiceConnection: 'MyAzureServiceConnection'
  resourcegroupName: "MyResourceGroup"

steps:
  - task: AzureCLI@2
    inputs:
      azureSubscription: $(azureServiceConnection)
      scriptType: 'pscore'
      scriptLocation: 'inlineScript'
      inlineScript: |
      az deployment group create --resource-group $(resourcegroupName)          --template-file sample.bicep

Multi tenant

As an extra, let's dive into one more challenge we've faced. As explained before, we love the idea of modules and adding them to the Azure Container Registry. However, this registry is by default only accessible from within the same tenant.

At Q42 we develop web and mobile applications for many different clients, each with their own Azure tenant. So, how do we make sure our bicep module registry can be accessed from all of those tenants?

One option would be to use the anonymous access feature, however this feature is not available in the basic tier. Hence we came up with a quick and dirty solution to fix this issue.

The idea is to use a separate azureServiceConnection for building the Bicep file and deploying the resources.

First, we need to create an extra service connection with a service principal with minimal AcrPull rights on the Bicep registry. This is the connection we'll use to build our Bicep file.

variables:
  azureServiceConnection: 'Bicep registry'

steps:
  - task: AzureCLI@2
    displayName: 'build bicep artifact'
    inputs:
      azureSubscription: $(azureServiceConnection)
      scriptType: 'pscore'
      scriptLocation: 'inlineScript'
      inlineScript: 'az bicep build --file sample.bicep'

Next, we use the default connection to deploy the resources.

Note that there are more sophisticated solutions, e.g. by using Azure Lighthouse as Jannick Oeben describes in his blog post.

Finally, a better long-term solution would be public registries. Something that is already on the roadmap for version 0.5.

Wrapping up

In this blog post I have shared our experience using Bicep for Infrastructure as Code. Overall we think it is a major improvement compared to the ARM templates.

We recently started with some new projects and implemented the Bicep templates from the start. With the use of modules the whole infrastructure could be automatically deployed within a couple of hours, saving us significant time, which is impressive!

For the beginning of 2022 there are some interesting features on the Bicep release roadmap. Starting with the addition of a public registry in the release of 0.5. I hope that we can replace some of our private modules with the ones from the public registry.

A bit further down the road are deployment stacks where the deployment state is saved and you are able to make changes on that state (like add, delete and rollback). Finally,there is also the idea to add extensibility to Bicep to also deploy Azure Active Directory resources.

This blog is based on the experience we had by implementing Bicep in some of our projects. I’m really interested in your experience with Bicep! Please drop me a message at jeroenvdb@q42.nl.

Are you a backend engineer who loves working with new technologies like Bicep? Check out our job vacancies at werkenbij.q42.nl.