Category Archives: Azure DevOps

Automatic Provisioning and Deprovisioning of Copilot for Security Capacity Unit

1 Reply

At time of writing this blog post, it’s only the first week after Microsoft Copilot for Security went GA April 1st, and already the #Security #Community are creating scripts, automations and tools for provisioning or deprovisioning Secure Capacity Units (SCU) that are required for running Copilot for Security in your environment.

A potential cost of 4$ per hour per SCU can amount to $2920 per month if you count 730 hours standard per month, so the motivation is clear, to save cost and use the service only when you need to.

I took inspiration from https://thoor.tech/Copilot-for-Security-deploy-and-destroy/, and decided to create my own solution based on Bicep, Deployment Stacks, and Azure DevOps Pipelines to automate creating a SCU on weekday mornings, and destroy again on the afternoon.

Disclaimer: This is a concept for saving cost in my own sponsored development and demo subscription, and the effects of removing and recreating Secure Compute Units in a Production environment must be carefully evaluated.

Bicep and Deployment Stacks

I wanted to use Bicep and often when I deploy Bicep I use a main.bicep file that deploys at subscription level, that also creates resource group(s) as well as contained resources which I place in modules.

Deployment Stacks, which is currently in Public Preview, let you deploy a set of resources grouped together, and I highly recommend reading this blog post by Freek Berson to understand more of the value: https://fberson.medium.com/deployment-stacks-for-bicep-are-awesome-a-full-walkthrough-sneak-peek-and-of-whats-coming-soon-4461078ec56a

I decided to use Deployment Stacks because I wanted to do a declarative approach, not only to do the actual deploy of the Secure Compute Unit, but also remove the resource from the resource group when the resource became unmanaged.

I think the code referenced at the end of this blog post is mostly self-explaining, but I will highlight a couple of elements. The first is using conditional deployment of the module in Bicep like this:

From the image above I can control whether the module will be managed in the deployment or not when I deploy the Bicep.

When running Bicep and using Deployment Stacks, you can control what happens to resources that are ‘unmanaged’. This is how I run the deployment if I want to NOT deploy the Secure Compute Unit for Copilot for Security Capacity, basically controlling CREATE operation with setting the parameter to true, and DELETE operation with setting the parameter to false:

az stack sub create --location WestEurope --name "stack-scu-elven-we" --template-file .\main.bicep --parameters deploySecureCapacityUnit=false --deny-settings-mode none --action-on-unmanage deleteResources

The magic parameter here is the –action-on-unmanage, which I set to deleteResources, which will delete all resources that falls out of bicep template (either by condition or removing resource from file). You can also do a detachResources, which will keep the resources, and you can specify deleteAll which will also remove the Resource Group (which I do not want in my scenario).

PS! Note that I run an az stack sub create, which lets me run a bicep deployment that creates resource groups as needed (yes, I could be a bit colored by my much longer Terraform experience where I also create resource groups).

In the Azure Portal, when I run a deploy that creates a SCU, it will look like this in the Deployment Stack:

Note the action on unmanage:

And when I set the parameter to false, to delete the SCU, the result is this:

Azure DevOps Pipelines

With the Bicep deployment using Az CLI verified succesfully, it should be straight forward to create Pipelines for deploying and destroying. The full YAML files are reference below, and should be easy to understand. The main components and requirements are:

  • Using a CRON expression and not a trigger for just scheduling when to run the pipeline. Note the use of always: true for the schedule, because I want the schedule to run in any case even if the files in the repo have not changed.
  • Remember you need to set up a Service Connection (Workload Identity Federation highly recommended), with approriate permissions to create resources in your reference Azure subscription.

If you look closer, you can see that the 2 pipelines, one for deploy and one for destroy, are almost exactly the same, with the exception of default value of either false or true for deploySecureCapacityUnit, and of course the morning and afternoon schedules are different.

FEATURE WHISH: I would have thought that I could use multiple schedules in the same YAML pipeline, and specifying different default value for the deploySecureCapacityUnit parameter based on schedule. Then I would be able to have just the one pipeline instead of two..

I’ll leave you to the rest, see reference to files I have published as Gists on my GitHub account https://github.com/janvidarelven.

Gist Reference

Complete Gist is linked below and is public, and feel free to Clone, Fork, Pull if you want to contribute, etc.

Gist consists of 4 files:

  • main.bicep – Parameters, variables, resource group and a module for Secure Compute Unit
  • secure-compute-unit.bicep – Module definition for deploying a Secure Compute Unit for Copilot for Security
  • deploy-security-copilot.yml – Pipeline with schedule for deploying Bicep from above files using Deployment Stack, creating a Secure Compute Unit.
  • destroy-security-copilot.yml – Pipeline with schedule for deploying Bicep from above files using Deployment Stack, deleting the Secure Compute Unit.
name: CD-$(rev:r)-Deploy-Security-Copilot-$(Date:dd.MM.yyyy) # build numbering format
trigger: none
schedules:
– cron: "0 7 * * 1-5"
displayName: Morning weekdays
branches:
include:
– main
always: true
parameters:
– name: azureServiceConnection
default: serviceconn-<your-wif-connection>
– name: azureSubscriptionTarget
default: '<your-sub-name-or-id'
– name: deploySecureCapacityUnit
type: boolean
default: true
pool:
vmImage: windows-latest
variables:
– name: deploymentDefaultLocation
value: westeurope
– name: deploymentBicepTemplate
value: .\SecurityCopilot-Bicep\main.bicep
jobs:
– job:
steps:
– task: AzureCLI@2
displayName: 'Deploy Security Copilot Compute Unit'
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
scriptType: pscore
scriptLocation: inlineScript
inlineScript: |
az –version
az account set –subscription '${{ parameters.azureSubscriptionTarget }}'
az stack sub create `
–location $(deploymentDefaultLocation) `
–name "stack-scu-yourorg-we" `
–template-file $(deploymentBicepTemplate) `
–parameters deploySecureCapacityUnit=${{ parameters.deploySecureCapacityUnit }} `
–deny-settings-mode none `
–action-on-unmanage deleteResources
view raw deploy-security-copilot.yml hosted with ❤ by GitHub
name: CD-$(rev:r)-Destroy-Security-Copilot-$(Date:dd.MM.yyyy) # build numbering format
trigger: none
schedules:
– cron: "0 14 * * 1-5"
displayName: Afternoon weekdays
branches:
include:
– main
always: true
parameters:
– name: azureServiceConnection
default: serviceconn-<your-wif-connection>
– name: azureSubscriptionTarget
default: '<your-sub-name-or-id'
– name: deploySecureCapacityUnit
type: boolean
default: false
pool:
vmImage: windows-latest
variables:
– name: deploymentDefaultLocation
value: westeurope
– name: deploymentBicepTemplate
value: .\SecurityCopilot-Bicep\main.bicep
jobs:
– job:
steps:
– task: AzureCLI@2
displayName: 'Deploy Security Copilot Compute Unit'
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
scriptType: pscore
scriptLocation: inlineScript
inlineScript: |
az –version
az account set –subscription '${{ parameters.azureSubscriptionTarget }}'
az stack sub create `
–location $(deploymentDefaultLocation) `
–name "stack-scu-elven-we" `
–template-file $(deploymentBicepTemplate) `
–parameters deploySecureCapacityUnit=${{ parameters.deploySecureCapacityUnit }} `
–deny-settings-mode none `
–action-on-unmanage deleteResources
view raw destroy-security-copilot.yml hosted with ❤ by GitHub
targetScope = 'subscription'
// If an environment is set up (dev, test, prod…), it is used in the application name
param environment string = 'dev'
param applicationName string = 'security-copilot'
param location string = 'westeurope'
param resourceGroupName string = 'rg-sec-copilot-scu-we'
param capacityName string = 'scu-<yourorg>-we'
param capacityGeo string = 'EU'
// Some params for provisioning the secure capacity unit, and if it should be deployed or not
param defaultNumberOfUnits int = 1
param deploySecureCapacityUnit bool = true
var defaultTags = {
Environment: environment
Application: '${applicationName}-${environment}'
Dataclassification: 'Confidential'
Costcenter: 'AI'
Criticality: 'Normal'
Service: 'Security Copilot'
Deploymenttype: 'Bicep'
Owner: 'Jan Vidar Elven'
Business: 'Elven'
}
resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
name: resourceGroupName
location: location
tags: defaultTags
}
// Deploy the secure capacity unit module dependent on the deploySecureCapacityUnit parameter
module scu 'secure-capacity/secure-compute-unit.bicep' = if (deploySecureCapacityUnit) {
name: capacityName
scope: resourceGroup(rg.name)
params: {
capacityName: capacityName
geo: capacityGeo
crossGeoCompute: 'NotAllowed'
numberOfUnits: defaultNumberOfUnits
resourceTags: defaultTags
}
}
view raw main.bicep hosted with ❤ by GitHub
// Secure Compute Unit – Bicep module
// Created by – Jan Vidar Elven
@description('The name of the Security Copilot Capacity. It has to be unique.')
param capacityName string
@description('A list of tags to apply to the resources')
param resourceTags object
@description('Number of Secure Compute Units.')
@allowed([
1
2
3
])
param numberOfUnits int
@description('If Prompts are are allowed to cross default region for performance reasons.')
@allowed([
'NotAllowed'
'Allowed'
])
param crossGeoCompute string
@description('Prompt evaluation region. Allowed values are EU, ANZ, US, UK.')
@allowed([
'EU'
'ANZ'
'US'
'UK'
])
param geo string
var locationMap = {
EU: 'westeurope'
ANZ: 'australiaeast'
US: 'eastus'
UK: 'uksouth'
}
var location = contains(locationMap, geo) ? locationMap[geo] : 'defaultlocation'
resource Copilot 'Microsoft.SecurityCopilot/capacities@2023-12-01-preview' = {
name: capacityName
location: location
properties: {
numberOfUnits: numberOfUnits
crossGeoCompute: crossGeoCompute
geo: geo
}
tags: resourceTags
}
view raw secure-compute-unit.bicep hosted with ❤ by GitHub

Connect to Microsoft Graph in Azure DevOps Pipelines using Workload Identity Federation

2 Replies

Microsoft recently announced that Workload Identity Federation for Azure Pipelines now is in Public Preview: https://devblogs.microsoft.com/devops/public-preview-of-workload-identity-federation-for-azure-pipelines/.

This opens for a lot of scenarios for Azure service connections, without the need to manage secrets for service principals and more security as there are no secrets that can be exposed or exfiltrated.

As I work a lot with Microsoft Graph and automation, I wanted to see if and how I could use Workload Identity Federation to connect to and send queries to Microsoft Graph using Azure Pipelines.

Create the Workload Identity Federation Service Connection

First of all, I need to create a service connection in my Azure DevOps project that will use the new Workload Identity Federation. To be able to do this, you need to have access to the preview functionality, see details here in this learn article: Create an Azure Resource Manager service connection using workload identity federation.

When you have access to the feature, you can create a new Workload Identity federation either by manual or automatic configuration:

I will now choose the Azure Subscription, and optionally a Resource Group. Choosing a resource group is a good idea, as the service connection will be given Contributor access only to that Resource Group, and not the whole subscription. But it also depends on what you want to use your Service Connection for, in my case it is a demo scenario for Microsoft Graph Access, so it makes sense to scope the permissions down:

After creating the Service Connection, I can find it my Entra ID tenant. Let’s look at the role assignments for the Resource Group first:

The service principal has been given the name of <DevOps Org>-<DevOps Project>-<guid>, and been assigned with Contributor access to that RG.

Next, let’s find the App Registration for the Service Connection. As you can see from below there has been no (0) credentials of secrets or certificates created, but there has been created a Federated credential:

If we look at the detail for the federated credential, we can se the issuer, subject and audience, and confirm that this service principal only can be access by the service connection in Azure DevOps:

Next, go to API permissions. Here I will add a Microsoft Graph permission, so that we can use that for queries in the pipeline later. In my case I add the Application permission User.Read.All, so I can look up user information:

We are now ready to set up an Azure Pipeline to use this service connection.

Create the Azure Pipeline to access Microsoft Graph API

In your DevOps project, if this is a new project, make sure that you initialize the Repository, and that you have at least a Basic or Visual Studio access level, then head to Pipelines and create a “New Pipeline”. For my environment I will just choose the following steps:

  1. Select Azure Repos Git (YAML)
  2. Select my repository
  3. Use a starter pipeline (or you can choose an existing if you have)

This is a sample YAML code that will use the service connection (se below picture) to get an access token for Microsoft Graph, and the use that access token to connect to Graph PowerShell SDK. In my example I’m just showing how to get some simple user information:

There are different ways you can go about this, in my case I was just using Azure CLI in one task to get the access token for the resource type that is Graph. (You can also use Az PowerShell task for this by the way). I also set and secure the variable for use in later steps in the pipeline job.

In the next task I use PowerShell Core to convert the token to a secure string, and then install the required Microsoft Graph PowerShell modules. I can then connect to Graph and get user information. Here is the complete YAML code:

# Pipeline for accessing Microsoft Graph using Federated Workload Identity Credential
# Created by Jan Vidar Elven, Evidi, 15.09.2023

trigger:
- none

pool:
  vmImage: windows-latest

steps:
- task: AzureCLI@2
  displayName: 'Get Graph Token for Workload Federated Credential'
  inputs:
    azureSubscription: 'wi-fed-sconn-ado-to-msgraph'
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
      $token = az account get-access-token --resource-type ms-graph
      $accessToken = ($token | ConvertFrom-Json).accessToken
      Write-Host "##vso[task.setvariable variable=secretToken;issecret=true]$accessToken"
- task: PowerShell@2
  displayName: 'Connect to Graph PowerShell with Token'
  inputs:
    targetType: 'inline'
    script: |
      # Convert the secure variable to a secure string
      $secureToken = ConvertTo-SecureString -String $(secretToken) -AsPlainText

      # Install Microsoft Graph Modules required
      Install-Module Microsoft.Graph.Authentication -Force
      Install-Module Microsoft.Graph.Users -Force

      # Connect to MS Graph
      Connect-MgGraph -AccessToken $secureToken

      # Get User Info
      Get-MgUser -UserId "[email protected]"
      
    pwsh: true

I can now try to run the pipeline. At first run you will have to validate and permit access to the service connection from the pipeline:

And then I can verify that it indeed can connect to the Graph via PowerShell SDK and get my resources via the Workload Identity Federation service connection:

Summary and Usage Scenarios

Most will use the new Workload Identity Federation for Azure Pipelines that access Azure subscriptions and resources, but I have shown as as long as this is using the Entra ID authentication platform and OIDC, it is possible to get access tokens for other API’s as well, in this case Microsoft Graph API.

You can use Graph API to get information about your tenant, to enrich and complement your exisiting CI/CD pipelines, or in some case automate consistent deployments also for Graph resources, like for example important settings and policies.