Using Azure automation and Events to monitor a parent Key Vault and update the secret in child Key Vault(s)

Before going into the actual blog, I wanted to explain the Azure resources that we are going to discuss in simple words,

Key Vault is a resource in Azure to store secret information that can be accessed by your apps through managed identities or service principal authentication.

Azure Automation is a service that gives you the ability to automate tasks, do configuration changes, do updates to your azure resources using Az Cmds through runbooks in it. So, practically, we can run any automation using the Az cmds.

Azure Event Grid is a service that fires events to your chosen event handlers or webhooks for the events that happen in your Azure resources. In this blog, we are going to have KVs as the event source and Webhooks as the handler.

Now, in this blog, I am going to explain how you can use Azure Automation to detect changes in a KV and update the secret value in another KV.

Why would someone do that? OK, consider having 100s of KVs in Azure and are being used by 100s of apps. If you have a secret value that is in all these KVs and you don't want your apps connecting to multiple KVs, then it's muscle work to update the secret value in that many KVs.

So, Azure Automation and Event Grid subscriptions come in to help us detect the new version of the secret in a KV and update its value in the other KVs.

Prerequisites

Make sure, you have access to do the below in Azure,

1. Create Automation resource
2. Create Key Vault
3. Create Events in Key Vault
4. Edit Access Policies in Key Vault

Step 1: Create Azure Automation

Follow this to create a new Azure Automation account with System assigned identity. We will use this system assigned identity to access the secrets in the KV.

By default, the Azure Key Vault module is enabled in the Azure Automation. Make sure you have it in the Modules section. I am using the Runtime version 5.1 for this blog.

Step 2: Create Key Vaults

Follow this to create a new KV. For this blog, I am creating 2 Key vaults - 1 for parent and 1 for children. I want to update the secret in child KVs when there is a new version of the secret in the parent.

Both the parent and child KVs needs to be set up with Access policies including the Automation account we created in Step 1 as like in the below pic,

Populate the KVs with secrets in it by following this.

Step 3: Create Runbook in the Azure Automation account

Follow this to create a new Runbook in the Azure Automation account. For this blog, we will use the PowerShell workflow runbook and we will use the Az cmds.

Follow this to create a Webhook under the newly created Runbook. Make sure you copy the Webhook URL while creating, as it cannot be retrieved later. We will need this URL to be added to the KV's Event Handler.

Step 4: Setup Event Subscription and handlers in Parent KV

1. Go to your parent KV.
2. Select Events from the left section.
3. Create an Event Subscription.
4. Give the subscription and the topic unique names.
5. For the Event Types, choose "Secret New Version Created" as this is what we are going to monitor in this blog.
6. For the Endpoint Type, choose "WebHook" and type in the URL we got from Step 3 for the endpoint.
7. Click on create to save the Event Subscription

As of now, we have subscribed to the new secret version create an event in the KV, and set Runbook's Webhook as the event handler.

Step 5: Catch the event data in the Azure Automation Runbook

In this step, we will catch the event data in the webhook as use Az cmds for setting the secret value in our child KV.

Go to your Runbook's Edit window and type in the below PS code in it.

param
(
[Parameter (Mandatory = $false)]
[object] $WebhookData
)

if ($WebhookData) 
{
    Write-Output $WebhookData.RequestBody

    $bodyJson =  $WebhookData.RequestBody | ConvertFrom-Json # Converting to json for easier manipulation

    # Getting the KV's name, Secret's name and the Event Type
    $secretName = $bodyJson[0].data.ObjectName
    $parentKv = $bodyJson[0].data.VaultName    
    $eventType = $bodyJson[0].eventType
    $childKv = ""

    if($eventType -eq "Microsoft.KeyVault.SecretNewVersionCreated")
    {
        # Authenticate using the System assigned identity of the Azure Automation account
        Connect-AzAccount -Identity

        # Using Az cmd for getting the secret's latest version value from the KV
        $secretValue = Get-AzKeyVaultSecret -VaultName $parentKv -Name $secretName -AsPlainText

        Write-Output "A new version for secret $($secretName) has been created in KV - $($parentKv) with value - $($secretValue)"

        if($secretName -eq "secret-child-1")
        {
            $childKv = "child-1-kv"           
        }
        elseif($secretName -eq "secret-child-2")
        {
            $childKv = "child-2-kv"
        }

        if($childKv -eq "")
        {
            Write-Output "No secrets have been set in the child KVs as the updated key in parent is not associated with the children"
        }
        else
        {
            # Setting the child KV with the secret value that is set in the parent KV
            $Secret = ConvertTo-SecureString -String $secretValue -AsPlainText -Force
            Set-AzKeyVaultSecret -VaultName $childKv -Name $secretName -SecretValue $Secret

            Write-Output "A new version of secret $($secretName) has been created in KV - $($childKv) with value - $($secretValue)"
        }        
    }
}
else
{
    write-Error "No input data found." 
}

$WebhookData is the parameter where we will get the event data. I have converted the data into JSON for easier manipulation and ripped the KV's name, Secret name and event type.

$bodyJson =  $WebhookData.RequestBody | ConvertFrom-Json
$secretName = $bodyJson[0].data.ObjectName
$parentKv = $bodyJson[0].data.VaultName    
$eventType = $bodyJson[0].eventType

After checking the event type we needed, I am using the Az cmd to authenticate the current execution.

Connect-AzAccount -Identity

Getting the secret's latest version value from the KV

$secretValue = Get-AzKeyVaultSecret -VaultName $parentKv -Name $secretName -AsPlainText

Setting the child KV with the secret value that is set in the parent KV

$Secret = ConvertTo-SecureString -String $secretValue -AsPlainText -Force
Set-AzKeyVaultSecret -VaultName $childKv -Name $secretName -SecretValue $Secret

Publish this runbook and we shall test the setup in the next step. For this blog, I have hardcoded the secret names; you can change the logic as per your need.

Step 6: Test the setup

Now, create a new version in the Parent KV's secret. This will fire an event to the Webhook. You can monitor this in the KV's metric as like in the below pic,

You can also check the Webhook for the Last triggered value,

Output section in the Jobs section will give you the output of the PS code that was executed for the event. Here is the example output,

[{"id":"6e1c172c-26f5-49e6-a065-f9ee3818b7e1","topic":"Removing this for security","subject":"secret-child-2","eventType":"Microsoft.KeyVault.SecretNewVersionCreated","data":{"Id":"Removing this for security","VaultName":"parent-kv","ObjectType":"Secret","ObjectName":"secret-child-2","Version":"Removing this for security","NBF":null,"EXP":null},"dataVersion":"1","metadataVersion":"1","eventTime":"2021-12-03T12:18:25.6002045Z"}]


Environments                                                                                                            
------------                                                                                                            
{[AzureChinaCloud, AzureChinaCloud], [AzureCloud, AzureCloud], [AzureGermanCloud, AzureGermanCloud], [AzureUSGovernme...

A new version for secret secret-child-2 has been created in KV - parent-kv with value - 741258963


SecretValue : System.Security.SecureString
Attributes  : Microsoft.Azure.Commands.KeyVault.Models.PSKeyVaultSecretAttributes
Enabled     : True
Expires     : 
NotBefore   : 
Created     : 12/3/2021 12:18:30 PM
Updated     : 12/3/2021 12:18:30 PM
ContentType : 
Tags        : 
TagsTable   : 
VaultName   : child-2-kv
Name        : secret-child-2
Version     : Removing this for security
Id          : Removing this for security

A new version of secret secret-child-2 has been created in KV - child-2-kv with value - 741258963