How-to use customer-managed keys with Azure Key Vault and Azure Storage encryption using ARM Template
I got a question from a reader asking how to use the Managed Identity of a storage account against Azure Key Vault to enable storage encryption using customer-managed keys. The documentation doesn't say storage accounts can have an identity. Testing a solution made me realize I was wrong, today I learned!
Genesis
It all started with this article on my blog for the best way to reference Managed Identity using ARM templates - there-is-a-new-way-to-reference-managed-identity-in-arm-template/
I always say that others' problems can make you grow in knowledge and it's exactly what happened with Soniya's question yesterday.
Hi Stephane, I am trying to use this method to get storage account Managed Identity and pass it to existing Keyvault's Access policy in the same template where Storage account needs to be provisioned. In the same template this storage account needs to access the keyvault to get key for user managed Storage encryption and hence the need to add the Manage identity to Keyvault access policy. I am trying to deploy it the way you have referenced the Managed Identity. But its failing. Any idea?
I wasn't aware that storage accounts could have a Managed Identity, I double-checked the documentation before answering her that it was not supported. It is available for Azure Data Lake so I was kinda intrigued. She replied that she was doing it in PowerShell but wanted a way to do it via ARM templates. Nothing more was required to trigger me and try to find the answer myself.
Managed Identity on Storage Account
After a quick test in with an ARM template, it turns out it IS possible to assign a Managed Identity to a storage account and it is fairly easy.
If you need more background on Managed Identity and how to use them in ARM templates, I have an article for you here: There is a new way to reference managed identity in ARM template
Like for any other resources, I only had to add the identity property to my resource like this:
"identity": {
"type": "SystemAssigned"
}It worked, the deployment was successful! Now my storage account has a Managed Identity, awesome!
Updating Key Vault Access
I wrote another article recently, How to incrementally update KeyVault access policies with ARM templates. Go read it if you weren't aware, it is a good trick to have in your toolbox.
There are 2 properties that you need to set on your vault if you want to use customer-managed keys with Azure Key Vault to manage Azure Storage encryption. Microsoft documentation says:
Using customer-managed keys with Azure Storage encryption requires that two properties be set on the key vault, Soft Delete and Do Not Purge. These properties are not enabled by default, but can be enabled using either PowerShell or Azure CLI on a new or existing key vault.
You can also do it in the Portal if you want. That being said, you need to update Key Vault to set those two properties. If you don't want to mess around with retrieving access policy via a script and injecting them into an ARM template, use PowerShell, Azure CLI or the Azure Portal. If you still want to update this via template, here are the two properties.
{
"type": "Microsoft.KeyVault/vaults",
"name": "[variables('uniqueResourceNameBase')]",
"apiVersion": "2016-10-01",
"location": "[parameters('location')]",
"properties": {
"sku": {
"family": "A",
"name": "standard"
},
"tenantId": "[subscription().tenantid]",
"accessPolicies": [...],
"enableSoftDelete": true,
"enablePurgeProtection": true
}
}We now have a properly configured Key Vault to support storage encryption, we still need a key for that encryption. There's several ways to do this, the simpliest is via the portal. It is not possible "natively" via template without going through a complicated plumbing. Still interested to create keys via template? I found this little gem from Jorge Cotillo
Let stick to the portal for simplicity
Add the new access policy
The storage account needs access to Key Vault. Let's add a new access policy, the access need to be wrapkey, unwrapkey and get permissions on keys.
{
"type": "Microsoft.KeyVault/vaults/accessPolicies",
"name": "[concat(variables('uniqueResourceNameBase'), '/add')]",
"apiVersion": "2019-09-01",
"properties": {
"accessPolicies": [
{
"tenantId": "[subscription().tenantid]",
"objectId": "[reference(resourceId('Microsoft.Storage/storageAccounts', variables('uniqueResourceNameBase')),'2019-06-01', 'full').identity.principalId]",
"permissions": {
"keys": [
"wrapkey",
"unwrapkey",
"get"
],
"secrets": [],
"certificates": []
}
}
]
}
}Configuring the encryption key on the storage account
Now that we have our key and appropriate permission in Key Vault. We need to reference and configure the key to use on the storage account.
We need to set the keySource to Microsoft.Keyvault and fill Key Vault and key details. If keyVersion is left blank, it will use the latest version of the key. You can also set it to a specific version if you prefer that.
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[variables('uniqueResourceNameBase')]",
"apiVersion": "2019-06-01",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Keyvault",
"keyvaultproperties": {
"keyvaulturi": "[reference(resourceId('Microsoft.KeyVault/vaults', variables('uniqueResourceNameBase')),'2016-10-01', 'full').properties.vaultUri]",
"keyname": "[parameters('keyName')]",
"keyversion": "[parameters('keyversion')]"
}
}
}
}Complete example
Here is a complete and functional ARM template that you need to deploy 2 times to fully configure it all. The first run will create the Key Vault and storage account. You then need to manually create the key in Key Vault and run the deployment again to let it do the rest of the work.
This template use a nested deployment to seperate the configuration of the storage account with key information from the creation of the 2 resources to have a complete example.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyName": {
"type": "string",
"defaultValue": "storage-enc-test"
},
"keyVersion": {
"type": "string",
"defaultValue": ""
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]"
},
"configureEncryptionKey": {
"type": "bool",
"defaultValue": false
}
},
"variables": {
"uniqueResourceNameBase": "[uniqueString(resourceGroup().id, parameters('location'), deployment().name)]"
},
"resources": [
{
"condition": "[not(parameters('configureEncryptionKey'))]",
"type": "Microsoft.KeyVault/vaults",
"name": "[variables('uniqueResourceNameBase')]",
"apiVersion": "2016-10-01",
"location": "[parameters('location')]",
"properties": {
"sku": {
"family": "A",
"name": "standard"
},
"tenantId": "[subscription().tenantid]",
// !BEWARE! Running this property with an empty bracket will remove all your existing access policies
// If you need to update the required properties for storage encryption,
// you can fetch existing access policies via script and pass them as template parameter
"accessPolicies": [],
"enableSoftDelete": true,
"enablePurgeProtection": true
},
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('uniqueResourceNameBase'))]"
]
},
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[variables('uniqueResourceNameBase')]",
"apiVersion": "2019-06-01",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"supportsHttpsTrafficOnly": true
},
"dependsOn": []
},
{
"condition": "[parameters('configureEncryptionKey')]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-07-01",
"name": "updateStorageAccount",
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', variables('uniqueResourceNameBase'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "0.1.0.0",
"resources": [
{
"type": "Microsoft.KeyVault/vaults/accessPolicies",
"name": "[concat(variables('uniqueResourceNameBase'), '/add')]",
"apiVersion": "2019-09-01",
"properties": {
"accessPolicies": [
{
"tenantId": "[subscription().tenantid]",
"objectId": "[reference(resourceId('Microsoft.Storage/storageAccounts', variables('uniqueResourceNameBase')),'2019-06-01', 'full').identity.principalId]",
"permissions": {
"keys": [
"wrapkey",
"unwrapkey",
"get"
],
"secrets": [],
"certificates": []
}
}
]
}
},
{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[variables('uniqueResourceNameBase')]",
"apiVersion": "2019-06-01",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Keyvault",
"keyvaultproperties": {
"keyvaulturi": "[reference(resourceId('Microsoft.KeyVault/vaults', variables('uniqueResourceNameBase')),'2016-10-01', 'full').properties.vaultUri]",
"keyname": "[parameters('keyName')]",
"keyversion": "[parameters('keyversion')]"
}
}
},
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults/accessPolicies', variables('uniqueResourceNameBase'), 'add')]"
]
}
]
}
}
}
]
}1 - First run
Connect-AzAccount
New-AzResourceGroup -name mi-storage -location eastus
New-AzResourceGroupDeployment -ResourceGroupName mi-storage -TemplateFile C:\temp\deploy.json -Verbose
VERBOSE: Performing the operation "Creating Deployment" on target "mi-storage".
VERBOSE: 11:30:12 AM - Template is valid.
VERBOSE: 11:30:14 AM - Create template deployment 'deploy'
VERBOSE: 11:30:14 AM - Checking deployment status in 5 seconds
VERBOSE: 11:30:19 AM - Resource Microsoft.Storage/storageAccounts 'ic626scfz2hba' provisioning status is running
VERBOSE: 11:30:19 AM - Checking deployment status in 17 seconds
VERBOSE: 11:30:37 AM - Checking deployment status in 5 seconds
VERBOSE: 11:30:42 AM - Resource Microsoft.KeyVault/vaults 'ic626scfz2hba' provisioning status is running
VERBOSE: 11:30:42 AM - Resource Microsoft.Storage/storageAccounts 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:30:42 AM - Resource Microsoft.Storage/storageAccounts 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:30:42 AM - Checking deployment status in 15 seconds
VERBOSE: 11:30:57 AM - Resource Microsoft.KeyVault/vaults 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:30:57 AM - Resource Microsoft.KeyVault/vaults 'ic626scfz2hba' provisioning status is succeeded
DeploymentName : deploy
ResourceGroupName : mi-storage
ProvisioningState : Succeeded
Timestamp : 7/30/2020 3:30:55 PM
Mode : Incremental
TemplateLink :
Parameters :
Name Type Value
======================== ========================= ==========
keyName String storage-enc-test
keyVersion String
location String eastus
configureEncryptionKey Bool False
Outputs :
DeploymentDebugLogLevel :2 - Create your key manually.
3 - Last, run with the special flag configureEncryptionKey set to true to run the configuration step on the storage account. There is a condition on the nested deployment that will only execute if that template parameter is set to true.
See below
{
"condition": "[parameters('configureEncryptionKey')]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-07-01",
"name": "updateStorageAccount",
...
}Trigger the second deployment like this:
New-AzResourceGroupDeployment -ResourceGroupName mi-storage -TemplateFile C:\temp\deploy.json -configureEncryptionKey $true -Verbose
VERBOSE: Performing the operation "Creating Deployment" on target "mi-storage".
VERBOSE: 11:34:10 AM - Template is valid.
VERBOSE: 11:34:11 AM - Create template deployment 'deploy'
VERBOSE: 11:34:11 AM - Checking deployment status in 5 seconds
VERBOSE: 11:34:17 AM - Resource Microsoft.Resources/deployments 'updateStorageAccount' provisioning status is running
VERBOSE: 11:34:17 AM - Resource Microsoft.KeyVault/vaults/accessPolicies 'ic626scfz2hba/add' provisioning status is succeeded
VERBOSE: 11:34:17 AM - Resource Microsoft.KeyVault/vaults 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:34:17 AM - Resource Microsoft.KeyVault/vaults 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:34:17 AM - Resource Microsoft.Storage/storageAccounts 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:34:17 AM - Resource Microsoft.Storage/storageAccounts 'ic626scfz2hba' provisioning status is succeeded
VERBOSE: 11:34:17 AM - Checking deployment status in 5 seconds
VERBOSE: 11:34:22 AM - Checking deployment status in 5 seconds
VERBOSE: 11:34:28 AM - Checking deployment status in 6 seconds
VERBOSE: 11:34:34 AM - Checking deployment status in 6 seconds
DeploymentName : deploy
ResourceGroupName : mi-storage
ProvisioningState : Succeeded
Timestamp : 7/30/2020 3:34:39 PM
Mode : Incremental
TemplateLink :
Parameters :
Name Type Value
======================== ========================= ==========
keyName String storage-enc-test
keyVersion String
location String eastus
configureEncryptionKey Bool True
Outputs :
DeploymentDebugLogLevel :If I take a look in the Azure portal, we can see that the key has been correctly configured.
Hope you liked this article, again, thank you Soniya, today you made me learned something new, storage can have an identity :)
Happy ARM template!