Configure Kasm Workspaces AutoScale on Microsoft Azure

Overview

Kasm Workspaces deployments on Microsoft Azure face workspace demand that rises and falls throughout the day. This guide configures AutoScale to provision and destroy Azure virtual machine agents automatically in response to that demand. The result is a deployment that meets user demand without paying for idle compute. AutoScale supports bursty contractor, training, investigation, and AI workloads while reducing the operational effort of capacity management.

Prerequisites

• Access to a Microsoft Azure subscription with permission to register apps and assign roles.
• A working Kasm Workspaces deployment. See the AutoScale settings for Agent Pools or AutoScale settings for Server Pools to create an AutoScale configuration.
• The Azure CLI, available in the Azure portal Cloud Shell. See Microsoft's Cloud Shell quickstart for basic usage.
• Familiarity with two grouping concepts. Zones are logical groupings of Kasm services for geographical or organizational segmentation. Pools are logical groupings of Kasm Docker Agents and Server Pools for load balancing.

Solution approach

This guide progresses through the following phases:

1. Prepare the Azure environment and credentials.
2. Configure the Azure VM provider in Kasm.
3. Build a custom image for startup script execution.
4. Validate provisioning and downscaling.

Detailed steps

Prepare the Azure environment and credentials

Kasm requires an API credential to interface with Azure. Azure calls this credential an App Registration, which produces a service principal. Complete the following steps to register the app, create a client secret, and assign the required roles.

1. Register an app from the Microsoft Entra ID service in the Azure portal.

Microsoft Entra ID

2. From the Add dropdown, select App Registration.

App Registration

3. Give the app a human-readable name such as Kasm Workspaces.

App Registration name

4. Note the Client ID (Application ID) and Tenant ID (Directory ID) from the App Registration page for use later.
5. Create a client secret from the App Registration. Click Add a certificate or secret to the right of Client credentials.
6. Click New client secret under the Client secrets table.
7. Fill in the Description and Expires fields, then click Add.
8. Note the Value (not the Secret ID) for use later. The Value cannot be copied again afterward, so save it now.
9. Assign the required roles. Go to Resource Groups and select the Resource Group that Kasm will autoscale in.

Azure Resource Groups

10. Select Access Control (IAM).

Access Control

11. From the Add dropdown, select Add role assignment.

Add Role Assignment

12. Assign the Virtual Machine Contributor role first. On the next page, select the app by typing in the name, for example Kasm Workspaces.

Virtual Machine Contributor

Assign Contributor

13. Repeat the role assignment process to add the Network Contributor, DNS Zone Contributor, and Compute Gallery Image Reader roles.

Network Contributor

DNS Zone Contributor

Compute Gallery Image Reader

Configure the Azure VM provider in Kasm

With the Azure credentials and roles in place, create the VM Provider Config in Kasm. Follow AutoScale settings for Agent Pools or AutoScale settings for Server Pools to create a new AutoScale config. If an AutoScale configuration already exists, go to the All VM Provider Configs page, click Add, and select Add New under VM Provider Configs. Configure the following settings.

Azure VM

Name	Description
Name	A name to use to identify the config.
Subscription ID	The Subscription ID for the Azure Account. This can be found in the Azure portal by searching for Subscriptions in the search bar in Azure home then selecting the subscription to use. (e.g 00000000-0000-0000-0000-000000000000)
Resource Group	The Resource Group the DNS Zone and/or Virtual Machines belong to (e.g dev)
Tenant ID	The Tenant ID for the Azure Account. This can be found in the Azure portal by going to Microsoft Entra ID using the search bar in Azure home. (e.g 00000000-0000-0000-0000-000000000000)
Client ID	The Client ID credential used to auth to the Azure Account. Client ID can be obtained by registering an application within Microsoft Entra ID. (e.g 00000000-0000-0000-0000-000000000000)
Client Secret	The Client Secret credential created with the registered application in Microsoft Entra ID. (e.g abc123)
Azure Authority	Which Azure authority to use, there are four, Azure Public Cloud, Azure Government, Azure China and Azure Germany.
Region	The Azure region where the Agents will be provisioned. (e.g eastus)
Max Instances	The maximum number of Azure VMs to provision regardless of the need for additional resources.
VM Size	The size configuration of the Azure VM to provision (e.g Standard_D2s_v3)
OS Disk Type	The disk type to use for the Azure VM. (e.g Premium_LRS)
OS Disk Size	Select the boot volume size to assign the compute instance. The admin console accepts Bytes/GiB and stores the value as bytes (azure_os_disk_size_bytes). API integrations must submit the number of bytes in whole GiB (2^30) increments.
OS Image Reference (JSON)	The OS Image Reference configuration for the Azure VMs (e.g {"publisher":"canonical","offer":"0001-com-ubuntu-server-jammy","sku":"22_04-lts-gen2","version":"latest"} or {"id":"/subscriptions/000.../resourceGroups/dev/providers/Microsoft.Compute/galleries/development-gallery/img/ubuntu-22.04-custom"})
Image is Windows	Is this a windows VM being created
Plan Name	Plan Name for the VM image. This is optional, and is only required for images from the marketplace or custom images derived from a marketplace image.
Plan Product	Plan Product for the VM image. This is optional, and is only required for images from the marketplace or custom images derived from a marketplace image.
Plan Publisher	Plan Publisher for the VM image. This is optional, and is only required for images from the marketplace or custom images derived from a marketplace image.
Security Type	Which features used (if any) to security the VM. This is optional, but may be required for certain Windows VMs such as Windows 10 and Windows 11.
Secure Boot Enabled	Whether or not Secure Boot is enabled for the VM. This is not enabled by default, but may be required for certain Windows VMs such as Windows 11.
TPM Enabled	Whether or not TPM is enabled for the VM. This is not enabled by default, but may be required for certain Windows VMs such as Windows 11.
Network Security Group	The network security group to attach to the VM (e.g /subscriptions/000.../resourcegroups/dev/providers/Microsoft.Network/networkSecurityGroups/example-nsg)
Subnet	The subnet to attach the VM to (e.g /subscriptions/000.../resourceGroups/dev/providers/Microsoft.Network/virtualNetworks/development-vnet/subnets/default)
Assign Public IP	If checked, the VM will be assigned a public IP. If no public ip IP is assigned the VM must ne attached to a standard load balancer of the subnet must have a NAT Gateway or user-defined route (UDR). If a public IP is used, the subnet must not also include a NAT Gateway. Reference
Tags (JSON)	A JSON dictionary of custom tags to assign to the VMs (e.g {"foo":"bar", "bin": "baz"} )
OS Username	The login username to assign to the new VM (e.g testuser)
OS Password	The login password to assign to the new VM. Note: Password authentication is disabled for SSH by default
SSH Public Key	The SSH public key to install on the VM for the defined user: (e.g ssh-rsa AAAAAAA....)
Agent Startup Script	When instances are provisioned, this script is executed and is responsible for installing and configuring the Kasm Agent.
Config Override (JSON)	Custom configuration may be added to the provision request for advanced use cases. The emitted json structure is visible by clicking JSON View when inspecting the VM in the Azure console. The keys in this configuration can be used to update top level keys within the emitted json config (e.g {"location":"eastus"}). Nested items can be updated by using dot notation in the key (e.g {"hardware_profile.vm_size":"Standard_D4s_v3"}) Exiting array elements can be updated by specifying the index in the dot notation (e.g {"os_profile.linux_configuration.ssh.public_keys.0.path":"/home/ubuntu/.ssh/authorized_keys"})

caution

The following instructions use the Azure CLI. Open the CLI in the Azure web portal and run these commands directly. See Microsoft's Cloud Shell quickstart for basic usage of the Azure Cloud Shell environment.

1. Select Azure from the Provider dropdown.
2. Provide a name for the configuration.
3. Enter the Subscription ID, Resource Group, and Tenant ID from the Azure setup above.
4. Enter the Client ID and Client Secret created during the Azure setup above.
5. Select the Azure Authority from the dropdown. Most users select Azure Public Cloud.
6. Enter the region name. Use the following Azure CLI command to list available regions, then use the Name column.

az account list-locations -o table

7. Enter the Maximum Instances. This value sets an upper limit on the number of VMs this AutoScale configuration can create.
8. Enter a VM Size by name. Use the following Azure CLI command to list all VM sizes available in the desired region.

az vm list-sizes --location "westus"

9. Enter the OS Disk Type, using the Azure Disk Type SKU. The following table lists the current available type names. Not all disk SKUs are compatible with all VM size types.

Disk Type Name
Standard_LRS
Standard_GRS
Standard_RAGRS
Standard_ZRS
Premium_LRS
Premium_ZRS
Standard_GZRS
Standard_RAGZRS

10. Enter the OS Disk Size in gigabytes. The size must be at least the size of the target OS Image Reference, defined in the next field.
11. Enter the OS Image Reference JSON. Use the following Azure CLI command to list images in the JSON format expected by Kasm. These are default Azure-provided images. In most cases, you will need to create a custom image.

# List Windows Server Editions using the offline list (faster)
az vm image list --architecture x64 --location westus --offer WindowsServer
# Search the full Marketplace for Desktop versions of Windows
az vm image list --architecture x64 --location westus --all --publisher MicrosoftWindowsDesktop

The following example shows an OS Image Reference JSON value returned by one of the above queries that can be used.

{
    "architecture": "x64",
    "offer": "windows-ent-cpc",
    "publisher": "MicrosoftWindowsDesktop",
    "sku": "win11-22h2-ent-cpc-os",
    "urn": "MicrosoftWindowsDesktop:windows-ent-cpc:win11-22h2-ent-cpc-os:22621.963.221213",
    "version": "22621.963.221213"
}

12. Select the Image is Windows checkbox.
13. Optionally provide the Plan Name for the VM image. This is only required for images from the marketplace or custom images derived from a marketplace image.
14. Optionally provide the Plan Product for the VM image. This is only required for images from the marketplace or custom images derived from a marketplace image.
15. Optionally provide the Plan Publisher for the VM image. This is only required for images from the marketplace or custom images derived from a marketplace image.
16. Select which features secure the VM, if any. This is optional, but may be required for certain Windows VMs such as Windows 10 and Windows 11.
17. Select the Enable Secure Boot checkbox to enable Secure Boot for the VM. This is not enabled by default, but may be required for certain Windows VMs such as Windows 11.
18. Select the Enable TPM checkbox to enable TPM for the VM. This is not enabled by default, but may be required for certain Windows VMs such as Windows 11.
19. Enter the target Network Security Group. In the Azure portal, find the Network Security Group, go to Properties, and find the Resource ID field.
20. Enter the target Subnet. In the Azure portal, find the subnet, go to Properties, and find the Resource ID field.
21. Select the Assign Public IP checkbox to give VMs public IP addresses. Omitting public IP addresses is more secure. Kasm provides a secure way to reach private systems, so a public IP is not strictly necessary.
22. Optionally provide additional tags for the VM. Use empty {} brackets if no additional tags are needed.
23. Enter a public SSH Key. This is required even for Windows systems.
24. For Windows deployments, provide an AutoScale PowerShell startup script. For complete Windows, Linux, and Docker Agent examples, see the AutoScale Startup Scripts in the Kasm Workspaces GitHub repository. Use PowerShell for Windows or Bash/cloud-init for Linux.
25. Optionally enter a Config Override as JSON.
26. Submit the Provider Config.

Build a custom image for startup script execution

Unlike most other cloud providers, Azure does not automatically execute the startup script on boot. Create a custom image and implement your own method of executing that script on boot. The following method has been tested by Kasm, but it is not the only method that can be used. These procedures are only necessary to use Kasm's startup script injection and perform custom actions on boot of the VMs.

1. Manually create a VM using the appropriate base image, such as Windows 11 Enterprise.
2. Install the required software and configure the OS as appropriate for your environment.
3. Create a file with the following contents at C:\AzureData\startup.cmd.

schtasks /Delete /TN "DomainJoin" /F
cd C:\AzureData

setlocal EnableExtensions EnableDelayedExpansion
set tries=0

:CheckForFile
IF EXIST CustomData.bin GOTO FoundIt
set /a tries+=1
IF !tries! GEQ 40 GOTO Timeout
TIMEOUT /T 15 >nul
GOTO CheckForFile

:FoundIt
ren CustomData.bin CustomData.ps1
PowerShell -Command "Set-ExecutionPolicy Unrestricted"
PowerShell -file C:\AzureData\CustomData.ps1
GOTO End

:Timeout
REM Timed out waiting for CustomData.bin (40 x 15s)

:End

4. Open a Windows Command Prompt as an Administrator and run the following command.

schtasks /create /tn "DomainJoin" /sc onstart /rl highest /ru system /tr "cmd /c C:\AzureData\startup.cmd  > C:\AzureData\startup.log 2>&1"

5. Run sysprep on the VM and shut it down. Run the following in an elevated command prompt.

cd %windir%\system32\sysprep
sysprep.exe /oobe /generalize /mode:vm /shutdown

caution

You may get an error message when running the above command on a Windows 11 system using the default Azure-provided image. Look at the log file indicated by the error message. If you see an error message near the bottom of the logs indicating OneDriveSync was installed for the user but not provisioned for all users, remove the software from an elevated PowerShell session with the following command. Try sysprep again after running this command.

get-appxpackage -allusers -name "microsoft.Onedrivesync" | Remove-appxpackage

6. Open a Cloud Shell in Azure and run the following command to get a list of VMs and find the VM that you just ran sysprep on.

az vm list -otable

7. The table provided by the above command contains the VM name and its resource group name. Plug your VM name and resource group name into the command below to generalize the VM.

az vm generalize -g RESOURCE_GROUP_NAME -n VM_NAME

8. Create a Gallery, if one is not already available for the location of the custom image VM. Galleries are how images are shared and organized in Azure. Plug in your resource group name and your desired Gallery name.

az sig create --resource-group RESOURCE_GROUP_NAME --gallery-name GALLERY_NAME

9. Obtain the full ID of the VM you want to create an image from. Plug your resource group name and VM name into the command below.

az vm get-instance-view -g RESOURCE_GROUP_NAME -n VM_NAME --query id

The output should look similar to the following and is used in step 11.

"/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP_NAME>/providers/Microsoft.Compute/virtualMachines/<VM_NAME>"

10. From the Azure portal search bar, search for Azure compute galleries and find the Gallery you created in step 8. Click into that gallery. Click the dropdown on the Add button and select VM Image Definition. Provide a VM image definition name and select Windows as the OS Type. The Security Type and VM generation may differ based on the base image you started from. For Windows 11, select Trusted Launch for the Security Type, which locks you into VM generation 2. The Publisher, Offer, and SKU are found in the Azure portal on the VM page, under Source Image Details. After creating the image definition, note the name for the subsequent steps.

11. Create a Managed Image using the command below for your desired MANAGED_IMAGE_NAME and appropriate os-type and hyper-v-generation, depending on what was selected in step 10. The example below uses Windows and V2.

az image create --resource-group RESOURCE_GROUP_NAME --name MANAGED_IMAGE_NAME --source "VM_ID" --os-type Windows --hyper-v-generation V2 --location "eastus"

12. Create an Image version. Plug your resource group name, gallery name, image definition, desired version number, location, and target regions into the following Azure CLI command. For the managed-image field, use the managed image name you created in step 11. Target regions must include both the region your image is in and the region your resource group is in, if different.

az sig image-version create \
--resource-group RESOURCE_GROUP_NAME \
--gallery-name GALLERY_NAME \
--gallery-image-definition IMAGE_DEFINITION_NAME \
--gallery-image-version 1.0.0 \
--location "eastus"
--target-regions "eastus" \
--replica-count 2 \
--managed-image MANAGED_IMAGE_NAME

Note the ID returned from the above command for the next step.

13. In the VM Provider configuration, step 11, put the ID returned by the previous step into the OS Image Reference field. The following is an example.

{
"id": "/subscriptions/<Subscription_ID>/resourceGroups/<RESOURCE_GROUP_NAME>/providers/Microsoft.Compute/galleries/<GALLERY_NAME>/img/IMAGE_DEFINITION_NAME/versions/1.0.0"
}

Validate provisioning and downscaling

After configuring the provider, confirm that Kasm provisions and removes agents as demand changes. If you configured non-zero Standby or Minimum Available Session values, agents start provisioning immediately. Otherwise, launch multiple workspaces to increase resource utilization, which prompts Kasm to autoscale new agents.

1. Provision a Workspace. Go to Workspaces > Registry and make multiple workspaces available.
2. Go to the Workspaces dashboard and launch enough workspace sessions to exceed your resource standby thresholds.
3. Monitor the provisioning of new agents under Infrastructure > Agents.
4. Verify new VM instances in Microsoft Azure.
5. Check downscaling. Terminate sessions to reduce resource usage.
6. Confirm that Kasm removes agents after the back-off period.

Common troubleshooting steps

• VMs provision but the startup script does not run. Azure does not execute the startup script automatically. Confirm that you built and selected a custom image that runs the script on boot.
• VMs fail to provision. Confirm that the App Registration holds the Virtual Machine Contributor, Network Contributor, DNS Zone Contributor, and Compute Gallery Image Reader roles on the target Resource Group.
• Authentication fails. Verify the Subscription ID, Resource Group, Tenant ID, Client ID, and Client Secret Value. The Client Secret Value cannot be retrieved after creation, so generate a new secret if it was not saved.
• sysprep fails on Windows 11. Review the log file in the error message. Remove the OneDriveSync package with get-appxpackage -allusers -name "microsoft.Onedrivesync" | Remove-appxpackage, then run sysprep again.
• VM size or disk type is rejected. Not all disk SKUs are compatible with all VM size types. Confirm the VM size is available in the target region with az vm list-sizes and choose a compatible disk type.