A complete, step-by-step guide to deploying self-hosted GitHub runners as ephemeral containers on Azure — with automatic scaling from zero.
GitHub-hosted runners work well for most scenarios. But as workloads grow, teams often need:
- Better cost optimization — pay only when jobs run
- More control over execution environments and installed tools
- Scalable parallel execution — run 10, 20, or 50 jobs simultaneously
- Network access to private resources (databases, internal APIs)
Traditionally, this is solved using self-hosted runners on Virtual Machines. But VMs come with challenges — always-on cost, manual scaling, patching, and maintenance overhead.
In this guide, we'll walk through a modern, serverless alternative:
👉 Running self-hosted GitHub runners on Azure Container Apps Jobs with KEDA autoscaling
What You Will Build
By the end of this guide:
| Capability | What You Get |
|---|---|
| Runner type | Self-hosted, ephemeral (one job = one container) |
| Scaling | Automatic via KEDA — scales to zero when idle |
| Cost | Zero cost when no jobs are running |
| Infrastructure | Fully managed by Azure Container Apps |
| Security | Secrets stored in Azure Key Vault, Managed Identity for auth |
Architecture Overview
Here's how the system works at a high level:
How It Works (Step by Step):
- A developer pushes code or triggers a GitHub Actions workflow
- GitHub queues the job and looks for a runner with matching labels
- KEDA (built into Container Apps) polls the GitHub Actions API for pending jobs
- When a pending job is detected, KEDA triggers the Container App Job to start a new execution
- A fresh container starts, registers itself as a self-hosted runner with GitHub
- The runner picks up the job, executes it, and reports results back to GitHub
- The container shuts down and is destroyed — fully ephemeral
- When no jobs are pending, KEDA scales back to zero — no cost
Runtime Flow
Pre-requisites
Before you begin, make sure you have:
| Requirement | Details |
|---|---|
| GitHub account | With a repository or organization where you want to run workflows |
| Azure subscription | With permissions to create resources (Contributor role or higher) |
| Azure CLI | Installed locally, OR use Azure Cloud Shell (no install needed) |
| Basic knowledge | Familiarity with GitHub Actions and Azure Portal |
Where You'll Run Commands
Throughout this guide, you'll need a terminal to create files and run CLI commands. You have three options:
| Option | When to Use | How to Open |
|---|---|---|
| VS Code Terminal (Recommended) | You have VS Code installed locally | Open VS Code → Ctrl + `` (backtick) → Terminal opens at the bottom |
| Azure Cloud Shell | No local tools installed, or restricted machine | Go to portal.azure.com → click the >_ icon in the top toolbar |
| Any terminal | PowerShell, CMD, Bash — whatever you prefer | Just ensure Azure CLI (az) is installed |
💡recommended to use VS Code because you'll create files (Dockerfile, start.sh) AND run commands — VS Code lets you do both in one place.
Azure Resources We Will Create
| Resource | Purpose |
|---|---|
| Resource Group | Logical container for all resources |
| Azure Container Registry (ACR) | Stores the runner Docker image |
| Azure Container Apps Environment | Hosting environment for container jobs |
| Azure Container App Job | The actual runner job definition |
| Azure Key Vault | Securely stores the GitHub PAT token |
| Managed Identity | Allows the container job to access ACR and Key Vault without passwords |
💡 Note: This guide covers organization-level runners. For repository-level runners, the only difference is the GitHub API endpoint used for registration. We'll call out the differences where applicable.
Step 1: Create a GitHub Personal Access Token (PAT)
Before we touch Azure, we need a token that allows our runner to register with GitHub. You can use either a Fine-grained token (recommended) or a Classic token.
Option A: Fine-Grained PAT (Recommended)
Fine-grained tokens let you scope access to specific repositories only. This is critical for two reasons:
- Avoid GitHub API rate limits: KEDA continuously polls the GitHub API for pending jobs. If your token has access to your entire org (potentially hundreds of repos), KEDA scans all of them on every polling cycle. GitHub allows only 5,000 API requests/hour — with broad access, you'll hit this limit quickly and KEDA will stop detecting jobs.
- Security: Least-privilege access — the token only works on the repos you explicitly select.
🔑 This is why we recommend fine-grained tokens over classic tokens. By selecting only the repos that need runners, KEDA polls fewer repos and stays well within API limits.
- Go to github.com → Settings → Developer settings
- Click Personal access tokens → Fine-grained tokens
- Click Generate new token
- Fill in:
| Field | Value |
|---|---|
| Token name | container-app-runner |
| Expiration | Choose based on your needs (e.g., 90 days) |
| Resource owner | Your GitHub username or org |
| Repository access | Only select repositories → pick the repos where you want runners |
⚠️ IMPORTANT — Remember these repo names! The repos you select here are the ONLY repos this token can access. Later in Step 8, when you configure the KEDA scale rule, you must list these exact same repos in the repos metadata field. KEDA uses this token to poll GitHub for pending jobs — if a repo isn't included in the token, KEDA can't see its jobs and your runners won't scale for it.
Example: If you select my-app and my-api here, your KEDA config must have repos: my-app,my-api.
- Under Permissions, set the following:
Repository permissions (required):
| Permission | Access Level | Why It's Needed |
|---|---|---|
| Actions | Read and write | Manage workflow runs and artifacts |
| Administration | Read and write | Register and manage self-hosted runners |
| Metadata | Read-only | (Auto-selected, required) |
| Workflows | Read and write | Update GitHub Action workflow files |
Organization permissions (only if using org-level runners):
| Permission | Access Level | Why It's Needed |
|---|---|---|
| Self-hosted runners | Read and write | Register runners at the org level |
📝 For personal accounts (no org): You only need the Repository permissions above. Skip the Organization permissions.
- Click Generate token
- ⚠️ IMPORTANT: Copy the token NOW — you won't be able to see it again!
Option B: Classic PAT
If you prefer a classic token (simpler but broader access):
- Go to Settings → Developer settings → Personal access tokens → Tokens (classic)
- Click Generate new token (classic)
- Select these scopes:
| Scope | Why It's Needed |
|---|---|
| ✅ repo | Full access to repositories |
| ✅ workflow | Manage workflows |
| ✅ admin:org | Required for org-level runners (skip for personal repos) |
- Click Generate token and copy it immediately
⚠️ Classic tokens give access to ALL repositories in your account. For better security and to avoid API rate limits, prefer fine-grained tokens scoped to specific repos.
Save the token somewhere safe temporarily. We'll store it in Azure Key Vault in the next steps.
Where Will Runners Appear on GitHub?
Once runners are deployed and register with GitHub, you can see them here:
For repository-level runners:
- Go to your repo → Settings → Actions → Runners
For organization-level runners:
- Go to your org → Settings → Actions → Runners
Runners automatically register themselves when the container starts. You'll see them appear as "Idle" or "Active" in the Runners list. You do NOT need to manually create individual runners on GitHub.
Setting Up Runner Groups (Organization-Level) — Recommended
Since this guide is for organization-level runners, it's recommended to create a Runner Group on GitHub. Runner Groups let you control which repositories in your org can use these runners.
Steps to Create a Runner Group:
- Go to your GitHub Organization page (e.g., https://github.com/your-org)
- Click Settings (top menu bar)
- In the left sidebar, expand Actions → click Runner groups
- Click New runner group
- Fill in:
| Field | Value |
|---|---|
| Name | container-app-runners (or any descriptive name) |
| Repository access | Choose one: |
| • All repositories — any repo in the org can use these runners | |
| • Selected repositories — pick specific repos (recommended for control) | |
| Allow public repositories | Uncheck this for security (unless needed) |
| Workflow access | Leave default (all workflows) |
⚠️ CRITICAL: If ANY of the repositories using these runners are PUBLIC, you MUST check "Allow public repositories". If this is unchecked, GitHub will silently refuse to dispatch jobs from public repos to runners in this group — the runner will register and show as "Idle", but workflows will stay stuck in "Queued" forever. This is the most common and hardest-to-debug issue with runner groups.
- Click Create group
Why Create a Runner Group?
| Without Runner Group | With Runner Group |
|---|---|
| Any repo in the org can use your runners | Only selected repos can use them |
| Harder to track which teams use runners | Clear visibility and access control |
| Potential security risk for public repos | Can block public repos from using runners |
📝 For personal GitHub accounts: Runner groups are not available. Runners will automatically appear under your repo's Settings → Actions → Runners. No extra setup needed.
Step 2: Create Azure Resources (Portal)
We'll create all the Azure infrastructure through the Azure Portal. If you prefer CLI, see the CLI alternative at the end.
⚠️ IMPORTANT: All Azure resources (Resource Group, ACR, Key Vault, Container Apps Environment, Container App Job) must be in the SAME region. Pick one region (e.g., Central US or West US 2) and use it for everything. Mixing regions can cause connectivity issues and increased latency.
2.1: Create a Resource Group
- Go to Azure Portal
- Search for "Resource groups" in the top search bar
- Click + Create
- Fill in:
| Field | Value |
|---|---|
| Subscription | Select your Azure subscription |
| Resource group | rg-github-runners (or your preferred name) |
| Region | West US 2 (or any region that supports Container Apps) |
- Click Review + create → Create
2.2: Create an Azure Container Registry (ACR)
This is where we'll store our runner Docker image.
- Search for "Container registries" in the Azure Portal
- Click + Create
- Fill in:
| Field | Value |
|---|---|
| Subscription | Your subscription |
| Resource group | rg-github-runners |
| Registry name | yourregistryname (must be globally unique, lowercase, letters and numbers only) |
| Location | Same as your resource group (e.g., West US 2) |
| Pricing plan | Basic (sufficient for this guide; use Standard/Premium for production) |
- Click Review + create → Create
- Once created, note down the Login server (e.g., yourregistryname.azurecr.io) — you'll need this later
2.3: Create an Azure Key Vault
- Search for "Key vaults" in the Azure Portal
- Click + Create
- Fill in:
| Field | Value |
|---|---|
| Subscription | Your subscription |
| Resource group | rg-github-runners |
| Key vault name | kv-github-runners (must be globally unique) |
| Region | Same region |
| Pricing tier | Standard |
- Go to the Access configuration tab:
- Select Azure role-based access control (RBAC) as the permission model
- Click Review + create → Create
2.4: Store the GitHub PAT in Key Vault
- Open your newly created Key Vault
- First, give yourself permission:
- Go to Access Control (IAM) → + Add role assignment
- Role: Key Vault Secrets Officer
- Assign to: Your own Azure account
- Click Review + assign
- Now go to Objects → Secrets → + Generate/Import
- Fill in:
| Field | Value |
|---|---|
| Upload options | Manual |
| Name | github-pat |
| Secret value | Paste your GitHub PAT from Step 1 |
- Click Create
Step 3: Create the Runner Docker Image
This is the Docker image that will run as your GitHub Actions runner. We need two files: a Dockerfile and a start.sh script.
3.1: Set Up Your Working Directory
- Open VS Code on your local machine
- Open the integrated terminal: Ctrl + ` (backtick) or Terminal → New Terminal
- Create a new folder and navigate into it:
mkdir github-runner-image
cd github-runner-image
- You'll create two files in this folder: Dockerfile and start.sh
- In VS Code, click File → Open Folder and open the github-runner-image folder (so you can edit files easily)
📌 All commands in Step 3 and Step 4 should be run from inside this github-runner-image folder.
3.2: Choose Your Approach
There are two approaches to creating the runner image:
| Approach | Best For | Docker Required Locally? |
|---|---|---|
| Option A: Build locally with Docker | Development/testing | ✅ Yes |
| Option B: Build remotely with ACR Tasks | Production / no Docker access | ❌ No |
We'll create the same files for both approaches. The only difference is the build command.
3.3: Create the start.sh Script
This script runs when the container starts. It registers the runner with GitHub, executes the job, and then the container shuts down.
Create a file named start.sh:
#!/bin/bash
set -e
# ────────────────────────────────────────────
# CONFIGURATION
# ────────────────────────────────────────────
# These values are passed as environment variables
# GITHUB_PAT → Your GitHub Personal Access Token
# GITHUB_OWNER → Your GitHub org or username
# GITHUB_REPO → (Optional) Repository name for repo-level runners
# RUNNER_SCOPE → "org" or "repo"
# RUNNER_LABELS → Comma-separated labels (e.g., "container-app,linux")
# RUNNER_GROUP → Runner group name (org-level only, default: "Default")
# Set this to the runner group you created in Step 1 (e.g., "container-app-runners")
# If not set, runners register in GitHub's "Default" group — which means
# ANY repo in the org can use them and you lose access control.
RUNNER_SCOPE="${RUNNER_SCOPE:-org}"
RUNNER_LABELS="${RUNNER_LABELS:-container-app}"
RUNNER_GROUP="${RUNNER_GROUP:-Default}"
# ⚠️ IMPORTANT: Always set the RUNNER_GROUP environment variable on your Container App Job
# to match the runner group you created on GitHub (e.g., "container-app-runners").
# The "Default" fallback above is only a safety net — do NOT rely on it.
# ────────────────────────────────────────────
# GET REGISTRATION TOKEN
# ────────────────────────────────────────────
if [ "$RUNNER_SCOPE" == "org" ]; then
echo "🔑 Requesting registration token for organization: $GITHUB_OWNER"
REG_TOKEN=$(curl -s -X POST \
-H "Authorization: token $GITHUB_PAT" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/orgs/${GITHUB_OWNER}/actions/runners/registration-token" \
| jq -r .token)
RUNNER_URL="https://github.com/${GITHUB_OWNER}"
else
echo "🔑 Requesting registration token for repository: $GITHUB_OWNER/$GITHUB_REPO"
REG_TOKEN=$(curl -s -X POST \
-H "Authorization: token $GITHUB_PAT" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/repos/${GITHUB_OWNER}/${GITHUB_REPO}/actions/runners/registration-token" \
| jq -r .token)
RUNNER_URL="https://github.com/${GITHUB_OWNER}/${GITHUB_REPO}"
fi
if [ -z "$REG_TOKEN" ] || [ "$REG_TOKEN" == "null" ]; then
echo "❌ Failed to get registration token. Check your GITHUB_PAT and permissions."
exit 1
fi
echo "✅ Registration token obtained successfully"
# ────────────────────────────────────────────
# CONFIGURE RUNNER
# ────────────────────────────────────────────
echo "⚙️ Configuring runner..."
./config.sh --unattended \
--name "runner-$(hostname)" \
--url "$RUNNER_URL" \
--token "$REG_TOKEN" \
--runnergroup "$RUNNER_GROUP" \
--ephemeral \
--labels "$RUNNER_LABELS" \
--replace
echo "🚀 Starting runner..."
./run.sh
Key flags explained:
-
--ephemeral: Runner processes one job then exits (container stops)
-
--runnergroup: Registers the runner in a specific runner group (org-level only)
-
--replace: Replaces any existing runner with the same name
-
--unattended: No interactive prompts
⚠️ Do NOT use --disableupdate! In newer GitHub versions, this flag prevents GitHub from dispatching jobs to the runner. The runner will appear as "Idle" but never pick up work.
3.4: Create the Dockerfile
Create a file named Dockerfile:
FROM ubuntu:22.04
# Prevent interactive prompts during package installation
ENV DEBIAN_FRONTEND=noninteractive
# Install required dependencies
RUN apt-get update && apt-get install -y \
curl \
git \
jq \
ca-certificates \
unzip \
wget \
apt-transport-https \
software-properties-common \
&& rm -rf /var/lib/apt/lists/*
# Create a non-root user for the runner (GitHub requires this)
RUN useradd -m runner
# Set up the runner directory
WORKDIR /home/runner/actions-runner
# Download the latest GitHub Actions Runner
# Check latest version: curl -s https://api.github.com/repos/actions/runner/releases/latest | jq -r '.tag_name'
ARG RUNNER_VERSION=2.334.0
RUN curl -L -o actions-runner.tar.gz \
"https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz" \
&& tar xzf actions-runner.tar.gz \
&& rm actions-runner.tar.gz
# Install runner dependencies
RUN ./bin/installdependencies.sh
# Copy the startup script
COPY start.sh .
RUN chmod +x start.sh
# Set ownership to the runner user
RUN chown -R runner:runner /home/runner
# Switch to non-root user
USER runner
ENTRYPOINT ["./start.sh"]
💡 Tip: Check the latest runner version by running:
curl -s https://api.github.com/repos/actions/runner/releases/latest | jq -r '.tag_name'
At the time of writing, 2.334.0 is the latest. Update the ARG RUNNER_VERSION value if a newer version is available.
⚠️ Using a deprecated runner version will cause runners to connect but refuse to pick up jobs. You'll see the error: "Runner version vX.X.X is deprecated and cannot receive messages." Always use a recent version.
Step 4: Build and Push the Docker Image
Now we need to build this image and push it to your Azure Container Registry (ACR).
Option A: Build Locally with Docker (Development)
Use this if you have Docker Desktop installed on your machine.
Where to run: In the VS Code terminal (or any terminal), make sure you're inside the github-runner-image folder where your Dockerfile and start.sh are located.
# 0. First, log in to Azure (this opens a browser window for authentication)
az login
# 1. Log in to your ACR (replace yourregistryname with your actual ACR name)
az acr login --name yourregistryname
# 2. Build the image
docker build -t yourregistryname.azurecr.io/github-runner:v1 .
# 3. Push the image to ACR
docker push yourregistryname.azurecr.io/github-runner:v1
📌 Make sure Docker Desktop is running before you execute these commands. If you see "Cannot connect to the Docker daemon", start Docker Desktop first.
🔒 Don't have az login or Docker on your machine? Use Azure Cloud Shell instead — it's a browser-based terminal at shell.azure.com that comes pre-authenticated with Azure CLI (no az login needed) and has Docker available. See Option B below if you can't use Docker at all.
Option B: Build Remotely with ACR Tasks (Production — No Docker Needed) ⭐
This is the recommended approach for production environments where:
- You don't have Docker installed
- Your production environment is fully private / locked down
- You want to build images directly in Azure without any local tooling
Where to run:
- VS Code terminal → Run az login first, then the build command
- Azure Cloud Shell (shell.azure.com) → No az login needed, you're already authenticated. Upload your Dockerfile and start.sh files using the Upload button in Cloud Shell, then run the build command
You must be inside the folder where your Dockerfile and start.sh are located.
# If running from VS Code terminal (skip this line if using Azure Cloud Shell)
az login
# Build directly in ACR — no Docker required!
az acr build --registry yourregistryname --image github-runner:v1 --file Dockerfile .
☝️ That's it — one single command. No Docker install, no Docker daemon, nothing.
Using Azure Cloud Shell? To upload files:
-
Open shell.azure.com
-
Click the Upload/Download button (📁 icon) in the toolbar
-
Upload Dockerfile and start.sh
-
They'll land in your home directory (~/). Run az acr build from there.
This command:
- Uploads your source code to ACR
- Builds the Docker image in Azure (not on your machine)
- Tags and stores it in your registry
- No Docker daemon needed at all!
🔒 Production Note: In locked-down environments where even az acr build isn't possible from your machine, you can:
-
Use Azure Cloud Shell (browser-based, always has Azure CLI)
-
Set up an ACR Task with a Git trigger — ACR automatically builds when you push to a repo
-
Use Azure DevOps / GitHub Actions pipeline to build and push the image
Example: Auto-build from a GitHub repo (single line for Cloud Shell)
az acr task create --registry yourregistryname --name build-runner-image --image "github-runner:{{.Run.ID}}" --context https://github.com/your-org/your-runner-repo.git --file Dockerfile --git-access-token YOUR_GITHUB_PAT
Verify the Image
After building, confirm the image exists in your registry:
Portal: Go to your ACR → Repositories → you should see github-runner listed
CLI:
az acr repository list --name yourregistryname --output table
Step 5: Create the Container Apps Environment
The Container Apps Environment is the hosting platform for your container jobs. Think of it as the "cluster" where your runners will live.
Steps (Azure Portal):
- Search for "Container Apps Environment" in the Azure Portal
- Click + Create
- Fill in:
| Tab | Field | Value |
|---|---|---|
| Basics | Subscription | Your subscription |
| Resource group | rg-github-runners | |
| Environment name | cae-github-runners | |
| Region | West US 2 (same as other resources) | |
| Environment type | Consumption only (or Consumption + Dedicated if you need workload profiles) | |
| Monitoring | Log Analytics workspace | Create new or select existing |
- Leave Networking as defaults for now (we'll discuss production networking later)
- Click Review + create → Create
⏳ This takes 1-2 minutes to create.
Step 6: Create the Container App Job
This is the core resource — the Container App Job that will run your GitHub runners.
⚠️ IMPORTANT: Complete Step 4 (Build & Push Image) BEFORE this step. The Container App Job creation form requires you to select a container image from your ACR. If your ACR is empty (no images pushed), the portal will show an error: "The ACR does not have any images. Please push an image to the ACR and try again." So make sure your image is pushed first!
Steps (Azure Portal):
- Search for "Container App Jobs" in the Azure Portal search bar
- Click + Create Container App Job
Tab 1: Basics
| Field | Value | Notes |
|---|---|---|
| Subscription | Your subscription | |
| Resource group | rg-github-runners | Same RG as other resources |
| Container app job name | github-runner-job | Lowercase, letters, numbers, hyphens |
| Region | West US 2 | Must match your CAE region |
| Container Apps Environment | cae-github-runners | Select the environment created in Step 5 |
Click Next: Container >
Tab 2: Container
| Field | Value | Notes |
|---|---|---|
| Name | github-runner | Name of the container within the job |
| Image source | Azure Container Registry | Select this radio button |
| Registry | yourregistryname.azurecr.io | Select your ACR from the dropdown |
| Image | github-runner | Select the image you pushed |
| Image tag | v1 | The tag you used during build |
| Registry authentication | Managed identity | Leave as default |
| Managed identity | System assigned Identity (environment) | Leave as default — Azure will auto-assign ACR Pull role |
| Command override | (Leave empty) | Our Dockerfile already has an ENTRYPOINT |
| Arguments override | (Leave empty) | Not needed |
| Workload profile | Consumption | Default is fine |
| CPU and memory | 0.5 CPU cores, 1 Gi memory | Increase if your jobs need more resources |
💡 CPU/Memory guidance:
- 0.5 CPU / 1 Gi — Light jobs (linting, simple tests)
- 1 CPU / 2 Gi — Medium jobs (building apps, running test suites)
- 2 CPU / 4 Gi — Heavy jobs (compiling large projects, ML workloads)
Environment Variables
Click + Add for each variable:
| Name | Source | Value |
|---|---|---|
| GITHUB_OWNER | Manual | Your GitHub org name (e.g., Quality-Framework) |
| RUNNER_SCOPE | Manual | org (or repo for repo-level runners) |
| RUNNER_LABELS | Manual | container-app |
| GITHUB_REPO | Manual | Comma-separated repo names that this runner will serve (e.g., my-app,my-api,my-infra). These should match the repos selected in your PAT (Step 1). |
| RUNNER_GROUP | Manual | The GitHub runner group name (e.g., container-app-runners). Must match the group created in Step 1. If not set, defaults to Default. |
| GITHUB_PAT | (We'll configure this as a secret reference in Step 7) |
⚠️ Do NOT put the PAT directly as an environment variable value. We will securely reference it from Key Vault in Step 7.
For now, add only GITHUB_OWNER, RUNNER_SCOPE, RUNNER_LABELS, GITHUB_REPO, and RUNNER_GROUP. We'll add GITHUB_PAT after setting up the identity and secret.
⚠️ Make sure RUNNER_SCOPE matches runnerScope in the scale rule below. If you're using org-level runners, both should be org.
Scale Rule Settings (same page, scroll down)
Below the environment variables, you'll see Scale rule settings:
| Field | Value | Notes |
|---|---|---|
| Min executions | 0 | Scale to zero when no jobs are pending |
| Max executions | 5 | Maximum parallel runners (adjust as needed). Do NOT set to 0 — this means unlimited and can cause hundreds of runner containers to spawn! |
| Polling interval | 30 | How often (in seconds) KEDA checks for pending jobs |
Scale Rules — Click + Add
A side panel opens with the "Add scale rule" form. Fill in:
| Field | Value |
|---|---|
| Rule name | github-runner-rule |
| Custom rule type | github-runner |
Scale parameters (key-value pairs — click + Add for each):
| Name | Value | Notes |
|---|---|---|
| githubAPIURL | https://api.github.com | Pre-filled by the portal, leave as-is |
| owner | your-github-org | Your GitHub org or username (e.g., Quality-Framework) |
| runnerScope | org | org for org-level, repo for repo-level |
| repos | your-repo-name | Must match the repos you selected in your PAT (Step 1). Comma-separated, no spaces. |
| targetWorkflowQueueLength | 1 | Number of pending jobs needed to trigger one runner |
| labels | container-app | Must match RUNNER_LABELS env var and runs-on in your workflow |
⚠️ CRITICAL: Do NOT skip the labels parameter! Without it, KEDA cannot match pending jobs to your runner and will always show MetricValue: 0.00 — meaning no containers will ever start. This is the most common setup mistake.
Authentication:
| Secret reference | Trigger parameter |
|---|---|
| (Leave empty for now — we'll configure this after creating the job and setting up Key Vault secrets in Step 7) | personalAccessToken |
💡 The portal may let you save the scale rule without a secret reference. If it blocks you, delete the Authentication row entirely, click Add scale rule, and we'll add the authentication after the job is created.
Click Add scale rule → Then click Review + create → Create
⏳ Creation takes about 1 minute.
⚠️ First-time creation may fail with an image pull error if you're creating a new Container Apps Environment at the same time. This happens because the environment's managed identity gets the AcrPull role during deployment, but the image pull happens before the role propagates. If this occurs, simply Redeploy or create the Container App Job again — the role is already assigned and the second attempt will succeed.
Step 7: Configure Managed Identity and Secrets
Now we need to:
- Enable Managed Identity on the Container App Job
- Grant it access to Key Vault and ACR
- Reference the GitHub PAT as a secret
7.1: Enable System-Assigned Managed Identity
- Open your Container App Job (github-runner-job)
- In the left menu, go to Settings → Identity
- Under System assigned, toggle Status to On
- Click Save → Click Yes to confirm
- Note the Object ID that appears — you'll need this
7.2: Grant Key Vault Access
- Go to your Key Vault (kv-github-runners)
- Go to Access Control (IAM) → + Add role assignment
- Fill in:
| Field | Value |
|---|---|
| Role | Key Vault Secrets User |
| Assign access to | Managed identity |
| Members | Search for github-runner-job and select it |
- Click Review + assign
7.3: Grant ACR Pull Access
- Go to your Container Registry (yourregistryname)
- Go to Access Control (IAM) → + Add role assignment
- Fill in:
| Field | Value |
|---|---|
| Role | AcrPull |
| Assign access to | Managed identity |
| Members | Search for github-runner-job and select it |
- Click Review + assign
7.4: Add GitHub PAT as a Secret Reference
- Go back to your Container App Job (github-runner-job)
- In the left menu, go to Settings → Secrets
- Click + Add
- Fill in:
| Field | Value |
|---|---|
| Type | Key Vault reference |
| Key | github-pat |
| Key Vault secret URL | Select your Key Vault and the github-pat secret |
| Managed Identity | System assigned |
- Click Add
7.5: Map the Secret to an Environment Variable
- Go to Settings → Containers
- Click on your container → Edit
- Go to the Environment variables tab
- Click + Add:
| Name | Source | Value |
|---|---|---|
| GITHUB_PAT | Reference a secret | github-pat |
- Click Save
💡 If the Save button is greyed out, try making a small edit to another field first (e.g., click into a value and click out) to trigger the save state. Alternatively, re-create the container with the correct env vars.
Step 8: Configure KEDA Scale Rule (GitHub Runner Scaler)
This is where the magic happens. KEDA's GitHub Runner scaler monitors the GitHub Actions API for pending workflow jobs and scales your Container App Job accordingly.
Steps:
- Open your Container App Job (github-runner-job)
- Go to Settings → Scale (or Scale and replicas)
- Under Scale rule, click + Add
- Fill in the scale rule:
| Field | Value | Notes |
|---|---|---|
| Name | github-runner-rule | Any descriptive name |
| Type | Custom | Select Custom |
| Custom rule type | github-runner | This is the KEDA scaler type |
- Under Metadata, add these key-value pairs:
| Key | Value | Notes |
|---|---|---|
| owner | your-github-org | Your GitHub org or username (e.g., Quality-Framework) |
| repos | repo1,repo2 | Must match the repos you selected in your PAT token (Step 1). Comma-separated, no spaces. E.g., qualityframework-demo,qualityframework-bicep. Do NOT leave empty — if left empty, KEDA scans ALL org repos and you'll hit GitHub API rate limits. |
| runnerScope | org | org for org-level runners, repo for repo-level |
| labels | container-app | Must match the RUNNER_LABELS env var and the runs-on labels in your workflow YAML |
| targetWorkflowQueueLength | 1 | Number of pending jobs needed to trigger one new runner instance |
- Under Authentication, add:
| Key | Value |
|---|---|
| Secret reference | github-pat |
| Trigger parameter | personalAccessToken |
- Click Add → Save
Understanding the Scale Rule
| Scenario | KEDA Action |
|---|---|
| 0 pending jobs with container-app label | 0 runners (scale to zero) |
| 1 pending job | Starts 1 container |
| 3 pending jobs | Starts 3 containers (up to max) |
| Jobs complete | Containers stop, scale back to zero |
Step 9: Test the Setup
9.1: Create a Test Workflow
In any repository within your GitHub organization, create a workflow file:
File: .github/workflows/test-container-runner.yml
name: Test Container App Runner
on:
workflow_dispatch: # Allows manual trigger from GitHub UI
push:
branches: [main]
jobs:
test-runner:
runs-on: [self-hosted, container-app] # Must match your RUNNER_LABELS
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Print runner info
run: |
echo "✅ Hello from Azure Container App Runner!"
echo "Runner Name: $RUNNER_NAME"
echo "Runner OS: $RUNNER_OS"
echo "Workspace: $GITHUB_WORKSPACE"
- name: Run a simple test
run: |
echo "Current directory: $(pwd)"
echo "Files in repo:"
ls -la
echo "System info:"
uname -a
echo "Memory:"
free -h
9.2: Trigger the Workflow
- Go to your repository on GitHub
- Click Actions tab
- Select "Test Container App Runner" from the left sidebar
- Click Run workflow → Run workflow
9.3: Watch It Work
- In the GitHub Actions tab, you'll see the job show as "Queued"
- In Azure Portal, go to your Container App Job → Execution history
- Within ~30 seconds (your polling interval), you should see a new execution start
- The job will:
- Container starts → Runner registers → Job executes → Container stops
- Back in GitHub, the workflow run should show as ✅ completed
Troubleshooting
| Problem | Likely Cause | Fix |
|---|---|---|
| Job stays "Queued" forever | KEDA not detecting jobs | Check scale rule metadata — ensure labels, owner, repos, runnerScope are all filled correctly |
| Container starts but workflow doesn't complete | Wrong secret reference or empty env vars | Verify GITHUB_PAT env var points to correct secret, and all env vars have values |
| "Permission denied" errors | PAT missing required scopes | Edit PAT and add missing permissions (Step 1) |
| Image pull errors on first deploy | ACR access timing issue | Redeploy — the AcrPull role was assigned but hadn't propagated yet |
| Runner registers but job doesn't run | Label mismatch | Ensure runs-on labels in workflow match RUNNER_LABELS env var AND labels in KEDA scale rule |
| Runner shows "Idle" but job stays queued | --disableupdate flag or wrong runner group | Remove --disableupdate from start.sh and rebuild the image. Also verify RUNNER_GROUP env var matches the runner group name on GitHub, and the group has the correct repos assigned |
| Runner version deprecated error | Outdated runner binary | Update RUNNER_VERSION in Dockerfile to the latest version and rebuild. Run curl -s https://api.github.com/repos/actions/runner/releases/latest | jq -r '.tag_name' to check |
| Hundreds of offline runners spawning | maxExecutions set to 0 (unlimited) | Set maxExecutions to a reasonable limit (e.g., 5 or 10) in the scale rule settings |
| Runner idle + job queued forever (public repo) | Runner group blocks public repos | Go to Org Settings → Actions → Runner groups → your group and check "Allow public repositories". Without this, GitHub silently refuses to dispatch jobs from public repos to runners in the group |
To check container logs:
Go to your Container App Job → Monitoring → Logs and run:
ContainerAppConsoleLogs_CL | where ContainerGroupName_s startswith "github-runner-job" | where TimeGenerated > ago(30m) | order by TimeGenerated desc | take 20
Production Considerations
🔒 Networking: Private Environments
In production, your Container Apps Environment may be deployed inside a VNet with no public internet access. Here's how to handle that:
Private ACR Access
Container App Job ──(private endpoint)──► ACR
- Create a Private Endpoint for your ACR
- Disable public access on ACR
- Ensure your Container Apps Environment is in the same (or peered) VNet
Private Key Vault Access
Same pattern — create a Private Endpoint for Key Vault and disable public access.
GitHub API Access
Your runner containers need outbound access to:
- github.com (runner registration)
- api.github.com (KEDA polling)
- *.actions.githubusercontent.com (downloading actions)
If using a firewall or NSG, ensure these are allowed.
🔐 Using GitHub App Instead of PAT (Recommended for Production)
PATs are tied to individual users and have broad scopes. For production, consider using a GitHub App:
- Create a GitHub App in your organization
- Grant it Organization Self-hosted runners: Read & Write permissions
- Install the app in your organization
- Use the App ID and Private Key instead of PAT
The KEDA GitHub Runner scaler supports GitHub App authentication natively.
📊 Monitoring and Alerting
Set up monitoring for your runners:
- Container App Job Metrics (Azure Monitor):
- Execution count
- Execution duration
- Failed executions
- Alerts to set up:
- Alert when executions fail repeatedly
- Alert when execution queue is growing (KEDA can't keep up)
- Alert when runner registration fails (PAT expired?)
- Log Analytics queries:
// Find failed executions
ContainerAppConsoleLogs_CL
| where ContainerGroupName_s startswith "github-runner-job"
| where Log_s contains "error" or Log_s contains "failed"
| order by TimeGenerated desc
💰 Cost Optimization
| Strategy | Impact |
|---|---|
| Scale to zero (min: 0) | No cost when idle |
| Right-size CPU/memory | Don't over-provision |
| Set reasonable max executions | Prevent runaway costs |
| Use Consumption plan | Pay per-second billing |
| Set replica timeout | Kill stuck jobs |
🔄 Keeping the Runner Image Updated
Runner versions get outdated. Set up automated rebuilds:
# Create a scheduled ACR Task to rebuild weekly (single line for Cloud Shell)
az acr task create --registry yourregistryname --name rebuild-runner-weekly --image github-runner:latest --context https://github.com/your-org/runner-image-repo.git --file Dockerfile --schedule "0 0 * * 0" --git-access-token YOUR_PAT
Appendix A: CLI Commands for All Steps
If you prefer CLI over Portal, here are all the commands. Run these in Azure Cloud Shell or any terminal with Azure CLI.
All commands are written as single lines so they work directly in Azure Cloud Shell without formatting issues.
# ─── Set your variables (update these values) ───
RESOURCE_GROUP="rg-github-runners"
LOCATION="westus2"
ACR_NAME="yourregistryname"
KV_NAME="kvgithubrunners"
CAE_NAME="cae-github-runners"
JOB_NAME="github-runner-job"
IMAGE_NAME="github-runner"
IMAGE_TAG="v1"
GITHUB_ORG="your-github-org"
# ─── Step 1: Create Resource Group ───
az group create --name $RESOURCE_GROUP --location $LOCATION
# ─── Step 2: Create ACR ───
az acr create --name $ACR_NAME --resource-group $RESOURCE_GROUP --sku Basic
# ─── Step 3: Create Key Vault ───
az keyvault create --name $KV_NAME --resource-group $RESOURCE_GROUP --location $LOCATION --enable-rbac-authorization
# ─── Step 4: Store PAT in Key Vault ───
az keyvault secret set --vault-name $KV_NAME --name "github-pat" --value "YOUR_PAT_HERE"
# ─── Step 5: Build Image with ACR Tasks (run from the folder with Dockerfile) ───
az acr build --registry $ACR_NAME --image $IMAGE_NAME:$IMAGE_TAG .
# ─── Step 6: Create Container Apps Environment ───
az containerapp env create --name $CAE_NAME --resource-group $RESOURCE_GROUP --location $LOCATION
# ─── Step 7: Create Container App Job (single command) ───
az containerapp job create --name $JOB_NAME --resource-group $RESOURCE_GROUP --environment $CAE_NAME --trigger-type Event --replica-timeout 1800 --replica-retry-limit 1 --replica-completion-count 1 --parallelism 1 --image "$ACR_NAME.azurecr.io/$IMAGE_NAME:$IMAGE_TAG" --cpu "0.5" --memory "1Gi" --min-executions 0 --max-executions 5 --polling-interval 30 --scale-rule-name "github-runner-rule" --scale-rule-type "github-runner" --scale-rule-metadata "owner=$GITHUB_ORG" "runnerScope=org" "labels=container-app" "targetWorkflowQueueLength=1" --scale-rule-auth "personalAccessToken=github-pat" --secrets "github-pat=keyvaultref:https://$KV_NAME.vault.azure.net/secrets/github-pat,identityref:system" --env-vars "GITHUB_PAT=secretref:github-pat" "GITHUB_OWNER=$GITHUB_ORG" "RUNNER_SCOPE=org" "RUNNER_LABELS=container-app" "RUNNER_GROUP=container-app-runners" --registry-server "$ACR_NAME.azurecr.io" --registry-identity "system"
Appendix B: Repo-Level Runner Changes
If you want runners at the repository level instead of organization level:
| Setting | Org-Level | Repo-Level |
|---|---|---|
| PAT scope | admin:org | repo only |
| RUNNER_SCOPE env var | org | repo |
| GITHUB_REPO env var | Not needed | Required (e.g., my-repo) |
| KEDA runnerScope | org | repo |
| KEDA repos | Optional | Required |
Summary
Here's what we built:
| Component | What It Does |
|---|---|
| Dockerfile + start.sh | Creates an ephemeral GitHub Actions runner image |
| ACR | Stores the runner image securely |
| Key Vault | Stores the GitHub PAT securely |
| Container Apps Environment | Provides the hosting platform |
| Container App Job | Runs the runner containers on demand |
| KEDA Scale Rule | Automatically scales runners based on pending GitHub jobs |
| Managed Identity | Connects everything securely without passwords |
The Result:
✅ Zero cost when idle — no VMs running 24/7 ✅ Automatic scaling — KEDA handles it ✅ Ephemeral runners — clean environment every time ✅ Secure — secrets in Key Vault, Managed Identity for auth ✅ No Docker required — ACR Tasks builds images in the cloud ✅ Production ready — private networking, monitoring, automated image updates
-------------------------------------------------------------------------------------------
Have questions or feedback? Drop a comment below!
Tags: Azure, Container Apps, GitHub Actions, KEDA, Self-hosted Runners, DevOps, Serverless