# Compare CI providers Try an existing build or test job on another provider before changing where it normally runs. Set up a repository with one pull request. Mirage routes the jobs you select through your machines, and each keeps the runner it uses today as its fallback. Afterwards, compare any of those jobs on GitHub, local infrastructure, Cloudflare, or Blacksmith, and promote a supported provider through a reviewed pull request. Each experiment runs an extra copy of a completed job on the same commit. The original run and its required checks remain unchanged. [Routing and providers](https://mirageci.com/docs/routing.md) explains how comparisons, defaults, and fallback fit together. For recommendations that Mirage discovers and tests automatically, see [Automatic CI improvements](https://mirageci.com/docs/ci-improvements.md). That flow includes dependency caching and uses separate service-funded trials, with optional Claude Code or ChatGPT proposal generation. ## Choose a job Start with build or test jobs on a runner whose image matches Mirage's machines: `ubuntu-latest`, `ubuntu-24.04`, `ubuntu-24.04-arm`, `macos-latest`, or `macos-26`, or a Blacksmith label on Ubuntu 24.04 or the latest or macOS 26 image, such as `blacksmith-4vcpu-ubuntu-2404`. Jobs using [cloudflare-github-actions-runner](https://github.com/biw/cloudflare-github-actions-runner) are also supported: `cloudflare-ubuntu-latest`, the `cloudflare-lite`, `cloudflare-basic`, and `cloudflare-standard-1` through `cloudflare-standard-4` presets, and valid custom sizes such as `cloudflare-vcpu:2-memory_mib:6144-disk_mb:12000` all map to Linux x64. Mirage keeps the exact original Cloudflare label as the fallback, including its custom size; keep the standalone runner's GitHub App and pool configured to serve that fallback. Jobs pinned to another image, such as `ubuntu-22.04` or `macos-15`, keep running where they are, because your machines would run a different OS. The repository must be private and selected for the Mirage GitHub App. Set up an available machine or Cloudflare account for the provider you want to try. Job discovery checks whether Mirage can route the job and, separately, whether it can copy the job into an independent experiment workflow. Mirage does not route jobs that already call reusable workflows, deployment and environment jobs, jobs whose name is computed with an expression, or matrix jobs whose runners are computed rather than listed. A matrix job whose children all use one runner is routed as a whole: every child runs on your machines and falls back to that runner on its own. A matrix that picks its runner with `runs-on: ${{ matrix.os }}` is routed child by child: each child runs on the Mirage machine type for its own runner, such as Linux for `ubuntu-latest` or macOS for `macos-latest`, and falls back to that same runner. Children on runners Mirage cannot serve, such as Windows, keep running where they do now. Matrix jobs, and jobs that read event context, define outputs, set job-level concurrency, or live in a workflow with a manual `workflow_dispatch` trigger, are routed without an experiment workflow, so they cannot be compared across providers. Discovery explains the reason per job. This is an optional setup path for existing GitHub-hosted, Blacksmith, or standalone Cloudflare jobs. A job already using the attempt expression from [Configure workflows](https://mirageci.com/docs/workflow-routing.md) is not eligible for this onboarding flow. Use a separate eligible job or restore its static GitHub-hosted label before onboarding it. ## Set up the repository You don't edit `runs-on` yourself: the dashboard walks you through the setup and opens the pull request for you. 1. Open [Repos](https://mirageci.com/repos) and select the repository. A repository that needs setup opens **Set up routing**; for one that already runs jobs through Mirage, choose **Set up routing** on its page. 2. **Ready check.** Mirage checks that the GitHub App can open pull requests in the repository and that a machine available to the account has each runner type the selected jobs need **Ready**. Routed jobs, other than matrices that pick their runner with `matrix.`, also need a Linux x64 machine for the routing step that runs first. Each problem names its fix, such as **Pair a machine**, **Open machine** for a runner that isn't ready, or **Review on GitHub** for App permissions you haven't approved. 3. **Jobs.** Mirage reads the default branch's workflows and lists each job with what it runs on now, where it will run on Mirage, and the runner it falls back to. Every eligible job starts selected; clear any you want to leave alone. Jobs Mirage won't route say why. 4. **Setup pull request.** Once the ready check passes, choose **Open setup PR**. Mirage opens one pull request for the whole repository, and the page follows it: its checks running on your machines, the merge, and the routes applying. 5. Review the pull request in GitHub, then merge it yourself; Mirage never merges it. The page shows the routes as applied, and the next runs use your machines. For each selected job, the pull request changes `runs-on` to Mirage's routing expression, adds the `.github/workflows/mirage.yml` routing workflow, and adds a standalone experiment workflow. Each job falls back to the runner it used before (a GitHub-hosted label, or its Blacksmith or standalone Cloudflare label) when its runner fails or no machine takes it within the repository’s [queue timeout](https://mirageci.com/docs/workflow-routing.md#queue-timeout). Later changes to where jobs run, such as retry policy or which machines serve them, need no further pull requests. If Mirage cannot run a job while the setup pull request is still open, that check fails with an explanation instead of falling back: fallback starts working once the routing workflow is on your default branch. That is why the ready check waits for your machines before the pull request opens. Routed jobs run on your machines only; Cloudflare stays available for [experiments](#run-an-experiment) and [direct Cloudflare jobs](https://mirageci.com/docs/cloudflare.md). ## Compare providers 1. In Runs, open a completed job that has finished onboarding. 2. Select **Experiment**, choose the available providers, and launch the comparison. 3. Compare each provider’s result and available capacity-wait, startup, and execution timings. Follow the GitHub links for full logs. Experiments use the source job’s commit, so a newer push does not change what is being compared. They consume runner capacity and any applicable provider usage. An unavailable provider or a failed experiment does not cancel the source run or other providers in the comparison. Only compare jobs whose steps are safe to run again. An experiment executes your code again even though its result does not gate the source pipeline. ## Change the default When a supported provider succeeds, select **Promote** in the comparison. Mirage opens a pull request to change the job’s runner selection. Review and merge that PR, then check Settings for the applied default and fallback before relying on the new route. A comparison alone does not change future jobs. Cloudflare is currently available only for experiments in this flow, so it has no promotion action. To use it directly, configure an explicit [Cloudflare workflow](https://mirageci.com/docs/cloudflare.md). ## Blacksmith Blacksmith runs jobs through its own GitHub App, so Mirage uses your Blacksmith account rather than providing capacity itself. Install Blacksmith’s GitHub App on each repository you compare, and run at least one job on a `blacksmith-*` label. Mirage offers Blacksmith in comparisons once it has seen Blacksmith run a job for your account in the last 30 days. A Blacksmith comparison uses its smallest runner for the job’s profile. If Blacksmith does not pick the job up within 10 minutes, usually because its app cannot see the repository, Mirage cancels that comparison, records Blacksmith as unavailable, and adds a failed **Mirage (experiment) - Blacksmith did not start the comparison** check to the commit. Its details explain what happened and how to retry the comparison. Billing shows Blacksmith as its own source, priced from Blacksmith’s published per-minute rates before its free monthly minutes.