Design and implement pipelines
The single densest sub-objective in the exam: which automation platform, what compute runs the work, what makes it start, how the YAML is structured, how work runs in parallel, and how you avoid writing the same pipeline nine times.
Selecting a deployment automation solution
only option] A -->|Azure Repos / Bitbucket| P A -->|GitHub| B{What matters most?} B -->|GitHub-native ecosystem
Dependabot · GHAS · Projects| G[GitHub Actions] B -->|Rich release management
stages · gates · deployment groups| H[Azure Pipelines
via GitHub App] B -->|Open source project| G
Caveat — the three decisive facts
1) GitHub Actions cannot build from TFVC — a legacy TFVC repo forces Azure Pipelines. 2) Azure Pipelines can build from GitHub, but not the reverse. 3) Only Azure Pipelines has stages, classic release gates and deployment groups. Everything else is largely preference.
Agent and runner infrastructure
Microsoft-hosted agents / GitHub-hosted runners
| Hosted | Self-hosted | |
|---|---|---|
| Maintenance | None — fresh VM per job | You patch, update and monitor it |
| State between runs | None — clean image every time | Persists — caches, Docker layers, tool installs |
| Private network access | No (public internet only) | Yes — the decisive advantage |
| Custom software / licensed tools | Only what's in the image | Anything you install |
| Cost model | Per-minute / parallel job purchase | Infrastructure + management effort |
| Best for | Short, standard, bursty builds | Long or frequent builds, private resources, compliance |
| Common image label | Notes |
|---|---|
ubuntu-latest | Cheapest and fastest; required for Linux containers and service containers |
windows-latest | Windows Server with VS Build Tools; ~2× the minute cost on GitHub |
macos-latest | Xcode; ~10× the minute cost on GitHub — use only for Apple builds |
Caveat — when the answer is "self-hosted"
Any scenario mentioning on-premises resources, a private VNet, a private SQL Server, a licensed proprietary tool, data that may not leave the network, or "the build takes 40 minutes mostly downloading dependencies" resolves to a self-hosted agent/runner. Hosted agents cannot reach private endpoints and start from a clean image every time.
Deploying self-hosted agents and runners
mkdir myagent && cd myagent
tar zxvf ~/Downloads/vsts-agent-linux-x64-*.tar.gz
# Configure. Prefer a service principal / managed identity over a PAT where supported.
./config.sh --unattended \
--url https://dev.azure.com/MyOrg \
--auth pat --token "$AZP_TOKEN" \
--pool MyLinuxPool \
--agent "$(hostname)" \
--acceptTeeEula
sudo ./svc.sh install && sudo ./svc.sh start # run as a service
pool:
name: MyLinuxPool # self-hosted pool
demands:
- Agent.OS -equals Linux
- HasDocker # custom capability set on the agent
- java -equals 21
| Scaling approach | Platform | Use for |
|---|---|---|
| Azure VM Scale Set agents | Azure Pipelines | Elastic self-hosted pool that Azure DevOps scales for you |
| Container jobs | Both | Run steps inside a specified image for a reproducible toolchain |
| Actions Runner Controller (ARC) | GitHub | Kubernetes-based ephemeral runners that autoscale to zero |
| Larger runners | GitHub | More vCPU/RAM, and static IP ranges for firewall allow-listing |
| Runner groups | GitHub | Restrict which repos/workflows may use sensitive runners |
Never use self-hosted runners on public repositories
A fork can open a PR that executes arbitrary code on your runner, and self-hosted runners persist state between jobs. GitHub explicitly recommends self-hosted runners only for private repositories; where they must be used, make them ephemeral (destroyed after each job) and place them in a restricted runner group.
Parallel jobs
Azure DevOps bills concurrency as parallel jobs: private projects get 1 free Microsoft-hosted parallel job with limited monthly minutes (self-hosted gets 1 free), and public projects get 10 free. Buying more parallelism — not a faster agent — is the fix for "builds queue for 20 minutes". See 3.6.
Integrating GitHub repositories with Azure Pipelines
| Connection type | How it authenticates | Recommended? |
|---|---|---|
| Azure Pipelines GitHub App | App installation with scoped repo permissions; posts checks natively | Yes — least privilege, no user token |
| OAuth | Acts as the authorising user's account | Fallback; breaks if that user leaves |
| Personal access token | A stored PAT | Last resort; rotation burden |
pr:
branches:
include: [main, 'release/*']
paths:
exclude: ['docs/*', '**/*.md']
drafts: false # don't burn agent minutes on draft PRs
Caveat — GitHub PR triggers ignore the YAML branch filter for the target
For GitHub repos, Azure Pipelines PR triggers are driven by the GitHub App/branch protection, and comment triggers (/azp run) can be required for PRs from forks. Also, secrets are not exposed to fork PR builds by default — a scenario about "a fork PR build fails because it can't read a secret" is working as designed, and the answer is to not loosen it.
Pipeline trigger rules
# CI trigger
trigger:
batch: true # queue changes while a run is in progress
branches:
include: [main, 'release/*']
exclude: ['experimental/*']
paths:
include: ['src/*']
exclude: ['docs/*', '**/*.md']
tags:
include: ['v*']
# trigger: none → disable CI entirely
pr:
branches: { include: [main] }
autoCancel: true # cancel older runs on a new push
schedules:
- cron: '0 2 * * 1-5' # 02:00 UTC, Mon–Fri
displayName: Nightly
branches: { include: [main] }
always: true # run even with no code changes
# Pipeline-completion trigger
resources:
pipelines:
- pipeline: buildArtifacts
source: 'MyApp-CI'
trigger:
branches: { include: [main] }
on:
push:
branches: [main, 'release/**']
paths-ignore: ['docs/**', '**/*.md']
tags: ['v*']
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review]
schedule:
- cron: '0 2 * * 1-5' # always UTC
workflow_dispatch: # manual, with inputs
inputs:
environment:
description: Target environment
required: true
default: staging
type: choice
options: [staging, production]
workflow_call: # reusable workflow
repository_dispatch: # external API trigger
types: [deploy-prod]
workflow_run: # after another workflow finishes
workflows: ['CI']
types: [completed]
| Requirement | Azure Pipelines | GitHub Actions |
|---|---|---|
| Manual run with parameters | parameters: + queue-time UI | workflow_dispatch: with inputs: |
| Trigger from an external system | REST API / webhook resource | repository_dispatch |
| Chain after another pipeline | resources: pipelines: trigger | workflow_run |
| Skip docs-only changes | paths: exclude: | paths-ignore: |
| Run nightly regardless of changes | schedules: + always: true | schedule: (always runs) |
| Cancel superseded PR runs | pr: autoCancel: true | concurrency: cancel-in-progress: true |
Caveats — three trigger traps
1) cron is always UTC in both platforms — a "9 am local" requirement needs the UTC offset, and daylight saving will shift it. 2) Azure Pipelines schedules: without always: true only runs if the source changed since the last scheduled run. 3) GitHub disables scheduled workflows in repositories with no activity for 60 days.
Developing pipelines with YAML
The object hierarchy
name: $(Date:yyyyMMdd)$(Rev:.r)
trigger:
branches: { include: [main] }
variables:
- name: buildConfiguration
value: Release
- group: 'Shared-Settings' # variable group from Library
- template: vars/common.yml # variables from a template file
stages:
- stage: Build
displayName: Build and test
jobs:
- job: BuildJob
pool: { vmImage: ubuntu-latest }
steps:
- checkout: self
fetchDepth: 0 # full history: needed by GitVersion / SonarQube
- template: templates/build-steps.yml
parameters:
buildConfiguration: $(buildConfiguration)
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: drop
- stage: DeployStaging
dependsOn: Build
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
jobs:
- deployment: DeployStaging # a DEPLOYMENT job, not a plain job
environment: staging
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: drop
- script: ./deploy.sh $(Pipeline.Workspace)/drop
- stage: DeployProd
dependsOn: DeployStaging
jobs:
- deployment: DeployProd
environment: production # approvals/checks configured on the environment
strategy:
runOnce:
deploy:
steps:
- script: ./deploy.sh
Caveat — deployment: vs job:
A deployment job (- deployment:) is what binds to an environment:, records deployment history, exposes strategies (runOnce, rolling, canary) with lifecycle hooks, and automatically downloads the current pipeline's artifacts. A plain - job: does none of that. If the requirement mentions approvals, deployment history or a rollout strategy, it must be a deployment job.
Variables, parameters and expressions
| Syntax | Evaluated | Use for |
|---|---|---|
${{ }} — compile time | When the YAML is parsed, before the run starts | parameters, conditional structure (adding/removing steps), template logic |
$[ ] — runtime expression | At the start of the stage/job | counter(), dependencies.* outputs, conditions on prior results |
$( ) — macro | Just before each step executes | Everyday variable substitution inside step inputs |
Caveat — parameters vs variables
Parameters are typed, are resolved at compile time, and can change the shape of the pipeline (add a stage, drop a step). Variables are always strings and are resolved at runtime — they can change values but not structure. "Let the user pick which environments get a stage" → parameters. "Pass a connection string to a task" → variable.
jobs:
- job: Setup
steps:
- script: echo "##vso[task.setvariable variable=version;isOutput=true]1.4.2"
name: setVar # the step NAME is required to reference the output
- job: Use
dependsOn: Setup
variables:
ver: $[ dependencies.Setup.outputs['setVar.version'] ]
steps:
- script: echo "Building $(ver)"
jobs:
setup:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.v.outputs.value }}
steps:
- id: v
run: echo "value=1.4.2" >> "$GITHUB_OUTPUT"
use:
needs: setup
runs-on: ubuntu-latest
steps:
- run: echo "Building ${{ needs.setup.outputs.version }}"
Job execution order, parallelism and multi-stage
jobs:
- job: TestLinux
steps: [ { script: dotnet test } ]
- job: TestWindows
steps: [ { script: dotnet test } ]
- job: Publish
dependsOn: [TestLinux, TestWindows]
condition: always() # run even if a test job failed
steps:
- script: echo "aggregate"
# dependsOn: [] → start immediately, ignore implicit ordering
jobs:
test-linux:
runs-on: ubuntu-latest
steps: [ { run: npm test } ]
test-windows:
runs-on: windows-latest
steps: [ { run: npm test } ]
publish:
needs: [test-linux, test-windows]
if: always()
runs-on: ubuntu-latest
steps: [ { run: echo aggregate } ]
Caveat — default ordering differs by level
In Azure Pipelines, stages run sequentially by default (each implicitly depends on the previous), while jobs within a stage run in parallel by default. To make stages parallel, set dependsOn: []. In GitHub Actions, all jobs run in parallel unless linked with needs:. Getting this backwards is a common mistake.
Conditions cheat sheet
Azure Pipelines: succeeded() (default), succeededOrFailed(), failed(), always(), canceled(), and expressions like and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main')). GitHub Actions: success() (default), failure(), always(), cancelled() inside if:. Note always() also runs on cancellation — use succeededOrFailed()/!cancelled() when you do not want that.
Reusable pipeline elements
| Mechanism | Platform | Reuses |
|---|---|---|
| Steps template | Azure Pipelines | A sequence of steps, inserted into a job |
| Jobs / stages template | Azure Pipelines | Whole jobs or stages |
| Variables template | Azure Pipelines | A block of variables |
extends template | Azure Pipelines | The entire pipeline shape — the governance mechanism |
| Task group | Azure Pipelines (classic only) | A reusable set of tasks in the UI; becomes a template on migration |
| Variable group | Azure Pipelines | Shared values/secrets, optionally Key Vault-linked |
| Reusable workflow | GitHub Actions | A whole workflow called with uses: at job level |
| Composite action | GitHub Actions | A sequence of steps bundled as one uses: step |
Azure Pipelines templates
parameters:
- name: buildConfiguration
type: string
default: Release
- name: runTests
type: boolean
default: true
steps:
- task: DotNetCoreCLI@2
displayName: Restore
inputs: { command: restore }
- task: DotNetCoreCLI@2
displayName: Build
inputs:
command: build
arguments: '-c ${{ parameters.buildConfiguration }} --no-restore'
# Compile-time conditional: the step does not EXIST when false
- ${{ if eq(parameters.runTests, true) }}:
- task: DotNetCoreCLI@2
displayName: Test
inputs: { command: test, arguments: '--no-build' }
steps:
- template: templates/build-steps.yml
parameters:
buildConfiguration: Debug
runTests: false
# From ANOTHER repository:
resources:
repositories:
- repository: templates
type: git
name: Platform/pipeline-templates
ref: refs/tags/v2.1.0 # pin the version!
steps:
- template: build-steps.yml@templates
Caveat — extends is how you enforce, template is how you share
An extends template owns the whole pipeline and can restrict what the consuming pipeline is allowed to do (stepList parameters, allowed tasks). Combined with the required template check on an environment (see 3.2), it guarantees that every production deployment runs your mandated steps. A plain - template: insertion is opt-in and can simply be omitted.
GitHub reusable workflows and composite actions
on:
workflow_call:
inputs:
environment:
required: true
type: string
outputs:
artifact-name:
value: ${{ jobs.build.outputs.name }}
secrets:
AZURE_CLIENT_ID:
required: true
jobs:
build:
runs-on: ubuntu-latest
outputs:
name: ${{ steps.pack.outputs.name }}
steps:
- uses: actions/checkout@v4
- id: pack
run: echo "name=app-${{ github.run_number }}" >> "$GITHUB_OUTPUT"
jobs:
call-build:
# uses: at JOB level (not a step) — this is a reusable workflow
uses: ./.github/workflows/reusable-build.yml
with:
environment: production
secrets:
AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
# or: secrets: inherit
deploy:
needs: call-build
runs-on: ubuntu-latest
steps:
- run: echo "${{ needs.call-build.outputs.artifact-name }}"
Reusable workflow vs composite action
A reusable workflow is called at job level, brings its own jobs and runners, and can target environments. A composite action is called at step level and runs inside the caller's job on the caller's runner. "Share a set of steps used inside many jobs" → composite action. "Share an entire build-and-publish job across repos" → reusable workflow.
Variable groups and Key Vault
variables:
- group: 'Prod-KeyVault-Linked' # Library → Variable group → Link secrets from Key Vault
- name: region
value: uksouth
steps:
- script: ./deploy.sh
env:
DB_PASSWORD: $(SqlAdminPassword) # resolved from Key Vault at runtime
Alternatively fetch secrets inline with the AzureKeyVault@2 task — see Domain 4. Variable groups can be scoped to specific pipelines and require pipeline permissions before first use.
Checks and approvals with YAML environments
An environment is a named deployment target that provides: deployment history, resource traceability (Kubernetes/VM resources), and — critically — the place where approvals and checks attach. Full check list in 3.2.
- stage: Production
jobs:
- deployment: DeployProd
displayName: Deploy to production
environment:
name: production
resourceType: Kubernetes # or VirtualMachine
resourceName: prod-cluster
strategy:
runOnce:
preDeploy:
steps: [ { script: ./precheck.sh } ]
deploy:
steps: [ { script: ./deploy.sh } ]
routeTraffic:
steps: [ { script: ./route.sh } ]
postRouteTraffic:
steps: [ { script: ./smoke-test.sh } ]
on:
failure:
steps: [ { script: ./rollback.sh } ]
success:
steps: [ { script: ./cleanup.sh } ]
Lifecycle hooks are the rollback mechanism
preDeploy → deploy → routeTraffic → postRouteTraffic → on: success | failure. Putting smoke tests in postRouteTraffic and the undo in on: failure is how you express automatic rollback declaratively — see 3.4.
3.3 rapid recap
| The requirement says… | Answer |
|---|---|
| Source is in TFVC | Azure Pipelines (Actions cannot build TFVC) |
| Build must reach an on-prem SQL Server | Self-hosted agent in that network |
| Bursty Kubernetes-based runners that scale to zero | Actions Runner Controller (ARC) |
| Allow-list CI egress IPs on a firewall | GitHub larger runners (static IPs) or self-hosted |
| Connect a GitHub repo to Azure Pipelines securely | Azure Pipelines GitHub App |
| Nightly build even with no commits | schedules: + always: true |
| Don't build on docs-only commits | paths: exclude: / paths-ignore: |
| Manual run where the user picks the environment | workflow_dispatch inputs / pipeline parameters |
| Let a user add or remove a whole stage at queue time | Parameters + ${{ if }} (compile time) |
| Run three OS test jobs at once | strategy: matrix |
| Aggregate job must run even if tests failed | dependsOn + condition: always() / if: always() |
| Two stages should run at the same time | dependsOn: [] on the second stage |
| Share steps across many pipelines | YAML template / composite action |
| Share a whole build job across repos | Reusable workflow (workflow_call) |
| Force every prod pipeline to run a security scan | extends template + required template check |
| Approval before production in a YAML pipeline | Approval configured on the environment; use a deployment job |
| Pipeline needs secrets from Key Vault | Variable group linked to Key Vault or AzureKeyVault@2 |