Design and implement a testing strategy for pipelines

Quality and release gates

A gate is an automated condition that must hold before a deployment proceeds (or is considered complete). An approval is the human equivalent. Together with checks they are how governance is expressed in a pipeline.

Azure Pipelines — environment checks

In YAML pipelines, governance lives on the environment, not in the YAML file. Configure it under Pipelines → Environments → [name] → Approvals and checks.

Classic release pipeline gates

Classic (non-YAML) release pipelines have pre-deployment and post-deployment gates configured on each stage: Azure Monitor alerts, Invoke REST API, Invoke Azure Function, Query Work Items, and security/compliance assessment. They evaluate on an interval with a timeout, and all gates must pass simultaneously in one sampling window.

GitHub Actions — environment protection rules

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment:
      name: production          # protection rules configured in repo Settings → Environments
      url: https://myapp.example.com
    steps:
      - run: ./deploy.sh

A comprehensive testing strategy

The test pyramid

Unit tests

Characteristics the exam expects you to recognise: isolated (dependencies mocked), fast (milliseconds), deterministic (same input → same result, always), independent (no ordering dependency), and with no network, file system or database access.

- task: DotNetCoreCLI@2
  displayName: Run unit tests
  inputs:
    command: test
    projects: '**/*Tests/*.csproj'
    arguments: '--configuration Release --collect:"XPlat Code Coverage"'
    publishTestResults: true        # auto-publishes the .trx

Integration tests and service containers

Service containers give a job real backing services (database, cache, queue) without any external infrastructure — the standard answer to "test against a real database without a shared test server".

resources:
  containers:
    - container: pg
      image: postgres:16
      env:
        POSTGRES_DB: testdb
        POSTGRES_USER: testuser
        POSTGRES_PASSWORD: testpass
      ports:
        - 5432:5432

jobs:
  - job: IntegrationTests
    pool:
      vmImage: ubuntu-latest
    services:
      postgres: pg
    steps:
      - script: dotnet test --filter Category=Integration
        env:
          DB_CONNECTION: >-
            Host=localhost;Database=testdb;
            Username=testuser;Password=testpass

jobs:
  integration:
    runs-on: ubuntu-latest
    services:
      redis:
        image: redis:7
        ports: ['6379:6379']
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:integration
        env:
          REDIS_URL: redis://localhost:6379

Load and performance testing

Azure Load Testing is the managed service: it runs Apache JMeter or Locust scripts (or simple URL-based tests) at scale from Azure, correlates results with server-side Azure Monitor metrics, and can fail the pipeline on threshold breach.

- task: AzureLoadTest@1
  inputs:
    azureSubscription: 'MyServiceConnection'
    loadTestConfigFile: 'loadtest/config.yaml'
    resourceGroup: 'my-rg'
    loadTestResource: 'my-load-test'

# loadtest/config.yaml — thresholds that fail the run
# failureCriteria:
#   - avg(response_time_ms) > 2000
#   - percentage(error) > 5
#   - p95(response_time_ms) > 3000

Implementing tests in a pipeline

Configuring test tasks

- task: PublishTestResults@2
  # Without this condition the step is SKIPPED when tests fail —
  # which is exactly when you need the report most.
  condition: succeededOrFailed()
  inputs:
    testResultsFormat: 'JUnit'          # JUnit | NUnit | VSTest | XUnit | CTest
    testResultsFiles: '**/test-results/*.xml'
    mergeTestResults: true
    failTaskOnFailedTests: true
    testRunTitle: 'Unit tests · $(Build.BuildNumber)'

Test agents and multi-configuration testing

jobs:
  - job: Test
    strategy:
      matrix:
        linux:
          imageName: 'ubuntu-latest'
        windows:
          imageName: 'windows-latest'
        mac:
          imageName: 'macOS-latest'
      maxParallel: 3
    pool:
      vmImage: $(imageName)
    steps:
      - script: dotnet test

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false     # see ALL failures, not just the first
      max-parallel: 4
      matrix:
        os: [ubuntu-latest, windows-latest]
        node: ['20', '22']
        exclude:
          - os: windows-latest
            node: '20'
    steps:
      - uses: actions/setup-node@v4
        with: { node-version: '${{ matrix.node }}' }
      - run: npm ci && npm test

• Slicing splits one large suite across N agents: Azure Pipelines uses parallel strategy with $(System.JobPositionInPhase) / $(System.TotalJobsInPhase); the VSTest task can distribute automatically.
• Test Impact Analysis (VSTest task, runOnlyImpactedTests: true) runs only the tests affected by the code changed since the last run — the answer to "reduce test time without losing relevant coverage".
• Agent demands (demands: HasSpecialSoftware) route a test job to agents with the right capabilities; self-hosted agents are required for private-network or licensed-tool testing.

Code coverage analysis

- task: DotNetCoreCLI@2
  inputs:
    command: test
    arguments: '--collect:"XPlat Code Coverage" --results-directory $(Agent.TempDirectory)'

- task: PublishCodeCoverageResults@2
  inputs:
    summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'
    # v2 infers the format; v1 required codeCoverageTool: 'Cobertura'

Enforcing a threshold

dotnet test \
  /p:CollectCoverage=true \
  /p:CoverletOutputFormat=cobertura \
  /p:Threshold=80 \
  /p:ThresholdType=line,branch \
  /p:ThresholdStat=minimum

- task: SonarQubePrepare@6
  inputs: { SonarQube: 'SonarConn', scannerMode: 'MSBuild', projectKey: 'my-app' }

- script: dotnet build && dotnet test --collect:"XPlat Code Coverage"

- task: SonarQubeAnalyze@6
- task: SonarQubePublish@6         # fails the build if the quality gate fails
  inputs: { pollingTimeoutSec: '300' }

3.2 rapid recap