Stop copying pipeline YAML between repos

ยท 1 min read

Ten services with ten slightly different build pipelines means ten places to fix every change. Azure Pipelines templates let you define the steps once and reuse them everywhere.

Every service starts with a copy of the last service's azure-pipelines.yml. A year later, each copy has drifted: one runs tests with coverage and one doesn't, one pins the .NET SDK version and one uses whatever is installed, one has the security scan that was added after an incident and the rest don't.

Templates let you define pipeline steps, jobs or stages once and reference them from every pipeline.

A step template

Put shared steps in their own file, with parameters for the parts that vary:

# templates/build-dotnet.yml
parameters:
  - name: project
    type: string
  - name: dotnetVersion
    type: string
    default: '10.0.x'
  - name: runTests
    type: boolean
    default: true

steps:
  - task: UseDotNet@2
    inputs:
      version: ${{ parameters.dotnetVersion }}

  - script: dotnet build ${{ parameters.project }} --configuration Release
    displayName: Build

  - ${{ if eq(parameters.runTests, true) }}:
    - script: dotnet test --configuration Release --no-build --collect:"XPlat Code Coverage"
      displayName: Test

  - script: dotnet publish ${{ parameters.project }} --configuration Release --no-build --output $(Build.ArtifactStagingDirectory)
    displayName: Publish

Use it in a pipeline

# azure-pipelines.yml
trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - template: templates/build-dotnet.yml
    parameters:
      project: src/Orders.Api/Orders.Api.csproj

The pipeline file is now a few lines long and only says what's specific to this service.

Share templates across repositories

Keep templates in a dedicated repository and reference it as a resource:

resources:
  repositories:
    - repository: templates
      type: git
      name: Platform/pipeline-templates
      ref: refs/tags/v3

steps:
  - template: build-dotnet.yml@templates
    parameters:
      project: src/Orders.Api/Orders.Api.csproj

Pinning ref to a tag means a change to the shared template doesn't break every pipeline at once. Teams move to v4 when they're ready.

Templates at every level

  • Step templates for a sequence of steps, like the build above
  • Job templates for a whole job, such as "run integration tests in a container"
  • Stage templates for a complete stage, such as "deploy to an environment with approvals and a slot swap"
  • Extends templates, where a pipeline extends a central template that controls its overall structure. This is how organizations enforce required steps, such as security scanning, that individual pipelines can't remove.

Template expressions vs runtime variables

${{ parameters.x }} is resolved when the pipeline is compiled, before anything runs. That's why it can include or exclude whole steps with ${{ if }}. $(variable) is resolved at runtime. Mixing them up is the most common source of confusing template behavior.

Takeaway

Move shared build and deploy steps into parameterized templates, keep them in their own repository, pin versions, and let each service's pipeline describe only what makes it different.