Y

CI/CD · Intermediate

Continuous integration with GitHub Actions

3 min read

Why it matters

CI/CD appeared in 98% of the DevOps job descriptions we read and in a third of software and data roles. Continuous integration means every change is built and tested automatically before it can merge. Teams with good CI merge small changes many times a day; teams without it merge big risky changes rarely.

The concept

  • A workflow is a YAML file in .github/workflows/. It runs on events (pull_request, push, a schedule).

  • A workflow has jobs; each job runs on a fresh runner machine and has steps, either shell commands (run) or reusable actions (uses).

  • A matrix runs the same job across versions or platforms.

  • Permissions on the automatic GITHUB_TOKEN should be minimal: read-only unless a job must write.

  • Concurrency groups cancel outdated runs when new commits arrive.

  • Fast feedback matters: cache dependencies, run the quickest checks first, and keep the whole pipeline under about ten minutes.

Worked example: CI for a Python package

For the retail-metrics package from the Python engineering shelf:

.github/workflows/ci.yml
name: CI

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip
      - run: pip install -e ".[dev]"
      - run: ruff check . && ruff format --check .
      - run: mypy src
      - run: pytest -q --junitxml=results.xml
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: test-results-${{ matrix.python-version }}
          path: results.xml

Check it before pushing:

lint-workflows.sh
pip install actionlint-py      # packages the actionlint binary
actionlint                     # run inside the repository

Then protect main in the repository settings: require the test jobs to pass and at least one review before merging. CI only helps if nobody can bypass it.

Each piece earns its place:

  • permissions: contents: read limits damage if a step or dependency is compromised.

  • concurrency with cancel-in-progress stops five queued runs when you push five quick fixes.

  • cache: pip speeds up installs; timeout-minutes stops hung jobs from burning minutes.

  • Uploading results only on failure keeps storage small and debugging easy.

Common mistakes

  • Secrets in workflow files or logs. Use repository or environment secrets, and never echo them.

  • Running untrusted pull request code with write tokens. Be careful with pull_request_target; it runs with the base repository's permissions.

  • Slow pipelines that people learn to ignore. Measure and cut the slowest steps.

  • Flaky tests left in place. Fix or quarantine them with a ticket; a red build that "always fails" trains people to ignore red builds.

Interview angle

"What happens in your CI pipeline?" Walk through triggers, jobs, the checks in order of speed, caching, permissions and branch protection. Add how you debug a failing job: read the logs, reproduce locally with the same versions, then fix.

Practice task

  1. Add this workflow to one of your repositories and make a pull request that fails lint on purpose.

  2. Add a job that runs only when files under docs/ change (paths filter).

  3. Measure the pipeline's duration and cut it by 30% with caching or parallel jobs.

Further reading