A CI/CD pipeline is an automated assembly line that takes your source code from a Git commit and safely delivers it to production without manual intervention. Every pipeline consists of a series of stages — each stage is a quality gate that must pass before the next begins.
Rendering interactive visual diagram...
Stage 1: Build
The build stage compiles your source code and produces an artifact — usually a Docker image for modern applications.
Goals: Verify the code compiles cleanly and produce a deterministic, immutable artifact.
# GitHub Actions example
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
target: runner # Production stage only
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha # Reuse layers from previous builds
cache-to: type=gha,mode=max
Key principle: Build once, deploy the exact same artifact everywhere. The Docker image tagged with the Git SHA that passes tests in staging is the same image deployed to production — no rebuilds.
Stage 2: Test
The test stage is the core of CI. It validates code correctness at multiple levels of the Testing Pyramid:
Rendering interactive visual diagram...
test:
runs-on: ubuntu-latest
services:
# Spin up a real Postgres for integration tests
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
POSTGRES_DB: testdb
options: >-
--health-cmd "pg_isready -U testuser"
--health-interval 5s
--health-retries 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: "20" }
- name: Install dependencies
run: npm ci
- name: Unit tests
run: npm run test:unit -- --coverage
- name: Integration tests
run: npm run test:integration
env:
DATABASE_URL: postgres://testuser:testpass@localhost:5432/testdb
- name: Upload coverage report
uses: codecov/codecov-action@v4
with:
fail_ci_if_error: true
threshold: 80 # Fail if coverage drops below 80%
Flaky tests are broken tests: A test that fails randomly destroys developer trust. When discovered, immediately quarantine it (skip it temporarily), fix it, and re-enable. A CI pipeline must be completely deterministic — a failure must always mean the code is broken, never "try again."
Stage 3: Security Scanning (Shift Left)
Security scans run in CI, not in a quarterly audit. This is called "shifting left" — catching vulnerabilities early when they're cheap to fix.
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# SAST: scan source code for vulnerability patterns
- name: Run CodeQL (SAST)
uses: github/codeql-action/analyze@v3
with:
languages: javascript, typescript
# SCA: scan dependencies for known CVEs
- name: Audit npm dependencies
run: npm audit --audit-level=high # Fail on HIGH and CRITICAL CVEs
# Secrets detection: catch accidentally committed API keys
- name: Scan for secrets
uses: trufflesecurity/trufflehog@main
with:
path: ./
base: ${{ github.event.repository.default_branch }}
head: HEAD
extra_args: --only-verified
# Container scanning: scan the built image
- name: Scan Docker image
uses: aquasecurity/trivy-action@master
with:
image-ref: ghcr.io/${{ github.repository }}:${{ github.sha }}
severity: CRITICAL,HIGH
exit-code: 1 # Break the pipeline on HIGH/CRITICAL CVEs
# IaC scanning: check Kubernetes manifests, Dockerfiles
- name: Scan IaC
uses: aquasecurity/trivy-action@master
with:
scan-type: config
scan-ref: .
exit-code: 1
Stage 4: Package and Publish Artifact
Once the image passes all tests and scans, it's tagged and published to the container registry with multiple tags:
# Tagging strategy:
GIT_SHA=$(git rev-parse --short HEAD)
VERSION=$(git describe --tags --always) # e.g., v1.2.3
# Tag with both SHA (immutable, for deployment systems)
# and version (human-readable, for releases)
docker tag myapp:build ghcr.io/yourorg/myapp:${GIT_SHA}
docker tag myapp:build ghcr.io/yourorg/myapp:${VERSION}
docker tag myapp:build ghcr.io/yourorg/myapp:latest
docker push ghcr.io/yourorg/myapp:${GIT_SHA}
docker push ghcr.io/yourorg/myapp:${VERSION}
docker push ghcr.io/yourorg/myapp:latest
Stage 5: Deploy to Staging
Staging is a production-like environment that real traffic never reaches. It's where integration and E2E tests run against the complete stack:
deploy-staging:
needs: [build, test, security] # All gates must pass first
runs-on: ubuntu-latest
environment: staging
steps:
- name: Deploy to staging server
uses: appleboy/ssh-action@master
with:
host: ${{ secrets.STAGING_HOST }}
username: deploy
key: ${{ secrets.STAGING_SSH_KEY }}
script: |
IMAGE=ghcr.io/${{ github.repository }}:${{ github.sha }}
docker pull $IMAGE
IMAGE=$IMAGE docker compose \
--env-file /home/deploy/.env.staging \
-f docker-compose.yml -f docker-compose.staging.yml \
up -d --no-deps api
# Smoke test: verify the deployment is healthy
sleep 10
curl -f http://localhost:3000/api/health || exit 1
- name: Run E2E tests against staging
run: npm run test:e2e
env:
BASE_URL: https://staging.yourapp.com
Stage 6: Manual Approval Gate
For regulated environments or high-risk deployments, add a human approval step between staging and production:
# GitHub Actions: environments with required reviewers
deploy-production:
needs: deploy-staging
runs-on: ubuntu-latest
environment:
name: production # ← Configure "Required reviewers" in GitHub repo settings
url: https://yourapp.com
steps:
- name: Deploy to production
# ... only runs after a human approves in the GitHub UI
This triggers a GitHub UI notification where designated reviewers click "Approve" before the production deployment runs — giving you an audit trail and human verification without slowing down automation.
Stage 7: Production Deployment
The final stage deploys to production with automated health verification and rollback capability:
deploy-production:
steps:
- name: Deploy
uses: appleboy/ssh-action@master
with:
script: |
set -e
IMAGE=ghcr.io/${{ github.repository }}:${{ github.sha }}
# Pull new image while old version is still serving traffic
docker pull $IMAGE
# Restart only the api service (not db/cache — no downtime there)
IMAGE=$IMAGE docker compose \
--env-file /home/deploy/.env.production \
-f docker-compose.yml -f docker-compose.prod.yml \
up -d --no-deps api
# Verify production is healthy
sleep 15
curl -f https://api.yourapp.com/health || {
echo "❌ Deployment failed health check — rolling back"
# Rollback: redeploy previous image tag
PREV_IMAGE=ghcr.io/${{ github.repository }}:${{ env.PREVIOUS_SHA }}
IMAGE=$PREV_IMAGE docker compose \
--env-file /home/deploy/.env.production \
-f docker-compose.yml -f docker-compose.prod.yml \
up -d --no-deps api
exit 1
}
echo "✅ Production deployment successful"
- name: Notify Slack
if: always()
uses: slackapi/slack-github-action@v1.26.0
with:
payload: |
{
"text": "${{ job.status == 'success' && '✅' || '❌' }} Production deploy: ${{ github.sha }}",
"channel": "#deployments"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}
Continuous Integration vs Continuous Delivery vs Continuous Deployment
| Term | Meaning | Human intervention |
|---|
| CI (Continuous Integration) | Every commit triggers automated build + tests | None after push |
| CD (Continuous Delivery) | Every passing build is ready to deploy; deploy is triggered manually | 1-click deploy |
| CD (Continuous Deployment) | Every passing build automatically deploys to production | Zero — fully automatic |
Most mature companies practice Continuous Delivery — fully automated pipelines up to staging, with a single-click or auto deploy to production. Continuous Deployment (fully automated to prod) requires exceptional test coverage and monitoring.
DORA Metrics: Measuring Pipeline Effectiveness
The DevOps Research and Assessment (DORA) team at Google identified 4 metrics that distinguish high-performing engineering teams:
| Metric | Elite Performer | Low Performer |
|---|
| Deployment Frequency | Multiple times/day | Monthly or less |
| Lead Time for Changes | < 1 hour | 1–6 months |
| Mean Time to Recovery | < 1 hour | 1 week–1 month |
| Change Failure Rate | 0–15% | 46–60% |
A well-implemented CI/CD pipeline directly improves all four metrics.
Summary
A production-grade CI/CD pipeline has 7 stages:
- Build: compile + build Docker image, push to registry with git SHA tag
- Test: unit → integration → coverage threshold (80%+)
- Scan: SAST (CodeQL), SCA (npm audit), secrets (TruffleHog), container (Trivy), IaC (Trivy config)
- Package: multi-tag with SHA + SemVer + latest
- Deploy to Staging: full stack deploy + smoke test + E2E test
- Manual Gate (optional): required reviewer approval in GitHub
- Production Deploy: pre-pull new image, restart service, verify health, auto-rollback on failure
In the next lesson, you will learn how artifacts — the immutable outputs of stage 4 — are managed, versioned, and secured throughout the pipeline.