A CI pipeline without rigorous automated tests isn't continuous delivery — it's continuous hope. Every stage of testing acts as an automated gatekeeper that prevents bad code from reaching the next environment. If any gate fails, the pipeline stops and the developer is notified immediately.
100%
Rendering interactive visual diagram...
Total gate time: ~10 minutes. Any failure stops the pipeline and blocks the merge.
The Testing Pyramid: Balance Speed and Coverage
100%
Rendering interactive visual diagram...
Unit Tests: The Foundation
javascript
// Example: unit testing a pure function (no DB, no network)
// src/utils/pricing.test.ts
import { calculateDiscount } from './pricing';
describe('calculateDiscount', () => {
it('applies 10% for orders over $100', () => {
expect(calculateDiscount(150, 'REGULAR')).toBe(135);
});
it('applies 20% for premium users', () => {
expect(calculateDiscount(100, 'PREMIUM')).toBe(80);
});
it('returns full price for orders under $100', () => {
expect(calculateDiscount(50, 'REGULAR')).toBe(50);
});
});
yaml
# In CI
- name: Unit tests with coverage
run: |
npm run test:unit -- \
--coverage \
--coverageThreshold='{"global":{"lines":80,"functions":80,"branches":70}}'
# Pipeline fails if coverage drops below thresholds
Integration Tests: Real Dependencies, Ephemeral
Integration tests verify that components work together — but should use ephemeral test databases spun up for the CI run, not shared staging databases:
yaml
# GitHub Actions with service containers for integration tests
test-integration:
services:
postgres:
image: postgres:16-alpine
env:
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
POSTGRES_DB: testdb
options: >-
--health-cmd "pg_isready -U testuser -d testdb"
--health-interval 5s
--health-retries 10
redis:
image: redis:7-alpine
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-retries 5
steps:
- name: Run integration tests
run: npm run test:integration
env:
DATABASE_URL: postgres://testuser:testpass@localhost:5432/testdb
REDIS_URL: redis://localhost:6379
NODE_ENV: test
End-to-End Tests: Only on Staging
E2E tests are expensive — they spin up a browser and simulate a real user. Run them on staging after deploy, not in every PR:
yaml
# Only run after staging deploy, not on every PR
e2e-tests:
needs: deploy-staging
runs-on: ubuntu-latest
steps:
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
- name: Run E2E tests
run: npx playwright test
env:
BASE_URL: https://staging.yourapp.com
- name: Upload test report
uses: actions/upload-artifact@v4
if: always() # Upload even on failure
with:
name: playwright-report
path: playwright-report/
Flaky Tests: The Enemy of CI/CD
A flaky test is one that fails intermittently without any code change. Flaky tests destroy team trust in CI:
text
Day 1: Pipeline fails → Developer checks code → Nothing wrong → Re-runs pipeline → Passes
Day 2: Same. Developer ignores failures and force-merges.
Day 3: A REAL failure is ignored because everyone assumes it's flaky.
Day 4: Bug reaches production.
The solution: Treat flaky tests as broken tests.
yaml
# Detection: run tests 3 times and report intermittent failures
- name: Detect flaky tests
run: npx jest --runInBand --testRetries=3 --reporters=default,jest-flaky-reporter
continue-on-error: true # Don't fail the build on flakiness (just report)
# Quarantine: temporarily disable a known flaky test
# In your test file:
it.skip('flaky test name', () => { // ← Skip until fixed
// TODO: Fix flakiness — tracked in issue #456
Rules:
- If a test fails randomly → quarantine it immediately (skip, track in an issue)
- Fix and re-enable within the current sprint — don't let quarantine become permanent
- A CI pipeline must be deterministic: failure = code is broken, not "try again"
Code Quality Gates
Beyond functional correctness, enforce structural quality:
yaml
quality-gates:
steps:
# Lint: enforce code style and catch common errors
- name: ESLint
run: npx eslint src/ --max-warnings=0 # Zero warnings allowed
- name: TypeScript type check
run: npx tsc --noEmit # No type errors allowed
- name: Prettier format check
run: npx prettier --check src/ # Formatting must match standard
# Code coverage threshold
- name: Check coverage
run: |
COVERAGE=$(npm run test:unit -- --coverage --json 2>/dev/null |
python3 -c "import json,sys; d=json.load(sys.stdin);
print(d['coverageMap']['total']['lines']['pct'])")
echo "Coverage: $COVERAGE%"
python3 -c "exit(0 if float('$COVERAGE') >= 80 else 1)" \
|| (echo "❌ Coverage below 80%" && exit 1)
# Complexity check: prevent unmaintainable code
- name: Check cyclomatic complexity
run: npx plato -r -d complexity-report src/ && \
node -e "
const r=require('./complexity-report/report.json');
const maxComplexity=r.summary.average.maintainability;
if(maxComplexity < 70) process.exit(1);"
Shifting Security Left: DevSecOps Pipeline
Security scans integrated into every CI run — not a quarterly audit:
SAST: Static Application Security Testing
Analyzes source code for vulnerability patterns (SQL injection, XSS, hardcoded secrets) without running the code:
yaml
- name: CodeQL Analysis (SAST)
uses: github/codeql-action/init@v3
with:
languages: javascript, typescript
queries: security-and-quality # Extended security query set
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:javascript"
SCA: Software Composition Analysis
Your app is 80% third-party libraries. SCA scans them for known CVEs:
yaml
# Dependabot: automatic PRs when dependencies have CVEs
# .github/dependabot.yml
version: 2
updates:
- package-ecosystem: npm
directory: /
schedule: { interval: weekly }
labels: [dependencies, security]
open-pull-requests-limit: 10
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-patch"] # Ignore patch updates in auto-PRs
# Snyk: detailed CVE reports in PRs
- name: Snyk SCA scan
uses: snyk/actions/node@master
with:
args: --severity-threshold=high --fail-on=upgradable
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
# npm built-in audit
- name: npm audit
run: npm audit --audit-level=high # Fail on HIGH or CRITICAL
Secrets Detection
Prevent accidentally committed API keys, database passwords, private keys:
yaml
# TruffleHog: detect secrets in git history
- name: TruffleHog secrets scan
uses: trufflesecurity/trufflehog@main
with:
path: ./
base: ${{ github.event.repository.default_branch }}
head: HEAD
extra_args: --only-verified --json
# GitLeaks: fast and highly configurable
- name: Gitleaks
uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE }}
# Detect-secrets: run locally before commit
# Install: pip install detect-secrets
# Initialize baseline: detect-secrets scan > .secrets.baseline
# Check for new secrets: detect-secrets scan | detect-secrets audit -
Container Vulnerability Scanning
yaml
# Trivy: scan the built image
- name: Scan container image
uses: aquasecurity/trivy-action@master
with:
image-ref: ghcr.io/${{ github.repository }}:${{ github.sha }}
format: sarif # Integrates with GitHub Security tab
output: trivy-results.sarif
severity: CRITICAL,HIGH
exit-code: 1 # Fail pipeline on HIGH or CRITICAL
- name: Upload scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: trivy-results.sarif
# IaC scanning: check Dockerfiles and Kubernetes manifests
- name: Scan Dockerfiles and manifests
uses: aquasecurity/trivy-action@master
with:
scan-type: config
scan-ref: .
exit-code: 1
severity: CRITICAL,HIGH
Severity-Based Gate Policy
Avoid paralysing developer velocity by treating all vulnerabilities the same. Use a tiered approach:
| Severity | Policy | Action |
|---|---|---|
| CRITICAL | Block pipeline immediately | Fix now, no exceptions |
| HIGH | Block pipeline | Fix before merge (7-day SLA if compensating control documented) |
| MEDIUM | Warning only | Add to backlog, fix within 30 days |
| LOW | Informational | Optional — address in scheduled maintenance |
bash
# Apply tiered policy
trivy image \
--exit-code 1 \
--severity CRITICAL,HIGH \ # Block on these
--ignore-unfixed \ # Don't block on CVEs with no fix yet
my-api:latest
# Generate a report for MEDIUM/LOW (informational only)
trivy image \
--exit-code 0 \ # Don't fail
--severity MEDIUM,LOW \
--format json \
--output medium-low-report.json \
my-api:latest
Full Quality Gate Configuration: GitHub Actions
yaml
# .github/workflows/quality.yml
name: Quality Gates
on:
pull_request:
branches: [main]
jobs:
# ── Gate 1: Fast feedback (< 1 minute) ────────────────────────────
fast-checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: "20", cache: "npm" }
- run: npm ci
- run: npx eslint src/ --max-warnings=0
- run: npx tsc --noEmit
- run: npx prettier --check src/
# ── Gate 2: Tests + coverage (< 5 minutes) ─────────────────────────
test:
needs: fast-checks
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16-alpine
env: { POSTGRES_USER: test, POSTGRES_PASSWORD: test, POSTGRES_DB: test }
options: --health-cmd "pg_isready -U test" --health-interval 5s --health-retries 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: "20", cache: "npm" }
- run: npm ci
- run: npm run test:unit -- --coverage
- run: npm run test:integration
env: { DATABASE_URL: postgres://test:test@localhost:5432/test }
- uses: codecov/codecov-action@v4
with: { threshold: 80 }
# ── Gate 3: Security (< 5 minutes) ────────────────────────────────
security:
needs: fast-checks
runs-on: ubuntu-latest
permissions: { contents: read, security-events: write }
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm audit --audit-level=high
- uses: github/codeql-action/init@v3
with: { languages: javascript }
- uses: github/codeql-action/analyze@v3
- uses: gitleaks/gitleaks-action@v2
env: { GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} }
# ── Gate 4: Container scan (< 3 minutes) ──────────────────────────
container-scan:
needs: [test, security]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: docker/build-push-action@v5
with: { context: ., push: false, tags: myapp:scan, load: true }
- uses: aquasecurity/trivy-action@master
with:
image-ref: myapp:scan
severity: CRITICAL,HIGH
exit-code: 1
Summary
Automated quality gates are the foundation of safe CI/CD:
- Unit tests (fast, isolated, mocked) → Integration tests (real ephemeral DB) → E2E tests (staging only)
- Flaky tests must be quarantined immediately — determinism is non-negotiable
- SAST (CodeQL): source code patterns; SCA (Dependabot/Snyk): dependency CVEs; Secrets (TruffleHog/Gitleaks): committed credentials; Container (Trivy): OS/runtime CVEs
- Use tiered severity: CRITICAL+HIGH block the pipeline; MEDIUM+LOW are warnings
- Structure gates for fast feedback: lint+typecheck first (~1min), then tests (~5min), then security (~5min), then container scan (~3min)
In the next lesson, you will learn deployment strategies — the techniques that let you release new versions with zero downtime and instant rollback capability.