liminfoGitHub Actions CI/CD Pipeline Setup
Build a complete CI/CD pipeline step by step with GitHub Actions: code quality checks, automated testing, Docker image builds, and staging/production deployments
Problem
Required Tools
Built-in CI/CD platform in GitHub. Define workflows in YAML that run automatically on events like push, PR, and tags.
Packages applications as container images. Provides consistent build environments and deployment units.
Container registry provided by GitHub. Integrates naturally with GitHub Actions.
Configure environment-specific secrets and protection rules (approval required, etc.) for staging/production.
Solution Steps
CI workflow: automated lint and testing
Automatically run code quality checks (ESLint, Prettier) and tests on PR creation/update. Test across multiple Node.js versions using a matrix strategy, and reduce dependency installation time with caching. Block PR merges when tests fail to ensure code quality.
# .github/workflows/ci.yml
name: CI
on:
pull_request:
branches: [main, develop]
push:
branches: [main]
# Cancel previous runs when new push arrives on same PR
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
name: Lint & Format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- name: ESLint
run: npx eslint . --max-warnings 0
- name: Prettier Check
run: npx prettier --check .
- name: TypeScript Type Check
run: npx tsc --noEmit
test:
name: Test (Node ${{ matrix.node-version }})
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18, 20, 22]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm test -- --coverage
- name: Upload coverage
if: matrix.node-version == 20
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage/Docker image build and registry push
Build Docker images and push to GitHub Container Registry when merged to main. Minimize image size with multi-stage builds and reduce build time with Docker layer caching. Use both git SHA and latest as image tags.
# .github/workflows/build.yml
name: Build & Push
on:
push:
branches: [main]
tags: ['v*']
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build:
name: Build Docker Image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=sha,prefix=
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=maxAutomated staging deployment
Automatically deploy to the staging environment when merged to main. Use GitHub Environments to separately manage environment-specific secrets (server access, environment variables). SSH into the server to pull the new image and replace the container.
# .github/workflows/deploy-staging.yml
name: Deploy to Staging
on:
workflow_run:
workflows: ["Build & Push"]
types: [completed]
branches: [main]
jobs:
deploy:
name: Deploy to Staging
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
environment: staging
steps:
- uses: actions/checkout@v4
- name: Deploy via SSH
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.STAGING_HOST }}
username: ${{ secrets.STAGING_USER }}
key: ${{ secrets.STAGING_SSH_KEY }}
script: |
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker pull ghcr.io/${{ github.repository }}:main
cd /opt/myapp
docker compose pull
docker compose up -d --remove-orphans
docker image prune -f
- name: Health Check
run: |
sleep 10
STATUS=$(curl -s -o /dev/null -w "%{http_code}" https://staging.example.com/health)
if [ "$STATUS" != "200" ]; then
echo "Health check failed: $STATUS"
exit 1
fi
echo "Staging deployment successful"
- name: Notify Slack
if: always()
uses: 8398a7/action-slack@v3
with:
status: ${{ job.status }}
text: "Staging deploy: ${{ job.status }}"
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}Production deployment (tag-based + approval required)
Production deployment is triggered only on semantic version tag creation (v1.0.0) and requires designated team member approval. Minimize downtime with blue-green deployment strategy and implement automatic rollback on failure.
# .github/workflows/deploy-production.yml
name: Deploy to Production
on:
push:
tags: ['v*.*.*']
jobs:
deploy:
name: Deploy to Production
runs-on: ubuntu-latest
environment:
name: production
url: https://example.com
# Requires "Required reviewers" in GitHub Environment settings
steps:
- uses: actions/checkout@v4
- name: Extract version
id: version
run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
- name: Deploy to Production
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.PROD_HOST }}
username: ${{ secrets.PROD_USER }}
key: ${{ secrets.PROD_SSH_KEY }}
script: |
set -e
VERSION=${{ steps.version.outputs.VERSION }}
IMAGE="ghcr.io/${{ github.repository }}:$VERSION"
echo "Deploying $IMAGE to production..."
# Backup current version (for rollback)
docker tag $(docker inspect --format='{{.Image}}' myapp-prod) ghcr.io/${{ github.repository }}:rollback 2>/dev/null || true
docker pull $IMAGE
cd /opt/myapp-prod
IMAGE_TAG=$VERSION docker compose up -d --remove-orphans
- name: Health Check
run: |
for i in {1..6}; do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" https://example.com/health)
if [ "$STATUS" = "200" ]; then
echo "Production is healthy"
exit 0
fi
sleep 10
done
exit 1
- name: Rollback on Failure
if: failure()
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.PROD_HOST }}
username: ${{ secrets.PROD_USER }}
key: ${{ secrets.PROD_SSH_KEY }}
script: |
echo "Rolling back..."
cd /opt/myapp-prod
IMAGE_TAG=rollback docker compose up -d
- name: Create GitHub Release
if: success()
uses: softprops/action-gh-release@v1
with:
generate_release_notes: trueSecret management and environment variables
Safely manage environment-specific sensitive information using GitHub Environments and Secrets. Repository secrets are accessible from all workflows, while environment secrets are only accessible from that specific environment. Generate .env files dynamically from secrets at build time.
# === GitHub Secrets Structure ===
# Repository Secrets (shared):
# - SLACK_WEBHOOK
# - DOCKER_USERNAME
#
# Environment: staging
# Secrets:
# - STAGING_HOST
# - STAGING_USER
# - STAGING_SSH_KEY
# - DATABASE_URL (staging DB)
#
# Environment: production
# Secrets:
# - PROD_HOST
# - PROD_USER
# - PROD_SSH_KEY
# - DATABASE_URL (production DB)
# Protection Rules:
# - Required reviewers: 2
# - Wait timer: 5 minutes
# --- Dynamic .env file creation in workflows ---
- name: Create .env file
run: |
cat << EOF > .env
NODE_ENV=production
DATABASE_URL=${{ secrets.DATABASE_URL }}
REDIS_URL=${{ secrets.REDIS_URL }}
JWT_SECRET=${{ secrets.JWT_SECRET }}
EOF
# --- Reusable Workflow to reduce duplication ---
# .github/workflows/reusable-deploy.yml
name: Reusable Deploy
on:
workflow_call:
inputs:
environment:
required: true
type: string
image-tag:
required: true
type: string
secrets:
SSH_HOST:
required: true
SSH_USER:
required: true
SSH_KEY:
required: true
jobs:
deploy:
runs-on: ubuntu-latest
environment: ${{ inputs.environment }}
steps:
- name: Deploy
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.SSH_HOST }}
username: ${{ secrets.SSH_USER }}
key: ${{ secrets.SSH_KEY }}
script: |
docker pull ghcr.io/myapp:${{ inputs.image-tag }}
docker compose up -dWorkflow optimization and monitoring
Optimize CI/CD pipeline speed with build caching, parallel execution, and conditional execution. Monitor workflow execution time and success/failure rates to improve bottlenecks.
# === Caching Strategy ===
# npm dependency caching (built into setup-node)
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm' # Auto-caching based on package-lock.json
# Docker layer caching (GitHub Actions Cache)
- uses: docker/build-push-action@v5
with:
cache-from: type=gha
cache-to: type=gha,mode=max
# === Conditional execution based on changed files ===
- uses: dorny/paths-filter@v3
id: changes
with:
filters: |
backend:
- 'src/**'
- 'package.json'
frontend:
- 'client/**'
# Run backend tests only when backend changes
- name: Run backend tests
if: steps.changes.outputs.backend == 'true'
run: npm test
# === Workflow execution time optimization ===
jobs:
lint:
runs-on: ubuntu-latest
test:
runs-on: ubuntu-latest
# Runs in parallel with lint
build:
needs: [lint, test] # Runs after both lint and test succeed
runs-on: ubuntu-latest
# === Status Badges ===
# Add to README.md:
# Core Code
Complete CI/CD pipeline: PR testing -> Docker build -> auto staging deploy -> tag-based production deploy. Environments separate staging from production.
# .github/workflows/ci-cd.yml
# Core: Complete PR -> Test -> Build -> Deploy pipeline
name: CI/CD Pipeline
on:
pull_request:
branches: [main]
push:
branches: [main]
tags: ['v*']
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20, cache: 'npm' }
- run: npm ci
- run: npm run lint
- run: npm test
build:
needs: test
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
permissions: { contents: read, packages: write }
steps:
- uses: actions/checkout@v4
- uses: docker/login-action@v3
with: { registry: ghcr.io, username: ${{ github.actor }}, password: ${{ secrets.GITHUB_TOKEN }} }
- uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy-staging:
needs: build
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: staging
steps:
- name: Deploy
run: echo "Deploy to staging..."
deploy-prod:
needs: build
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
environment: production
steps:
- name: Deploy
run: echo "Deploy to production..."Common Mistakes
Not minimizing GITHUB_TOKEN permissions creating security vulnerabilities
Use the permissions key to set minimum GITHUB_TOKEN permissions per job. Example: contents: read, packages: write. Apply the principle of least privilege by default.
Low cache hit rate due to improperly configured cache keys
Use package-lock.json for npm and Dockerfile hash for Docker as cache keys. Setting restore-keys in actions/cache for partial match fallback also improves hit rates.
No rollback mechanism on deployment failure causing extended outages
Always perform health checks after deployment and add an automatic rollback step on failure. Store the previous image tag and trigger rollback with if: failure() condition.