How to use
- Copy the installation command and review the source package before installing it in your own agent environment.
- Check the source document for required tools, credentials and supported hosts.
- Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/netresearch/concourse-ci-skill --skill concourse-ciBefore you start
- Installation does not include a model subscription, API allowance or permission to access your files.
- Repository stars and installation counts describe activity, not an independent quality or security test.
Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.
Concourse CI Pipeline Development
Expert guidance for writing, refactoring, and optimizing Concourse CI pipelines (v8.0+).
When to Use
- Creating or modifying Concourse pipelines
- Configuring resources (git, registry-image, custom types)
- Building container images with
oci-build-task - Troubleshooting resource check failures or build issues
- Migrating from legacy patterns (docker-image, duplicate jobs)
Quick Reference
| Task | Modern (Recommended) | Legacy (Avoid) |
|---|---|---|
| Building images | oci-build-task + registry-image | docker-image resource |
| Multi-env deploys | across step modifier | Duplicate jobs per env |
| Dynamic pipelines | set_pipeline + instanced pipelines | Manual pipeline duplication |
| Notification symbols | UTF-8 characters (e.g. \u2714 for checkmark, \u274c for X) | HTML entities (e.g. ✓, ✗) |
| Resource styling | Always use icon: property | No icon |
Core Concepts
Pipelines consist of resources (external versioned artifacts), jobs (sequences of steps), and optional groups (UI organization). All execution runs in containers.
Key step types: get, put, task, set_pipeline, in_parallel, do, try, load_var. Job hooks: on_success, on_failure, on_error, on_abort, ensure. Note: on_failure (non-zero exit) differs from on_error (infrastructure crash/OOM) -- handle both. Use fly execute to test tasks locally.
See references/core-concepts.md for step types table, lifecycle hooks, and fly CLI essentials.
Critical Gotchas
- Git tag detection after force-push -- Escape regex dots, enable
clean_tags: true, separate read/write resources, force recheck withfly -t T check-resource -r pipeline/resource. Seereferences/resources-guide.md. - registry_mirror format mismatch --
registry-imageexpects an object (host: mirror),docker-imageexpects a URL string. Provide separate formats inCONCOURSE_BASE_RESOURCE_TYPE_DEFAULTS. Seereferences/resources-guide.md. - GitLab Container Registry JWT auth -- The JWT endpoint lives on the GitLab host, not the registry host. Discover via
Www-Authenticateheader. Seereferences/resources-guide.md. - git push --mirror and default branch -- Target repo's default branch must exist upstream. If absent, the pre-receive hook rejects the push with "pre-receive hook declined". Set it before the first mirror push.
References
references/pipeline-syntax.md-- Complete YAML schema for pipelines, jobs, resourcesreferences/core-concepts.md-- Step types, lifecycle hooks, fly CLI essentialsreferences/resources-guide.md-- Git-resource, registry-image, docker-image migration, gotcha detailsreferences/best-practices.md-- Optimization, troubleshooting, notifications, deployment patternsreferences/resource-types-catalog.md-- Available resource types (Ansible, Terraform, etc.)
Examples
Working examples in examples/:
basic-pipeline.yml-- Build-test-deploy with versioningmodern-ci-cd.yml-- oci-build-task, across, buildlogretentionmulti-branch.yml-- Dynamic branch pipelines with set_pipelinedocker-build.yml-- Container image build and pushvars-template.yml-- Variable file organization
Validation
Use scripts/validate-pipeline.sh to check pipeline syntax before deployment.
