CI/CD Pipeline

This guide covers the continuous integration (CI) pipeline used in HERITRACE to ensure code quality and automate testing.

GitHub Actions Workflows

HERITRACE uses GitHub Actions for continuous integration and release automation. The project includes two main workflows:

1. Testing Workflow (.github/workflows/python-tests.yml): Triggered on every push to any branch and on pull requests to the main branch
2. Release Workflow (.github/workflows/release.yml): Triggered on pushes to main branch containing [release] in the commit message

Testing Workflow Steps

The CI pipeline performs the following steps:

1. Set up Python: It tests the project against multiple Python versions (3.10, 3.11, 3.12, and 3.13) to ensure compatibility.
2. Install Dependencies: It uses Poetry to install project dependencies, including development dependencies.
3. Start Test Databases: It runs the ./tests/start-test-databases.sh script to set up the required Virtuoso databases for the integration tests.
4. Run Tests: It executes the test suite using pytest and generates a coverage report.
5. Upload Coverage Report: The HTML coverage report is uploaded as a build artifact, allowing for inspection and analysis for all branches and Python versions.
6. Generate Coverage Badge: A coverage badge is generated for each branch using the results from the Python 3.10 test run. This provides a quick visual indicator of the test coverage status directly in the README.md file.

Coverage Badge Setup

The workflow uses the BYOB (Bring Your Own Badge) GitHub Action to generate coverage badges. To set this up:

1. Create a Personal Access Token (PAT) with access to the repository where badges will be stored.
2. Add the token as a secret in your GitHub repository:
   • Go to your GitHub repository.
   • Click on “Settings” > “Secrets and variables” > “Actions”.
   • Click “New repository secret”.
   • Name: GIST_PAT
   • Value: Your Personal Access Token
   • Click “Add secret”.

The badge will be generated for each branch and will include the branch name in the badge label.

Release Workflow

The release workflow automates the creation of releases and publication of Docker images. It is triggered when a commit to the main branch contains [release] in the commit message.

Release Workflow Steps

1. Semantic Release: Uses semantic-release to automatically create GitHub releases based on conventional commit messages
2. Docker Image Building: Builds multi-platform Docker images (AMD64 and ARM64)
3. Docker Hub Publishing: Publishes images to Docker Hub under the configured username
4. GitHub Container Registry: Publishes images to GitHub Container Registry (ghcr.io)

Required Secrets

To enable Docker publishing, the following secrets must be configured in your GitHub repository:

• DOCKER_HUB_USERNAME: Your Docker Hub username
• DOCKER_HUB_ACCESS_TOKEN: Docker Hub access token for authentication

The workflow automatically uses the GITHUB_TOKEN for GitHub Container Registry authentication.

Docker Image Tags

The workflow generates multiple tags for each image:

• latest: For the main branch
• Version tags: v1.0.0, 1.0.0, 1.0, 1 (based on semantic versioning)
• Branch tags: For feature branches
• PR tags: For pull requests

Published Images

Images are published to:

• Docker Hub: [username]/heritrace
• GitHub Container Registry: ghcr.io/[owner]/[repository]

Customizing the Workflows

You can customize the workflows by editing the respective files:

• .github/workflows/python-tests.yml for testing configuration
• .github/workflows/release.yml for release and Docker publishing