Best Practices#

This guide provides recommendations for organising tests and getting the most out of core-tests and the broader testing toolchain.

Test Organization#

Folder Structure#

Organise tests by type for clarity and easier selective execution:

tests/
├── unit/              # Fast, isolated tests (no I/O, no network)
│   ├── tests_models.py
│   └── tests_utils.py
├── integration/       # Tests with real dependencies (DB, cache, etc.)
│   ├── check_database.py
│   └── check_api_client.py
└── functional/        # End-to-end tests
    └── check_workflows.py

Tests that connect to real external services (e.g. AWS, third-party APIs) should use a naming convention outside the built-in discovery patterns (e.g. check_aws_*.py) so they are never run unintentionally.

Naming Conventions#

core-tests recognizes three built-in patterns (all active by default):

  • tests_*.py — primary convention for this ecosystem

  • test_*.py — standard Python/pytest convention

  • *_test.py — alternative convention

Pick one pattern and apply it consistently across the project.

Coverage Best Practices#

Set Coverage Thresholds#

Configure minimum coverage requirements in .coveragerc:

[run]
source = your_package
omit =
    */tests/*
    */migrations/*

[report]
fail_under = 80
show_missing = True

Focus on Meaningful Coverage#

Aim for high coverage on:

  • Critical business logic

  • Complex algorithms

  • Error handling paths

  • Edge cases

Avoid chasing 100% at the cost of meaningful tests, trivial getters and framework boilerplate can be excluded via # pragma: no cover.

Incremental Coverage#

Use coverage to track improvements over time:

# Generate baseline
python manager.py run-coverage

# Add tests for uncovered code, then re-run to see improvements

CI/CD Integration#

GitLab CI Example#

# .gitlab-ci.yml
stages:
  - test

unit-tests:
  stage: test
  image: python:3.12
  script:
    - pip install -e ".[dev]"
    - python manager.py run-tests --test-type unit
  parallel:
    matrix:
      - PYTHON_VERSION: ["3.9", "3.10", "3.11", "3.12"]

coverage:
  stage: test
  image: python:3.12
  script:
    - pip install -e ".[dev]"
    - python manager.py run-coverage
  coverage: '/TOTAL.*\s+(\d+%)$/'

GitHub Actions Example#

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: pip install -e ".[dev]"
      - name: Run tests
        run: python manager.py run-tests --test-type unit
      - name: Generate coverage
        run: python manager.py run-coverage

Using pytest directly#

When you need pytest-specific features (fixtures, parametrize, parallel execution, plugins), run pytest directly — install it separately or via core-dev-tools:

pip install pytest pytest-xdist pytest-cov

Parallel execution#

pytest -n auto                          # Use all available CPU cores
pytest tests/unit/ -n auto              # Unit tests only, parallel

Limit parallelism for rate-limited external services:

pytest tests/functional/ -n 2           # Avoid overwhelming external APIs

Coverage with pytest#

pytest -n auto --cov=your_package --cov-report=html

Test markers#

pytest -m "unit and not slow" -n auto

Define markers in pyproject.toml:

[tool.pytest.ini_options]
markers = [
    "unit: Unit tests",
    "integration: Integration tests",
    "slow: Slow-running tests",
]

Debugging failed tests#

pytest -v -s                                            # Verbose with print output
pytest -x                                               # Stop on first failure
pytest --maxfail=3                                      # Stop after 3 failures
pytest tests/unit/tests_foo.py::FooTestCase::test_bar   # Run one test

Continuous Improvement#

  1. Review coverage reports regularly — identify untested code paths.

  2. Keep unit tests fast — mock I/O and external services.

  3. Name real-service tests differently — prevent accidental execution.

  4. Monitor CI/CD times — optimise as the test suite grows.