Python for Actuarial Students · Course 4
Lesson 2

Automated Testing, Code Quality & CI

pytest, test-driven development, property-based tests, ruff and mypy, and a GitHub Actions pipeline that checks every change before it can reach a customer’s premium.

Lesson 2 · OverviewBy the end of this lesson you can…

  • Write unit tests with pytest, fixtures, and parametrization
  • Use test-driven development on premium calculations and their edge cases
  • State properties that must always hold, and let hypothesis hunt for counterexamples
  • Measure what your tests cover, and judge whether they would catch a bug
  • Lint and format with ruff; type-check with mypy
  • Run all of it automatically on every change with GitHub Actions
75 min
Concepts & worked examples
75 min
Lab: a test suite that bites

Companion notebook: lesson-02-testing-ci.ipynb — worked examples and self-checking exercises.

WhyIn a regulated line, “it runs” is not the standard

Lesson 1’s monolith ran for a year and overcharged 324 vehicles by $130,000, against the manual its own insurer filed.

  • A premium must follow the filed manual, to the cent, for every customer
  • A regulator or auditor may ask you to reproduce any premium, years later
  • Rating code changes every rate cycle, and each change can break an edge nobody looked at

Tests are executable specifications. “Drivers aged exactly 25 pay the 25–29 factor” written as a test is checked on every change, forever. Written in a memo, it is checked once.

Actuarial standards on models ask for evidence the model does what it is documented to do. A test suite, its coverage report, and a CI log are that evidence.

Part 1

Testing with pytest

Small, fast, specific.

pytestA test is a function that asserts

# tests/test_engine.py
def test_base_risk_pays_the_base_rates(engine, base_vehicle):
    assert engine.quote(base_vehicle).premiums == {
        "LIAB": 938.09, "COLL": 576.85, "COMP": 158.35}

def test_hand_calculated_quote(engine, base_vehicle):
    """Sports, driver 22, 16,000 miles, $45,000,
    $1,000 COLL deductible, $250 COMP."""
    v = base_vehicle.model_copy(update={...})
    q = engine.quote(v)
    assert q.premiums["LIAB"] == round(938.09 * 1.40 * 1.60 * 1.20, 2)
    assert q.total == 4881.88
# tests/conftest.py: fixtures, shared by every test file
@pytest.fixture(scope="session")
def manual() -> RatingManual:
    return load_manual()

@pytest.fixture
def base_vehicle() -> Vehicle:
    """Every factor 1.00: Sedan, 40, 12,000 miles,
    $30,000, $500 deductibles."""
    return Vehicle(driver_age=40, ...)
  • Files test_*.py, functions test_*: pytest finds them
  • A fixture is an argument pytest builds for you, once per test or per session
  • A failing assert shows both sides of the comparison

ParametrizeTest the boundaries, all of them

@pytest.mark.parametrize(("age", "factor"), [
    (16.0, 1.60), (24.99, 1.60), (25.0, 1.15),
    (29.99, 1.15), (30.0, 1.00), (64.99, 1.00),
    (65.0, 1.10), (74.99, 1.10), (75.0, 1.35)])
def test_age_factor(age, factor):
    assert age_factor(age) == factor

@pytest.mark.parametrize(("miles", "factor"), [
    (7_499, 0.85), (7_500, 1.00),
    (14_999, 1.00), (15_000, 1.20)])
def test_mileage_factor(miles, factor):
    assert mileage_factor(miles) == factor

The same 13 cases, run against the logic copied out of the monolith:

..F.........F                               [100%]
__________ test_age_factor[25.0-1.15] __________
>       assert age_factor(age) == factor
E       assert 1.6 == 1.15
E        +  where 1.6 = age_factor(25.0)
________ test_mileage_factor[15000-1.2] ________
>       assert mileage_factor(miles) == factor
E       assert 1.0 == 1.2
2 failed, 11 passed in 0.12s

Bugs live at the edges. Test the last value in a band and the first value of the next, for every band.

Test-driven developmentRed, green, refactor

Redwrite a test for the rule; watch it fail
→
Greenthe least code that passes
→
Refactortidy, tests stay green

Writing the test first forces the question what exactly should happen? before any code exists: what is earned premium for an exposure of −0.25 years?

# red: the rule, as a test
@pytest.mark.parametrize("exposure", [-0.25, 1.5, float("nan")])
def test_impossible_exposure_is_refused(engine, base_vehicle,
                                        exposure):
    with pytest.raises(ValueError, match="exposure"):
        engine.quote(base_vehicle).earned(exposure)

# green: the code
def earned(self, exposure: float) -> float:
    if not 0 <= exposure <= 1:     # also False for NaN
        raise ValueError(f"exposure must be between 0 "
                         f"and 1 year, got {exposure}")
    return money(self.total * exposure)

Edge casesThe inputs that break pricing code

CaseWhat went wrong without a testTest
Band boundaries25.0 and 15,000 priced in the wrong bandparametrized boundaries
Negative or > 1 exposurenegative earned premium, silentlypytest.raises(ValueError)
NaN exposureNaN premium flows into totals and makes them NaNNaN is refused too
Empty bookthe monolith’s summary divides by zerosummarize(empty) returns a row of 0 and NaN
Half centsround(836.645, 2) == 836.64money(836.645) == 836.65
Unknown algorithm namecrash mid-run, or worse, a silent defaultfactory raises, listing valid names
Deductible without covera COLL deductible on a LIAB-only policyValidationError, “if and only if”

Property-based testingRules that hold for every vehicle

from hypothesis import given, strategies as st

vehicles = st.builds(Vehicle,
    driver_age=st.floats(16, 110),
    vehicle_style=st.sampled_from(STYLES),
    vehicle_value=st.floats(500, 500_000),
    annual_mileage=st.floats(1, 150_000), ...)

@given(vehicles, st.floats(1, 50_000))
def test_more_miles_never_cost_less(v, extra):
    further = v.model_copy(update={
        "annual_mileage": min(v.annual_mileage + extra, 150_000)})
    assert ENGINE.quote(further).total >= ENGINE.quote(v).total

@given(vehicles)
def test_higher_deductible_never_costs_more(v): ...
@given(vehicles)
def test_liability_ignores_vehicle_value(v): ...
  • hypothesis generates 100 vehicles per property (more on request), aiming at extremes
  • A failure is shrunk to the simplest input that still fails
  • Properties encode actuarial common sense: monotone in risk, deductibles reduce premium, a factor applies only where the manual says

In CI, add @settings(derandomize=True): the same examples every run, so a test can’t fail on Tuesday with no code change.

How good are the tests?Coverage, and mutants

Name                     Stmts  Miss Branch BrPart  Cover
---------------------------------------------------------
src/ratingkit/algorithms.py 35     0      0      0   100%
src/ratingkit/book.py       33     0      6      0   100%
src/ratingkit/cli.py        27     2      6      3    85%
src/ratingkit/engine.py     21     0      4      0   100%
src/ratingkit/factory.py    18     0      2      0   100%
src/ratingkit/manual.py     29     0      8      0   100%
src/ratingkit/money.py       3     0      0      0   100%
src/ratingkit/schemas.py    26     0      8      0   100%
---------------------------------------------------------
TOTAL                      196     2     34      3    98%
Required test coverage of 90.0% reached.
57 passed in 0.88s

Coverage says which lines ran during the tests, not whether the tests would notice if those lines were wrong.

Mutation testing answers that: plant one bug at a time (“25 counts as young”, “round halves to even”, “forget the Sports factor”) and see if a test fails. A mutant that survives marks a missing test.

In the notebook, a single happy-path test catches 0 of 6 mutants; a boundary-focused suite catches all 6.

Part 2

Code Quality Tools

Let machines check what machines can.

ruffLinter and formatter in one, and fast

price_book.py:6:1: I001 Import block is un-sorted
price_book.py:8:8: F401 `math` imported but unused
price_book.py:27:5: SIM115 Use a context manager
                   for opening files
price_book.py:100:7: SIM115 Use a context manager
                     for opening files
price_book.py:108:7: UP031 Use format specifiers
                     instead of percent format
  ... (2 more UP031)
Found 7 errors.
[tool.ruff]
line-length = 100

[tool.ruff.lint]
select = ["E", "F", "W", "I", "B", "UP", "SIM", "RUF"]
  • With the package’s rule set: 8 findings (one line too long)
  • ruff check --fix fixes the safe ones; ruff format rewrites layout (29 lines of diff here)
  • Formatting is not a matter of taste once a tool decides: code reviews discuss logic, not spaces

mypyType errors before the code runs

price_book.py:63: error: Incompatible types in assignment
  (expression has type "float", variable has type "int")
price_book.py:64: error: Incompatible types in assignment
  ... (4 more)
price_book.py:94: error: No overload variant of
  "__setitem__" of "list" matches argument types
  "int", "float"
Found 7 errors in 1 file

The monolith sets coll = 0 (an int) and later assigns a float premium; its running totals start as ints too. Harmless here, but it is the kind of inconsistency that becomes a bug.

$ uv run mypy src tests
Success: no issues found in 16 source files

mypy did not find either boundary bug: age <= 25 is perfectly typed. Types check shapes; tests check behavior. You need both.

Part 3

Continuous Integration

Every change, checked, on a clean machine.

GitHub Actions.github/workflows/ci.yml

name: CI
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.12", "3.14"]
    steps:
      - uses: actions/checkout@v7
      - uses: astral-sh/setup-uv@v10
        with:
          python-version: ${{ matrix.python-version }}
      - run: uv sync --locked
      - run: uv run ruff check .
      - run: uv run ruff format --check .
      - run: uv run mypy src tests
      - run: uv run pytest --cov --cov-report=term-missing
  • Runs on every push to main and every pull request
  • A fresh virtual machine each time: nothing from your laptop leaks in
  • --locked fails if uv.lock is out of date: CI tests exactly what you locked
  • A matrix runs the suite on two Python versions in parallel
  • The workflow passes actionlint, a linter for workflow files

Branch protection makes the check required: a pull request can’t merge while it is red.

ProofThe same steps on a clean machine

$ docker run --rm -v "$PWD":/src:ro \
    ghcr.io/astral-sh/uv:python3.12-bookworm-slim sh -c '
    cp -r /src /w && cd /w && uv sync --locked &&
    uv run ruff check . && uv run ruff format --check . &&
    uv run mypy src tests && uv run pytest -q --cov'
All checks passed!
17 files already formatted
Success: no issues found in 16 source files
TOTAL        211      2     34      3    98%
Required test coverage of 90.0% reached.
57 passed in 3.67s

29 seconds from an empty container, including installing every dependency. That is roughly what each CI run costs, and why the suite must stay fast.

Branchmake the change
→
Pull requestCI runs; a colleague reviews
→
Mergeonly when green

Actuarial Lab · 75 minA test suite that bites

  1. Add tests/ and fixtures (manual, engine, base vehicle) to your Lesson 1 package
  2. Parametrized tests for every age and mileage boundary; a hand-calculated quote
  3. TDD: exposure outside [0, 1], NaN, an empty book, half cents
  4. Three hypothesis properties: miles, deductibles, vehicle value
  5. A golden-master test against the monolith that explains every difference
  6. ruff and mypy clean; coverage above 90%
  7. A CI workflow; run its steps in a clean environment

Answer key

Monolith boundary tests2 failed, 11 passed
ruff on the monolith7 (8 with E501)
mypy --strict on the monolith7 errors
Reference suite57 tests, 98%
Notebook mutants caught6 of 6

RecapLesson 2 checklist

  • My tests pin down every band boundary
  • I write the test for a rule before the code
  • Impossible inputs raise clear errors, and I test that they do
  • Properties check what must hold for every vehicle
  • I know what coverage does and doesn’t prove
  • ruff and mypy run clean
  • CI runs every check on every pull request, on a clean machine
  • Nothing merges while CI is red
Up next
Lesson 3

Stochastic Simulation & Performance

A Monte Carlo ALM model for an annuity book: reproducible random numbers, profiling, vectorization, numba, and economic capital from a million scenarios.

← Course home