A Monte Carlo asset-liability model for an annuity book: reproducible random numbers, profiling, vectorization, and compiled loops with numba, all the way to economic capital from a million scenarios.
cProfile and line_profiler before optimizingnumba and run scenarios in parallelCompanion notebook: lesson-03-simulation-performance.ipynb — worked examples and self-checking exercises.
| In-force immediate annuities | |
|---|---|
| Annuitants | 5,000, ages 60–100 (mean 72.6) |
| Payments | $69.4m a year, at each year end while alive |
| Reserve at 4% | $762.2m (10.98 years of payments) |
| Horizon | 60 years: the youngest reaches 120 |
Mortality: the Standard Ultimate Survival Model, μx = A + Bcx (A = 0.00022, B = 2.7×10−6, c = 1.124). First check: ä65 at 5% = 13.5498, the textbook value.
What can go wrong, one draw per scenario:
Economic capital: what covers the liabilities’ cost in all but the worst 1 scenario in 200, above the average cost.
# rates: exact annual Vasicek steps
e = math.exp(-a)
sd = sigma * math.sqrt((1 - e * e) / (2 * a))
r[:, t] = r[:, t - 1] * e + b * (1 - e) + sd * z[:, t]
# discount factor for a payment at the end of year t
D = np.exp(-np.cumsum(r, axis=1))
C = np.cumsum(D, axis=1) # C[k] = D_1 + ... + D_k
# a life aged x survives t years with probability S_x(t);
# it receives K payments, K = max{t : S_x(t) > U}
pv = sum(payment[i] * C[K[i]] for i in lives)
Each scenario is a full 60-year projection of 5,000 lives: 300,000 life-years, and one present value at the end.
Scenario 0 of the test set: $890,175,321.12, above the $762m reserve because rates start at 3%, not 4%.
The trick for speed: C(k) turns “sum the discounted payments” into one lookup per life, and the survival curve depends only on age, so compute 41 curves, not 5,000.
A result you can’t reproduce can’t be audited.
rng = np.random.default_rng(2026) # a Generator
z = rng.standard_normal((n, 60)) # rates
zg = rng.standard_normal(n) # longevity
u = rng.random((n, 5_000)) # lives
# parallel work: independent child streams
children = np.random.SeedSequence(2026).spawn(8)
rngs = [np.random.default_rng(s) for s in children]
np.random.seed: any library call can move itSeedSequence.spawn gives streams that don’t overlap, so workers never share numbersIn parallel code, which thread runs which scenario changes from run to run. The model on the next slides seeds each scenario from seed + s, so the answer doesn’t depend on the thread count: 2 threads and 6 threads give identical results.
Measure, then fix the part that matters.
992244 function calls in 0.586 seconds
ncalls tottime cumtime function
198281 0.373 0.409 alm.py:83(legacy_q)
1 0.136 0.586 alm.py:89(legacy_pv)
396673 0.035 0.035 {built-in method math.exp}
198403 0.023 0.023 {method 'random' ...}
198333 0.018 0.018 {built-in method math.log}
Hits % Time Line Contents
38993 7.2 df *= math.exp(-rates[t - 1])
38993 78.8 if rnd.random() < legacy_q(age + t - 1,
t, g, ...):
36993 7.3 pv += pay * df
The legacy model recomputes qx from the Makeham formula 198,281 times for two scenarios, the same 60 ages over and over. 79% of the time is on one line.
Guessing where the time goes is usually wrong. cProfile says which function; line_profiler says which line.
| Version | Idea | vs legacy |
|---|---|---|
| loops | precomputed survival tables | ×9 |
| NumPy | all lives at once, per scenario | ×208 |
| numba, 1 core | compiled loops | ×2,547 |
| numba, 6 cores | prange over scenarios | ×9,723 |
On the same random numbers, the versions agree to $0.000005 on present values near $900m. Always test that a fast version gives the slow version’s answer.
uniq, idx = np.unique(ages, return_inverse=True) # 41 ages
for lo in range(0, n, chunk): # 100 scenarios at a time
S = np.stack([survival(q, uniq, g)[:, 1:]
for g in g_chunk]) # (100, 41, 60)
K = (S[:, idx, :] > u[lo:hi, :, None]).sum(axis=2)
pv[lo:hi] = (payments *
np.take_along_axis(C[lo:hi], K, axis=1)).sum(axis=1)
Broadcasting compares every life with every year in one step: 100 × 5,000 × 60 booleans per chunk.
Vectorization spends memory to save time. One chunk of 100 scenarios is 30 MB of booleans; a million scenarios at once would be 300 GB. Chunk, or move to compiled loops that need almost no memory.
import numba as nb
@nb.njit(parallel=True, cache=True)
def simulate(q, uniq, idx, payments, n, seed, r0, a, b,
sigma, g_mean, g_sd, cf_expected):
pv = np.empty(n)
for s in nb.prange(n): # scenarios in parallel
np.random.seed(seed + s) # one seed per scenario
g = g_mean + g_sd * np.random.standard_normal()
S = _survival_nb(q, uniq, g) # 41 curves
... # rates, C(k)
total = 0.0
for i in range(len(idx)): # plain loops: fine here
uu = np.random.random()
k = 0
while k < T and S[idx[i], k + 1] > uu:
k += 1
total += payments[i] * C[k]
pv[s] = total
return pv
@njit compiles the function to machine code on first call (1.6 s here); cache=True keeps it for next timeprange splits scenarios across cores; each scenario writes only its own pv[s]Order of attack: profile → better algorithm (precompute) → NumPy → numba. Most code stops at step 2 or 3.
What the speed buys.
Cost (matched) = bond price + PV(actual) − PV(expected)
EC = 99.5th percentile of cost − mean cost
| Scenarios | Run time | EC, cash | EC, matched |
|---|---|---|---|
| 10,000 | 1.5 s | $319.5m | $90.6m |
| 100,000 | 3.1 s | $320.3m | $93.6m |
| 1,000,000 | 30.4 s | $325.7m | $94.7m |
Best-estimate cost $859.6m. Ten batches of 100,000 give cash EC with a standard deviation of $2.5m: at 100,000 scenarios the 99.5th percentile rests on the worst 500.
| Only this risk switched on | EC, cash |
|---|---|
| Interest rates | $301.6m |
| Longevity trend | $76.2m |
| Who dies when (5,000 lives) | $13.9m |
Matching the cash flows removes most of the interest-rate risk and cuts capital from $326m to $95m. What’s left is longevity, which no bond portfolio hedges. Capital estimates need their own error bars.
| ä65 at 5% | 13.5498 |
| Reserve at 4% | $762,202,760 |
| Hot spot | legacy_q |
| EC, 1m scenarios: cash | $325.7m |
| … matched | $94.7m |
| Batch s.d. of cash EC | $2.5m |
The Course 3 champion model behind an HTTP API: request and response schemas, validation, authentication, OpenAPI documentation, and what async does and doesn’t buy.