Kieran Duff — "The Letter": what we add to the PANDORA arsenal
Source: https://kieranduff.com/letters/ (Kieran Duff, XAQP, live systematic ~$3.7M on Darwinex). 12 letters studied on running a systematic book: sizing, risk, drawdown, correlation, anti-overfit validation. Goal: sort what really complements PANDORA (gauntlet, v_joint, multi-asset sleeve) vs what we already have.
TL;DR — what is worth adding (prioritized)
| # | Idea | PANDORA today | Action |
|---|---|---|---|
| 1 | Time-under-water + recovery time | missing from backtest/engine.py |
✅ pandora/backtest/extended_metrics.py |
| 2 | Risk-of-ruin at live size | missing | ✅ extended_metrics.py |
| 3 | Correlation budgeting (P&L, haircut) | sleeve = fixed weights 50/30/20 | ✅ strategies/multi_asset_sleeve.py (correlation_budgeted_allocation) |
| 4 | Regime-filter degradation test | never "degraded" | ✅ lab/regime_degradation.py |
| 5 | Decay kill-switch (3 components) | manual kill-switch only | ✅ pandora/risk/decay_kill_switch.py |
| 6 | Crisis replay battery | no battery | ✅ lab/crisis_replay.py |
| 7 | Live latency simulator | — | ✅ lab/latency_sim.py |
| 8 | Portfolio heat cap | daily limits only | ✅ pandora/risk/portfolio_heat.py |
| 9 | Parameter stability map | — | ✅ lab/param_stability.py |
| 10 | Regime bootstrap stress | — | ✅ lab/regime_bootstrap.py |
| 11 | Vol-targeting vs flat (comparator) | — | ✅ lab/vol_targeting.py |
| 12 | Capacity/slippage model | — | ✅ lab/capacity_model.py |
| 13 | Skylit meta-labeling (decision dataset) | — | ✅ pandora/skylit/decision_log.py |
| 14 | Trials ledger + honest DSR | — | ✅ pandora/research/trials_ledger.py |
| 15 | Signal-health monitoring of the live book | pre-live gauntlet only | ✅ pandora/pipeline/signal_health.py (wired into the brief) |
| — | Anti-flash-crash overlay | tested → rejected | lab/fastcrash_overlay.py (COMPLEXITY TAX) |
| — | AI = generator + red-team / Sharpe-Sortino-Calmar | already OK | Nothing to do |
🔑 What these tools REVEALED about v_joint (the real findings)
- Fragile to DELAY, not to bad calls (
regime_degradation+latency_sim) — flipping 20% of the days costs little (Sharpe 1.09→0.89), but 2 days of latency collapse it (→0.47). Expected live Sharpe ≈ 0.75 (−31%) at realistic delay. ⟹ the DIX-GEX execution window / latency is live risk #1. - Bear-market insurance, not anti-flash-crash (
crisis_replay) — crushes DD on slow breakdowns (COVID −1.1% vs B&H −33.4%) but EXPOSED on the 2024 yen V-crash. And the fast overlay does not save it (fastcrash_overlay= complexity tax: it confirms the crash too late). - Not curve-fit (
param_stability) — the 4 frozen thresholds sit on a plateau, and the frozen value is never the peak = conservative choices. Good sign for the freeze. - The displayed MaxDD is lucky (
regime_bootstrap) — −12.1% is at the 87th pctile; provision −26%. - Flat-0.6 stays optimal (
vol_targeting) — vol-targeting degrades Sharpe AND MaxDD (v_joint already de-risks through the regime). Capacity ≈ $500M (capacity_model).
The 12 letters in brief + the PANDORA link
Sizing & risk
- The maths of ruin —
P(ruin) ≈ ((1-edge)/(1+edge))^(risk units). Halving the size does roughly the same job on ruin as doubling the edge. → Every candidate must pass a probability-of-ruin check at the proposed live size. (add #2) - Too much risk (portfolio heat) — cap on summed open risk (3-5% equity); entries compete for a "heat" budget. Per-trade discipline ≠ portfolio discipline. (add #8)
Correlation (≈ our SPY/QQQ/IWM sleeve)
- Correlation: the silent killer — compute correlation on P&L (not returns), and on the residuals after a vol-regime factor: two strategies that lever the same way with vol look uncorrelated raw and become correlated in the residual.
- The correlation trap — budgeting: <0.3 independent; 0.3-0.6 proportional size haircut; >0.6 = a single strategy in two outfits. Window ≥6 months, monthly update, read by the sizing code. → Our sleeve hard-codes 50/30/20 weights while SPY/QQQ/IWM are highly correlated: we oversize. (add #3)
Distribution metrics
- Is your Sharpe lying? (Sortino/Calmar) — skew diagnostic:
Sortino > √2·Sharpe= positive skew (underestimated by Sharpe);<= negative skew (unpriced tail). Present the Sharpe+Sortino+Calmar triple. → We already have them; adding the skew diagnostic to the tearsheet would be free. - Time under water — longest time below the previous peak + MAR = CAGR/MaxDD. DD depth hides what the allocator lives through (a 5% that recovers in 3 weeks ≠ 5% over 10 months). (add #1)
Anti-overfit validation (≈ our gauntlet)
- Don't trust your backtest — walk-forward/Monte-Carlo/sensitivity do not test the break. Replay each strategy cold on SNB 2015, CNY 2015, COVID March 2020, US election 2016 → 4 outputs: intra-event DD, event P&L, max slippage, risk-engine reaction time. (add #6 — for PANDORA: COVID 2020, Volmageddon Feb 2018, yen carry Aug 2024, bear 2022.)
- The regime filter trap — a mediocre filter sometimes costs more than the regimes it avoids. Before building: per-regime attribution, then deliberately degrade the filter (delay the switches by a realistic lag, flip a share of the calls) and compare with unfiltered. If the edge survives the degraded filter → keep; otherwise = complexity tax. → To apply directly to v_joint. (add #4)
- Drag. Drop. Overfit. — "a curve is not an edge": code is no longer the barrier, edge design is. → validates our discipline (we do not ship a pretty curve without thesis/regime/failure mode).
- Testing never stops — a live strategy stays a hypothesis. Monitor signal-health metrics that move before the equity curve: live-vs-backtest variance, win-rate drift, average R falling while the win rate holds (EV repriced), execution quality. + double out-of-sample (recent AND before the in-sample). (add #7)
Strategy death & execution
- Your trading strategy will fail (kill switch) — rule written BEFORE going live, 3 components: DD threshold = backtested 95th pctile; recovery threshold = 1.5× the 90th pctile; factor drift flag (rolling decomposition). When it fires: off, not pause (pause reopens the door to override). (add #5)
- AI quant — AI = strong at candidate generation + red-team + infra code + factor exposure; weak at portfolio construction / selection / reading live variance. Three-check to trust an AI output: source of the stat, instrument specificity, falsifiability. → Exactly our usage; we can formalize the three-check in the lab workflow.
Method note
These ideas come from a multi-strategy FX/indices/metals book (20-30 strats). PANDORA is more concentrated (v_joint + sleeve). The most transferable as-is: #1, #2, #3, #4 (metrics + sleeve + regime validation). #5-#8 are mostly relevant once Skylit runs several simultaneous positions.