Kieran Duff — "The Letter": what we add to the PANDORA arsenal

Source: https://kieranduff.com/letters/ (Kieran Duff, XAQP, live systematic ~$3.7M on Darwinex). 12 letters studied on running a systematic book: sizing, risk, drawdown, correlation, anti-overfit validation. Goal: sort what really complements PANDORA (gauntlet, v_joint, multi-asset sleeve) vs what we already have.

TL;DR — what is worth adding (prioritized)

# Idea PANDORA today Action
1 Time-under-water + recovery time missing from backtest/engine.py ✅ pandora/backtest/extended_metrics.py
2 Risk-of-ruin at live size missing ✅ extended_metrics.py
3 Correlation budgeting (P&L, haircut) sleeve = fixed weights 50/30/20 ✅ strategies/multi_asset_sleeve.py (correlation_budgeted_allocation)
4 Regime-filter degradation test never "degraded" ✅ lab/regime_degradation.py
5 Decay kill-switch (3 components) manual kill-switch only ✅ pandora/risk/decay_kill_switch.py
6 Crisis replay battery no battery ✅ lab/crisis_replay.py
7 Live latency simulator — ✅ lab/latency_sim.py
8 Portfolio heat cap daily limits only ✅ pandora/risk/portfolio_heat.py
9 Parameter stability map — ✅ lab/param_stability.py
10 Regime bootstrap stress — ✅ lab/regime_bootstrap.py
11 Vol-targeting vs flat (comparator) — ✅ lab/vol_targeting.py
12 Capacity/slippage model — ✅ lab/capacity_model.py
13 Skylit meta-labeling (decision dataset) — ✅ pandora/skylit/decision_log.py
14 Trials ledger + honest DSR — ✅ pandora/research/trials_ledger.py
15 Signal-health monitoring of the live book pre-live gauntlet only ✅ pandora/pipeline/signal_health.py (wired into the brief)
— Anti-flash-crash overlay tested → rejected lab/fastcrash_overlay.py (COMPLEXITY TAX)
— AI = generator + red-team / Sharpe-Sortino-Calmar already OK Nothing to do

🔑 What these tools REVEALED about v_joint (the real findings)

  1. Fragile to DELAY, not to bad calls (regime_degradation + latency_sim) — flipping 20% of the days costs little (Sharpe 1.09→0.89), but 2 days of latency collapse it (→0.47). Expected live Sharpe ≈ 0.75 (−31%) at realistic delay. ⟹ the DIX-GEX execution window / latency is live risk #1.
  2. Bear-market insurance, not anti-flash-crash (crisis_replay) — crushes DD on slow breakdowns (COVID −1.1% vs B&H −33.4%) but EXPOSED on the 2024 yen V-crash. And the fast overlay does not save it (fastcrash_overlay = complexity tax: it confirms the crash too late).
  3. Not curve-fit (param_stability) — the 4 frozen thresholds sit on a plateau, and the frozen value is never the peak = conservative choices. Good sign for the freeze.
  4. The displayed MaxDD is lucky (regime_bootstrap) — −12.1% is at the 87th pctile; provision −26%.
  5. Flat-0.6 stays optimal (vol_targeting) — vol-targeting degrades Sharpe AND MaxDD (v_joint already de-risks through the regime). Capacity ≈ $500M (capacity_model).

Sizing & risk

  • The maths of ruin — P(ruin) ≈ ((1-edge)/(1+edge))^(risk units). Halving the size does roughly the same job on ruin as doubling the edge. → Every candidate must pass a probability-of-ruin check at the proposed live size. (add #2)
  • Too much risk (portfolio heat) — cap on summed open risk (3-5% equity); entries compete for a "heat" budget. Per-trade discipline ≠ portfolio discipline. (add #8)

Correlation (≈ our SPY/QQQ/IWM sleeve)

  • Correlation: the silent killer — compute correlation on P&L (not returns), and on the residuals after a vol-regime factor: two strategies that lever the same way with vol look uncorrelated raw and become correlated in the residual.
  • The correlation trap — budgeting: <0.3 independent; 0.3-0.6 proportional size haircut; >0.6 = a single strategy in two outfits. Window ≥6 months, monthly update, read by the sizing code. → Our sleeve hard-codes 50/30/20 weights while SPY/QQQ/IWM are highly correlated: we oversize. (add #3)

Distribution metrics

  • Is your Sharpe lying? (Sortino/Calmar) — skew diagnostic: Sortino > √2·Sharpe = positive skew (underestimated by Sharpe); < = negative skew (unpriced tail). Present the Sharpe+Sortino+Calmar triple. → We already have them; adding the skew diagnostic to the tearsheet would be free.
  • Time under water — longest time below the previous peak + MAR = CAGR/MaxDD. DD depth hides what the allocator lives through (a 5% that recovers in 3 weeks ≠ 5% over 10 months). (add #1)

Anti-overfit validation (≈ our gauntlet)

  • Don't trust your backtest — walk-forward/Monte-Carlo/sensitivity do not test the break. Replay each strategy cold on SNB 2015, CNY 2015, COVID March 2020, US election 2016 → 4 outputs: intra-event DD, event P&L, max slippage, risk-engine reaction time. (add #6 — for PANDORA: COVID 2020, Volmageddon Feb 2018, yen carry Aug 2024, bear 2022.)
  • The regime filter trap — a mediocre filter sometimes costs more than the regimes it avoids. Before building: per-regime attribution, then deliberately degrade the filter (delay the switches by a realistic lag, flip a share of the calls) and compare with unfiltered. If the edge survives the degraded filter → keep; otherwise = complexity tax. → To apply directly to v_joint. (add #4)
  • Drag. Drop. Overfit. — "a curve is not an edge": code is no longer the barrier, edge design is. → validates our discipline (we do not ship a pretty curve without thesis/regime/failure mode).
  • Testing never stops — a live strategy stays a hypothesis. Monitor signal-health metrics that move before the equity curve: live-vs-backtest variance, win-rate drift, average R falling while the win rate holds (EV repriced), execution quality. + double out-of-sample (recent AND before the in-sample). (add #7)

Strategy death & execution

  • Your trading strategy will fail (kill switch) — rule written BEFORE going live, 3 components: DD threshold = backtested 95th pctile; recovery threshold = 1.5× the 90th pctile; factor drift flag (rolling decomposition). When it fires: off, not pause (pause reopens the door to override). (add #5)
  • AI quant — AI = strong at candidate generation + red-team + infra code + factor exposure; weak at portfolio construction / selection / reading live variance. Three-check to trust an AI output: source of the stat, instrument specificity, falsifiability. → Exactly our usage; we can formalize the three-check in the lab workflow.

Method note

These ideas come from a multi-strategy FX/indices/metals book (20-30 strats). PANDORA is more concentrated (v_joint + sleeve). The most transferable as-is: #1, #2, #3, #4 (metrics + sleeve + regime validation). #5-#8 are mostly relevant once Skylit runs several simultaneous positions.