What 70 Time-Series Folds Taught Us About Trading Model Selection

2026-07-12

A trading model can look exceptional in one market regime and still have no durable edge. We recently optimized BitBank's BTC strategy over hourly data from 2021 through 2026. The most useful result was not a complicated neural network. It was a better validation process that rejected several attractive but fragile ideas and promoted a deliberately simple, risk-controlled trend model.

The Result

We compared every candidate with the deployed adaptive selector on the same 70 chronological test folds. Each fold covered 504 hours, used a separate 336-hour training window, and left a 24-hour purge gap between training and testing. Signals used information available at the candle close, executed at the following candle open, and paid fees plus a 5-basis-point slippage allowance.

The paired comparison produced a t-statistic of 2.34. More importantly, the improvement did not depend entirely on the latest market: the candidate averaged +0.628% over the older 46 folds and +0.194% over the most recent 24-fold holdout. At twice the normal modeled execution cost, it remained slightly positive at +0.172% per fold.

What We Deployed

The new BTC-only model follows a roughly 21-day exponential moving average with a 1% deadband. It uses hysteresis, so it retains an established position while price is inside the deadband instead of paying spread on every small crossing. Exposure is volatility-targeted at 40% annualized volatility and capped at 0.5 times equity. The parameters are fixed rather than reselected from a noisy two-week leaderboard.

Lesson 1: Time-Series Folds Must Respect Causality

Random K-fold validation is usually inappropriate for trading. Neighboring rows share market state, and future observations can leak indirectly into training. Chronological folds preserve order. A purge or embargo between train and test windows also prevents a forward-return label or long holding period from crossing the boundary.

Lesson 2: Compare Candidates Fold by Fold

Comparing two grand totals discards useful information. Because both strategies experienced the same BTC market in each fold, we measured the return difference within every fold. This paired design removes much of the market-regime noise and asks the practical question: did the candidate improve on the incumbent under the same conditions?

Lesson 3: A Larger Dataset Can Reverse the Decision

An earlier eight-month sample favored two-times leverage and short-horizon mean reversion. After extending the history back to 2021, that apparent edge disappeared: the leveraged version averaged roughly -8% per three-week fold. This is a classic regime-selection failure. A model chosen only during a mean-reverting period can look precise while learning a temporary property of the market.

Lesson 4: Stress the Assumption That Creates the Profit

Trading ML is inseparable from execution. We reran the candidate with wider costs rather than assuming the historical candle always offered an ideal fill. It survived a 22-basis-point cost setting, but turned modestly negative at an extreme 40-basis-point setting. That result is why we kept the 0.5-times exposure cap instead of converting a statistical improvement into an aggressive leverage increase.

Lesson 5: Offline and Live Feature Context Must Match

Poloniex caps a public candle response at 500 bars. A research feature that silently uses thousands of preceding bars cannot be reproduced by the live service. The production EMA is therefore explicitly bounded to the same 500-bar context, and a regression test verifies that changing older candles cannot change today's signal.

Why the Simpler Model Won

We also evaluated GPU-friendly boosted-tree forecasts and cross-sectional ranking ideas. Their recent in-sample results sometimes looked appealing, but honest fold performance was negative after turnover. Complexity is valuable only when it improves unseen outcomes. In this round, a simple trend state, conservative sizing, and a stricter experimental design produced the better result.

This is evidence of improvement over the previous simulator baseline, not a guarantee of future profit. We will repeat the same process pair by pair and publish both successful and rejected rounds. Transparent negative results are part of good model development: they stop the next sweep from rediscovering the same overfit idea.