Inside BitBank · Engineering an Autonomous AI Research Lab · 3 of 10

Forecasting Models: From Baselines to Transformers

2026-07-24 · 12 min read

Complexity earns its place only when it improves unseen decisions. We treat model families as competing tools, not identities.

There is no universally best forecasting model. A model is useful only relative to a horizon, dataset, decision, latency budget, and baseline. BitBank therefore treats forecasting as a portfolio of testable approaches rather than a search for one permanent architecture.

Begin With the Embarrassing Baseline

Every sophisticated model should face simple competitors: last value, rolling mean, exponential trend, seasonal repetition, volatility-scaled momentum, and sometimes no trade at all. These baselines are fast, interpretable, and surprisingly difficult to beat after costs.

If a transformer cannot improve on an exponential moving average under the same causal protocol, that result is not embarrassing for the baseline. It is evidence that the extra capacity has not earned production complexity.

Forecasts and Decisions Are Different Objects

A model may forecast a return distribution, price path, volatility, rank, or regime probability. A trading policy converts that estimate into action. Conflating the two makes diagnosis difficult.

Suppose a model predicts direction slightly better than chance but changes its mind every hour. The forecast score can improve while trading performance deteriorates through turnover. A decision layer may add thresholds, hysteresis, abstention, exposure caps, and volatility targeting. These controls should be evaluated separately from predictive accuracy.

Boosted Trees

Gradient-boosted trees remain strong for structured features. They train quickly, handle nonlinear interactions, and provide useful diagnostics. With carefully lagged returns, ranges, volume, volatility, and cross-sectional context, they can test a large family of hypotheses cheaply.

Their flexibility is also dangerous. Hundreds of correlated features and repeated sweeps can fit transient regimes. Honest folds, feature ablations, monotonic expectations where appropriate, and recent holdouts matter more than the in-sample importance chart.

Time-Series Foundation Models

Chronos-style models approach numerical sequences with ideas borrowed from language modelling. Pretraining across many series can provide a useful prior for trend, seasonality, scale, and uncertainty. Instead of fitting every pair from scratch, a foundation model can produce probabilistic forecasts from context.

Zero-shot ability is appealing, but financial use still requires validation. The pretraining distribution may differ from crypto. A model that predicts price levels well may not rank tradable returns well. Context length, normalization, covariates, sampling frequency, and quantile calibration all change the result.

Probabilities Are More Useful Than Certainty

A point forecast hides uncertainty. Quantiles or simulated paths let the decision layer ask better questions: How wide is the distribution? Is expected upside large relative to downside? Does the conclusion survive plausible fees? Should the system abstain because uncertainty is unusually high?

Calibration is essential. A nominal 90 percent interval should contain outcomes at roughly that rate on relevant unseen data. Narrow intervals that miss constantly create false confidence; very wide intervals provide little decision value.

Global and Pair-Specific Models

A global model shares statistical strength across assets and encounters more regimes. A pair-specific model can preserve behaviour that global averaging erases. We do not settle this question by philosophy.

Shared models, per-pair heads, pair embeddings, and fixed pair-specific policies can all enter the same tournament. The production choice may share feature code and validation infrastructure while using different decision boundaries for Bitcoin, Ethereum, and other markets.

Model Selection Is a Multiple-Testing Problem

Training ten thousand configurations virtually guarantees a lucky winner. The more ideas we test, the stronger our evidence controls must become. Nested validation, locked holdouts, paired fold comparisons, false-discovery awareness, and replication across regimes reduce the probability that search itself becomes the hidden source of performance.

We also record the full candidate population, not only the winner. A result means something different if one model narrowly wins among five sensible candidates than if it is the maximum of fifty thousand unstable trials.

Operational Cost Is Part of Model Quality

Models differ in memory, cold-start time, throughput, numerical portability, and failure surface. A marginal forecast improvement may not justify a service that is ten times slower or impossible to reproduce live. Export formats, bounded context, deterministic preprocessing, and fallback behaviour belong in the selection scorecard.

The Champion, Challenger, and Baseline

A healthy production system keeps three roles visible. The champion serves current decisions. Challengers run through the same evaluation and may shadow production. The baseline remains as a sanity check. If both champion and challenger suddenly lose to “do nothing,” the system needs investigation, not a more elaborate story.

The model family is therefore less important than the comparison discipline around it. Chapter four turns to the machine that makes those comparisons possible at scale: GPUs, CUDA, storage, distributed work, experiment scheduling, and the engineering of autonomous research.