Primarily based on: Potosnak, W., Wolff, M., Cao, M., Ma, R., Konstantinova, T., Efimov, D., Mahoney, M.W., Oreshkin, B., & Olivares, Ok.G. “Forking-Sequences: Statistically and Computationally Environment friendly Multi-Horizon Forecasting with Decreased Volatility.” Transactions on Machine Studying Analysis, 2026.
(Disclaimer: Code implementation not used within the paper; not affiliated with Amazon — offered as a reference for forking-sequences and forecast ensembling)
TL;DR
- Ensembling, almost free of charge. Forking-sequences already produces overlapping forecasts for each goal date throughout FCDs in a single ahead cross, so ensembling them at inference provides no further encoder computation in contrast with window-sampling.
- Two new forecast volatility metrics. scaled Forecast Proportion Change (sFPC) measures uncooked revision measurement in actual time (no floor fact wanted); Extra Volatility (EV) goes additional, rewarding accuracy-improving revisions and solely penalizing those that transfer forecasts away from the reality or overshoot it.
- Decreased volatility with out sacrificing accuracy. Exponential-smoothing forecast ensembling (α = 0.9) reduces sEV by 10–13% throughout all encoder varieties, with lower than 0.1% accuracy degradation.
- Works zero-shot on fashions pretrained with window-sampling. Forecast ensembling utilized to pretrained Time Collection Basis Fashions (TSFMs) — Chronos-2, Toto 2.0, TimesFM, PatchTST, N-BEATS — cuts volatility by ~10% with negligible accuracy value (lower than 0.1%).
In Half I, we launched forking-sequences, a neural community architectural design that collectively encodes and decodes a time sequence throughout all forecast creation dates (FCDs) in a single ahead cross. We confirmed why it is a statistically and computationally extra environment friendly coaching paradigm than window-sampling. In Half II, we flip to a unique however equally necessary drawback: forecast volatility.
Why Forecast Volatility Issues
Accuracy is often the headline metric for a forecasting mannequin, however it is not the one factor that issues in manufacturing. As a multi-horizon forecasting system operates over time, it generates a number of overlapping forecasts for a similar future goal date — one from every new FCD as extra information turns into out there. This sequence of updates is a forecast revision, and the way constant (or erratic) these revisions are is what we outline as forecast volatility.
Fig. 1: Forecasts (a) with out and (b) with forecast ensembling utilized. Forecast ensembling reduces volatility throughout FCDs, leading to extra secure and constant forecast distributions. Pink arrows point out the route of forecast revisions. Traces present P50 (median) forecasts throughout totally different FCDs. By reusing encoder computations, forking-sequences permits computationally environment friendly forecast ensembling with negligible further value.
Think about {an electrical} grid operator utilizing load forecasts to plan energy provide. If a forecast revises from 45 GW to 65 GW forward of a warmth wave, that is a helpful revision; it tells operators to activate reserve crops. But when forecasts bounce round erratically between FCDs with out new info justifying the change, that undermines belief and complicates planning. The aim is not to get rid of revisions, it is to differentiate benign, informative revisions from extreme, erratic ones.
This raises two questions we deal with instantly within the paper:
?
How can we measure forecast volatility in a method that separates helpful revisions from dangerous ones?
?
Are there architectural designs that cut back volatility with out hurting accuracy?
Forking-Sequences as a Pure Forecast Ensembling Mechanism
As a result of forking-sequences generates forecasts for each FCD in a single ahead cross, it naturally produces a number of overlapping predictions for a similar goal date. Recall the forecast revision relationship: the prediction for a given goal made at FCD t+1 is a revision of the prediction made at FCD t for a similar date. Forecast revisions with the forking-sequences paradigm are proven in Fig. 2.
This overlapping grid construction means forking-sequences fashions could be ensembled free of charge (or almost so) at inference time when it comes to saving encoder computation in contrast with window-sampling, which requires a number of impartial mannequin ahead passes. Given forecasts outputs through forking-sequences, we simply common (or in any other case mix) the totally different FCD-level predictions for a similar goal date portrayed because the diagonal band in Fig. 3a:
[
begin{equation}
widetilde{mathbf{Y}}_{t,h} = frac{1}{H}sum_{k=0}^{H} widehat{mathbf{Y}}_{t-k, h+k} qquad text{for } tgeq H.
label{eq:forking_sequences_ensemble}
end{equation}
]
Fig. 3: We adapt forking-sequences throughout inference to ensemble a number of forecasts of the identical future date by computing a operate (ex., shifting common) throughout predictions generated from earlier FCDs. b) Forking-sequences ensembling reduces forecast volatility, lowering the estimators variance with a linear convergence charge analogous to the weak legislation of enormous numbers.
Though it’s tempting to anticipate a variance-reduction habits much like the outcomes of Theorem 1, you will need to acknowledge that forecast variance naturally will increase the additional a forecast is from its corresponding commentary. Consequently, there’s an inherent restrict to how a lot ensembling can cut back volatility: older forecast revisions carry considerably greater uncertainty, whereas more moderen revisions are each extra correct and fewer variable. This makes it fascinating for an ensemble to position higher weight on newer forecasts moderately than treating all revisions equally.
New Forecast Volatility Metrics
We introduce scaled Forecast proportion Change (sFPC) to measure the relative change in predicted quantiles throughout consecutive forecast creation dates, offering a quantitative view of temporal volatility or forecast revision charges. Impressed by the sMAPE metric, sFPC makes use of a symmetric denominator, based mostly on each present and former forecasts, to mitigate problems with numerical instability [1]. This design ensures robustness when coping with small predicted values and avoids the division-by-zero issues frequent in conventional percentage-based metrics.
[
mathrm{sFPC}^{(q)}left(hat{mathbf{y}}^{(q)}_{[b][t][h]}proper)
= frac{200}{B occasions T occasions H} sum_{b,t,h} frac{|hat{y}^{(q)}_{b,t+1,h}-hat{y}^{(q)}_{b,t,h+1}|}{|hat{y}^{(q)}_{b,t+1,h}| + |hat{y}^{(q)}_{b,t,h+1}|} .
]
Computing sFPC between consecutive forecasts treats all revisions as equally undesirable, even ones that clearly enhance accuracy. To deal with this, we additionally introduce scaled Extra Volatility (sEV), a metric for probabilistic forecasts that solely penalizes revisions that transfer a forecast away from the reality, or that overshoot it. sEV is designed to reward accuracy-improving forecast revisions whereas distinguishing them from dangerous volatility. sEV is outlined as:
[
mathrm{sEV}left(mathbf{y}_{[b][t][h]}, hat{mathbf{y}}_{[b][t][h]}proper) = frac{sum_{b,t,h} mathrm{EV}(y_{b,t,h}, mathbf{hat{y}}_{b,t,h+1}, mathbf{hat{y}}_{b,t+1,h})}{sum_{b,t,h} |y_{b,t,h}|}, quad textual content{the place}
]
[
mathrm{EV}(y,;mathbf{hat{y}}_1,; mathbf{hat{y}}_2) = mathrm{QL}(mathbf{hat{y}}_2,mathbf{hat{y}}_1) – (mathrm{QL}(y,mathbf{hat{y}}_1)-mathrm{QL}(y,mathbf{hat{y}}_2)), quad text{and}
]
[
mathrm{QL}_q(y, hat{y}^{(q)}) = q(y-hat{y}^{(q)})_+ + (1-q)(hat{y}^{(q)}-y)_+ .
]
EV has three helpful properties, confirmed formally within the paper:
- Zero penalty for enhancing revisions, proven in Fig. 4a: if a revision strikes proportionally nearer to the bottom fact, touchdown on the direct path between the reality and the prior forecast, EV = 0.
- Most penalty for deteriorating revisions, proven in Fig. 4b: if a revision strikes the forecast farther from the reality, with the previous forecast sitting between the reality and the brand new one, EV equals the total accuracy degradation, the distinction in quantile loss between the brand new forecast and the previous one.
- Overshoot penalty, proven in Fig. 4c: if a revision strikes in the precise route however overshoots, with the reality touchdown between the previous and new forecast, EV penalizes solely the brand new forecast’s quantile loss towards the reality.
Fig. 4: Instance penalty habits of the Extra Volatility (EV) metric. EV distinguishes accuracy-improving revisions from accuracy-degrading ones, assigning no penalty when revisions enhance accuracy, whereas asymmetrically penalizing each deteriorating and overshooting revisions in response to their impression on accuracy.
One necessary distinction: sFPC could be computed at prediction time for real-time monitoring, because it would not require floor fact. sEV, against this, depends upon the ground-truth worth, so it could solely be utilized retroactively to evaluate forecast volatility.
Empirical Outcomes: Volatility Discount With out Sacrificing Accuracy
The core empirical declare: for forking-sequences fashions, making use of exponential-smoothing ensembling at inference (α = 0.9) reduces forecast volatility (sEV) considerably whereas sustaining forecast accuracy.
We present that for forking-sequences fashions, forecast ensembling throughout inference can cut back forecast volatility in comparison with forecasts with out ensembling for all encoders. Particularly, making use of exponential smoothing at inference to fashions skilled with forking-sequences yields median proportion enhancements in sEV throughout datasets of 13.2%, 13.0%, 10.9%, 10.2%, and 11.2% for RNN, LSTM, CNN, Transformer, and StateSpace-based architectures, respectively, whereas sustaining forecast accuracy (lower than 0.1% degradation in sCRPS as proven in Fig. 5).
Fig. 5: Distribution of proportion enchancment in (a) sCRPS, (b) sEV, and (c) sFPC metrics throughout datasets for various encoder varieties with forking-sequences forecast ensembling in contrast with no ensembling. Every dataset’s metric is averaged over 5 random seed runs. Proportion enchancment higher than zero signifies forecast ensembling achieves decrease forecast error or volatility.
We embody an ablation research throughout totally different ensembling methods (shifting common, shifting median, cumulative common, exponential smoothing at α = 0.1/0.5/0.9), and discover that exponential smoothing with excessive α (0.9) offers the most effective trade-off; it weights near-term (extra correct) forecasts extra closely, minimizing the accuracy value of smoothing out volatility. Decrease α values cut back volatility additional however at a better value to accuracy.
A Bonus: Zero-Shot Volatility Reductions for Pretrained Basis Fashions
Forecast ensembling profit is not restricted to fashions particularly skilled with forking-sequences. We are able to apply forecast ensembling to pretrained fashions initially skilled with window-sampling by accumulating forecast revision outputs. We reveal this with pretrained Time Collection Basis Fashions (TSFMs), together with Chronos-2, Toto 2.0, TimesFM, and pretrained PatchTST and NBEATS, in a zero-shot setting.
Fig. 6: Distribution of proportion enchancment in (a) sCRPS, (b) sEV, and (c) sFPC metrics throughout datasets for various encoder varieties with forking-sequences forecast ensembling in contrast with no ensembling. Proportion enchancment higher than zero signifies forecast ensembling achieves decrease forecast error or volatility. Forecast ensembling can considerably cut back forecast volatility (sEV, sFPC) whereas sustaining forecast accuracy (sCRPS), demonstrating its utility as a general-purpose inference method for fashions skilled with both forking-sequences or window-sampling.
Throughout the M-series benchmark, this easy method achieved a median ~10% discount in forecast volatility, with lower than 0.1% degradation in accuracy (sCRPS). In different phrases: forecast ensembling through forking-sequences-style aggregation is a general-purpose, nearly-free method that can be utilized in forecasting pipelines no matter whether or not the underlying mannequin was initially skilled with forking-sequences.
🔑 Takeaways
1
Forking-sequences’ grid construction naturally produces overlapping forecasts throughout FCDs, enabling near-free ensembling at inference time by reusing already-computed encoder outputs.
2
The brand new scaled Extra Volatility (sEV) metric distinguishes accuracy-improving revisions from dangerous ones — a significant enchancment over naive percentage-change volatility measures.
3
Ensembling forking-sequences forecasts through exponential smoothing cuts volatility by ~10–13% throughout encoder architectures with out sacrificing accuracy.
4
This profit extends to zero-shot use with pretrained basis fashions like Chronos-2, Toto 2.0, and TimesFM, attaining roughly 10% diminished forecast volatility with <0.1% accuracy value.
We acknowledge that ensembling could be built-in throughout each coaching and inference with forking-sequences, and might be additional prolonged with learnable parameters as explored in [2]. We go away training-time ensembling integration to future work.
Collectively, Components I and II purpose to construct broader consciousness of forking-sequences and promote its adoption as a default architectural possibility in open-source neural forecasting libraries and future analysis. This work additionally advocates for higher consciousness of volatility metrics as a complement to straightforward accuracy metrics, encouraging their routine adoption in forecasting analysis.
References:
[1] Rob J. Hyndman and Anne B. Koehler. One other take a look at measures of forecast accuracy. Worldwide Journal of Forecasting, 22(4):679 – 688, 2006. ISSN 0169-2070.
[2] Carson Eisenach, Yagna Patel, and Dhruv Madeka. MQTransformer: Multi-Horizon Forecasts with Context Dependent and Suggestions-Conscious Consideration. In Maria Florina Balcan and Marina Meila, editors, Submitted to Proceedings of the thirty eighth Worldwide Convention on Machine Studying. PMLR. Working Paper model out there at arXiv:2009.14799, 8 2021.
