The Order E book isn’t a Metronome: Why Transformers Fail at Excessive-Frequency Buying and selling
Introducing LOBMamba-Hawkes v3: A continuous-time, asynchronous notion engine for microstructure information.
By Lyle Dickie — Alpha Labs Engineering
The restrict order e book is just not a metronome.
It doesn’t tick at common intervals. It doesn’t arrive in neat, equally-spaced packets. It’s a residing ecosystem of asynchronous impulses — the place a microsecond burst of cancelled orders typically carries extra sign than ten minutes of quiet resting liquidity.
And but, for the final 5 years, the quantitative finance neighborhood has been making an attempt to shove this basically continuous-time information into architectures designed for discrete sequences. The result’s predictable: fashions that may detect patterns in backtests however lose capital the second they contact a reside unfold.
Right this moment, Alpha Labs Engineering is open-sourcing the bottom class for LOBMamba-Hawkes v3 — a notion engine that treats the restrict order e book as what it really is: a self-exciting level course of in steady time. It combines the linear-time scaling of the Mamba-3 Selective State Area Mannequin with the temporal physics of a Hawkes exponential decay filter and a novel sparse landmark consideration mechanism that gives precise historic recall with out quadratic price.
⭐ [Star the repo on GitHub → Alpha Labs Engineering / LOBMamba-Hawkes](https://github.com/Alpha-Labs-Engineering/LOBMamba-Hawkes-v3)
Should you construct buying and selling programs, research market microstructure, or work on sequence modeling past Transformers — this text is for you.
The Drawback: Your Notion Engine Is Deaf to Time
Earlier than we construct the answer, we have to perceive why the usual strategy fails. And it fails for causes which are structural, not parametric — you can’t tune your means out of the mistaken inductive bias.
The Quadratic Wall
The self-attention mechanism on the coronary heart of each Transformer computes pairwise relevance between each token in a sequence. For a sequence of size L, this prices O(L²) in each time and reminiscence. In pure language processing, the place sequences not often exceed just a few thousand tokens, that is manageable. In high-frequency buying and selling, it’s deadly.
A single liquid instrument on the CME can generate 1000’s of Degree 3 Market-by-Order (MBO) updates per second. To seize the structural order circulate that institutional desks depart behind — the gradual accumulation of resting liquidity, the sudden sweep of a e book stage, the cascade of triggered stops — an agent wants a receptive subject spanning tens of 1000’s of ticks.
Here’s what occurs whenever you strive:
| Sequence Size (L) | Transformer Latency (ms) | Mamba Latency (ms) |
|--- |--- |--- |
| 50 | 1.71 | 2.15 |
| 200 | 1.74 | 2.16 |
| 1,000 | 1.84 | 2.14 |
| 5,000 | **21.79** | 4.48 |
Desk: Ahead-pass latency on an NVIDIA L4 GPU. Benchmarked on LOBTransformer vs. LOBMamba architectures (Dickie, 2026).
At L=200, the Transformer is barely quicker as a consequence of excessive parallelization. However at L=5,000 — the size the place structural order circulate really lives — latency explodes by 12×. The Transformer breaches sub-millisecond viability. The Mamba spine barely notices the distinction.
This isn’t an implementation situation. It’s a mathematical ceiling. Duman Keleş et al. (2023) proved that the O(L²) complexity of self-attention holds “until the Robust Exponential Time Speculation is fake.” You can’t approximate your means out of it with out destroying the very consideration patterns that give Transformers their energy.
The Metronome Delusion
However computational price is just half the issue. The deeper failure is representational.
Commonplace Transformers inject sequence order via sinusoidal positional encodings — a scheme that assigns every place a novel frequency signature. Place 1, place 2, place 3. The belief is that tokens are equally spaced. For language, that is advantageous. For market microstructure, it’s catastrophic.
Take into account two situations in a restrict order e book:
Situation A: Ten aggressive market promote orders arrive inside 500 microseconds. That is institutional liquidation — a hearth sale. The e book is being swept.
Situation B: Ten passive restrict order additions arrive over 30 seconds. That is background market-making. Nothing is occurring.
A Transformer with positional encodings sees precisely the identical construction in each instances: ten tokens at positions [1, 2, 3, …, 10]. The temporal info — the very dimension that distinguishes structural contagion from background noise — has been discarded earlier than the primary consideration head ever fires.
Zeng et al. (2023) demonstrated this empirically by shuffling the enter sequences to Transformer time-series fashions. The outcome: almost equivalent outputs. The positional encodings weren’t restoring temporal which means. The eye mechanism was working as a permutation-invariant set perform, not a sequence mannequin.
If you destroy the temporal dimension of market information, you destroy the alpha.
The Structure: How LOBMamba-Hawkes Truly Works
LOBMamba-Hawkes solves each issues concurrently. It replaces the quadratic consideration mechanism with a linear-time state house spine, and it replaces discrete positional encoding with a physics-based temporal filter derived from the self-exciting dynamics of monetary markets.
The structure has three elements. Each addresses a particular failure mode of ordinary approaches.
Element 1: The Hawkes Exponential Filter — Educating the Community to Really feel Time
Earlier than a single function ever reaches the neural spine, it passes via a learnable Hawkes exponential time-decay filter. That is the part that makes your entire structure “continuous-time conscious.”
The thought is grounded in a well-established commentary in market microstructure: order circulate is self-exciting. A big market order will increase the likelihood of subsequent orders — triggered stop-losses, momentum-ignition algorithms, latency-arbitrage reactions. This clustering of occasions is exactly what Hawkes processes mannequin, and it has been a foundational software in quantitative finance since Bacry, Mastromatteo, and Muzy’s landmark survey (2015) demonstrated its utility for LOB dynamics.
However fairly than utilizing the Hawkes course of as a standalone statistical mannequin, we embed it immediately into the notion layer of the neural community as a learnable gate.
Right here is the arithmetic. We outline a single learnable parameter θ, mapped via softplus to ensure positivity:
“`
κ = ln(1 + exp(θ))
“`
For each incoming MBO tick, we compute a Hawkes Weight based mostly on the elapsed microseconds because the earlier occasion:
“`
W_i = exp(−κ · Δt_i)
“`
The conduct is intuitive. When Δt → 0 (rapid-fire burst), W → 1 — the tick retains its full structural vitality. When Δt → ∞ (remoted noise), W → 0 — the tick’s affect decays to nothing. The community bodily perceives contagion.
The Worth Anchor — Why You Can not Globally Decay
A vital engineering element: you can’t naively multiply your entire enter vector by the Hawkes weight. If the value function have been multiplied by W = 0.01 throughout a gradual market interval, the community would understand the asset’s worth collapsing towards zero.
LOBMamba-Hawkes applies selective function dampening. The decay is utilized solely to the “vitality” dimensions of the tick — Dimension, Aspect, and the MBO motion flags — whereas leaving the Worth dimension fully unanchored:
# The Hawkes Exponential Filter
decay_rate = F.softplus(self.hawkes_decay)
hawkes_weight = torch.exp(-decay_rate * dt) # [B, L, 1]# Selective dampening: worth preserved, vitality decayed
filtered_ticks = x * hawkes_weight
This ensures the notion engine at all times maintains an absolute spatial anchor to the present market worth, whereas dynamically modulating the perceived depth of surrounding order circulate. When the market is quiet, the agent observes a steady worth with near-zero vitality — and stays dormant. When contagion strikes, the vitality dimensions spike, and the agent acts.
The Initialization Entice
One sensible element that can prevent days of debugging: the initialization of the Hawkes decay parameter is numerically vital.
As a result of Δt is measured in microseconds, a naïve initialization of θ ≈ 0 produces κ = softplus(0) ≈ 0.693. Multiply this by a regular microsecond interval (Δt = 1,000,000) and the exponential time period underflows to precisely 0.0. Each gradient is zero. The community learns nothing. The Hawkes filter turns into a useless node.
We initialize the pre-softplus weight to −12.0, producing an preliminary decay fee of κ ≈ 6.14 × 10⁻⁶. This ensures the exponential decay operates gracefully throughout the microsecond timescale from the very first ahead cross:
# Essential: initialize for microsecond-scale dynamics
self.hawkes_decay = nn.Parameter(torch.tensor([-12.0]))
That is the form of element that separates an structure that works on dummy information from one which works on CME forex futures.
Element 2: The Mamba-3 Spine — Linear-Time State Area Modeling
As soon as the ticks are temporally filtered, they enter a Mamba-3 Selective State Area Mannequin spine. That is the place the structure resolves the latency drawback.
Why SSMs Are Pure for Order Books
There’s a deep mathematical purpose why state house fashions are the right inductive bias for restrict order e book information — and it isn’t nearly computational complexity.
A state house mannequin maintains a compressed hidden state h(t) that evolves repeatedly via time, updating selectively as new inputs arrive. The order e book is itself a type of state — a repeatedly evolving illustration of provide and demand that updates asynchronously as messages arrive. The architectural metaphor is precise: the SSM’s hidden state is the microstructure.
Albert Gu’s analysis group put it elegantly: “Transformers are like databases — they deal with each new commentary as an essential merchandise that’s filed away. SSMs are like brains — finite-sized recollections which are at all times on, processing new inputs and producing new outputs in real-time.” For top-frequency buying and selling, you want a mind, not a database.
The Mamba-3 Recurrence
LOBMamba-Hawkes v3 makes use of the Mamba-3 structure (Gu et al., ICLR 2026) — the newest and most important evolution of the SSM household. The core recurrence implements a 3-term exponential-trapezoidal discretization:
“`
h_t = exp(Δ_t · A) · h_{t-1}
+ (1 − λ_t) · Δ_t · exp(Δ_t · A) · B_{t-1} · x_{t-1}
+ λ_t · Δ_t · B_t · x_t
“`
It is a second-order correct integration scheme, a major improve over the first-order Euler methodology utilized in Mamba-1 and Mamba-2. The info-dependent parameter λ ∈ (0,1) controls the trapezoidal mix: when λ = 1, the mannequin recovers Mamba-2’s exponential-Euler discretization; when λ = 0.5, it implements the usual trapezoidal rule.
For monetary information, the important thing innovation is that the step dimension Δ is itself data-dependent — it modulates based mostly on the enter. Mixed with our Hawkes-filtered inputs, this creates a strong synergy:
When the market is quiet, the Hawkes filter drives the vitality dimensions towards zero. Mamba’s data-dependent Δ contracts, and the state transition matrix approaches identification. The community coasts, preserving its structural reminiscence with near-zero computational friction.
When a high-intensity burst arrives, the Hawkes filter passes the total vitality via. Δ expands, overriding the stale hidden state with the brand new micro-structural actuality.
The spine doesn’t should be taught when to replace and when to coast. The Hawkes filter handles the physics. Mamba handles the reminiscence. Collectively, they kind a notion engine that’s natively attuned to the intermittent, bursty dynamics of the restrict order e book.
Section A Stability Alignments
Monetary information is notoriously noisy, and naïve SSMs can endure from vanishing or exploding hidden states over lengthy rollouts. LOBMamba-Hawkes v3 implements three vital stability alignments drawn from the Mamba-3 paper’s ablation research:
BC Biases Initialized to Ones. The B and C projection biases are initialized to 1.0 fairly than the usual zero initialization. That is the one largest ablation part within the Mamba-3 paper — eradicating it prices +0.77 perplexity. Mixed with the 3-term recurrence’s implicit width-2 convolution, this eliminates the necessity for the brief causal Conv1d utilized in Mamba-1/2, whereas offering a robust baseline sign earlier than studying even begins:
# BC biases initialized to ONES (Mamba-3 Section A alignment)
self.B_bias = nn.Parameter(torch.ones(d_state))
self.C_bias = nn.Parameter(torch.ones(d_state))
A-Flooring Clamping. The continual A parameter — which controls state decay fee — is clamped to forestall it from approaching zero:
A = -torch.exp(self.A_log)
A = torch.clamp(A, max=-self.A_floor) # A_floor = 1e-4
With out this clamp, A can drift towards zero throughout coaching, inflicting near-infinite state retention and numerical instability. In monetary information, the place regime shifts could cause sudden distributional modifications, this instability is especially harmful.
Advanced-Valued States by way of RoPE
Rotary Positional Embeddings utilized to the B and C projections give the mannequin complex-valued state dynamics. This permits the SSM to characterize rotational and oscillating patterns — vital for monitoring periodic market buildings like intraday seasonality, public sale cycles, and amassed order circulate directionality. The Mamba-3 paper demonstrated that with out complicated states, the mannequin fails fully on elementary state-tracking duties (0.9% accuracy on parity checking, versus 100% with RoPE).
Element 3: Hawkes Landmark Consideration — Actual Recall With out O(L²) Price
Mamba excels at sustaining a compressed operating state. However compression essentially loses info. Over 1000’s of ticks, the mannequin could “neglect” a single excessive occasion from the distant previous — a large institutional print, a flash crash wick, a sudden liquidity vacuum — that basically altered the microstructure.
Transformers remedy this via full consideration over all previous tokens. However that brings again the O(L²) drawback.
LOBMamba-Hawkes introduces a 3rd path: Hawkes-gated sparse landmark consideration. Slightly than attending to all previous tokens, the community makes use of the Hawkes decay weights — the identical weights computed by the contagion filter — to establish the historic ticks that retained the very best structural vitality. It then performs a extremely environment friendly multi-head consideration cross over solely these landmarks.
# Choose top-k ticks by Hawkes weight as consideration landmarks
_, landmark_idx = torch.topk(hawkes_weights, okay, dim=1)# Collect landmark keys and values from skip connection
landmark_kv = torch.collect(hawkes_skip, 1, idx_expanded)
# Question = terminal Mamba state, KV = top-k Hawkes landmarks
Q = self.W_Q(mamba_output[:, -1:, :]) # Solely the present state queries
Okay = self.W_K(landmark_kv)
V = self.W_V(landmark_kv)
The fee is O(L · okay) the place okay is the variety of landmarks (default: 100) — dramatically cheaper than O(L²) whereas offering precise retrieval of probably the most structurally essential historic occasions.
This provides us the Hawkes filter’s twin goal: it’s concurrently a temporal noise filter on the enter and a landmark selector for sparse consideration. The contagion weights do double responsibility — architectural leverage that emerged naturally from treating the restrict order e book as a degree course of.
Gated Fusion
The terminal Mamba state and the eye output are mixed via a realized sigmoid gate:
gate = torch.sigmoid(self.fusion_gate(torch.cat([mamba_terminal, attn_out], dim=-1)))
fused = gate * mamba_terminal + (1 - gate) * attn_out
The gate bias is initialized to +2.0, producing sigmoid(2.0) ≈ 0.88 — closely favoring the Mamba state at coaching begin. This implies the eye pathway should earn its affect by offering info the SSM can not. Over coaching, the gate learns to extend the eye contribution exactly when the landmarks include structurally vital occasions that the compressed SSM state has forgotten.
The Proof: Unfold Bleed and the Step-Operate Coverage
Concept means nothing with out proof. We validated the LOBMamba-Hawkes structure in opposition to strict baselines inside a customized Tensorized Prop Agency Simulator — an atmosphere designed to copy the bodily friction {of professional} proprietary buying and selling.
The Unfold Bleed Drawback
The first failure mode of RL brokers in high-frequency environments is just not dangerous prediction. It’s over-execution.
A PPO agent outputs a steady motion a_t ∈ [−1, 1] representing directional conviction. In a frictionless simulation, the motion sequence [0.612, 0.614, 0.611] seems to be like a steady lengthy place. In an actual restrict order e book, every micro-adjustment requires crossing the unfold, paying fee, and consuming slippage. The agent bleeds to demise — not from being mistaken, however from being noisy.
We time period this Unfold Bleed: the continual accumulation of transaction prices pushed by continuous-action house jitter that overwhelms no matter directional alpha the agent really possesses.
Structure Ablation: Isolating the Hawkes Impact
We ran a proper ablation research throughout three architectures on 180 days of CME 6N (NZD/USD) futures — a intentionally difficult instrument with decrease liquidity and wider spreads than main pairs:
| Structure | Commerce Rely | Gross PnL | Frictional Drag | Internet PnL |
|--- |--- |--- |--- |--- |
| LOBTransformer | 5,102 | −$1,420 | −$33,850 | −$35,270 |
| Vanilla LOBMamba (no Hawkes) | 4,918 | +$2,657 | −$32,412 | −$29,755 |
| **LOBMamba-Hawkes** | **3,653** | **+$19,716** | **−$24,073** | **−$4,357** |
Desk: Structure Ablation Research (6N, 24-hour epoch). From Dickie (2026).
The outcomes inform a exact story.
The LOBTransformer failed on each fronts. It couldn’t course of enough context at sub-millisecond latency (the O(L²) wall), producing poor directional forecasts (destructive Gross PnL) compounded by extreme execution friction. Complete collapse: −$35,270.
The Vanilla LOBMamba solved prediction however not execution. With linear-time complexity, it efficiently modeled the order circulate, producing +$2,657 in gross directional alpha. However with out temporal filtering, the uncooked MBO stream vibrated with each micro-update, triggering fixed place changes. The agent’s alpha was totally consumed by friction. Internet outcome: −$29,755.
LOBMamba-Hawkes solved each. The Hawkes contagion filter basically modified the agent’s behavioral topology. Commerce frequency dropped 25.7% (from 4,918 to three,653 executions). Gross alpha elevated 7.4× (from +$2,657 to +$19,716). Frictional drag decreased by 85%. The agent’s conduct transitioned from steady noise-driven oscillation to a step-function of conviction — remaining dormant throughout low-contagion intervals and deploying capital solely when structural order circulate punched via the anti-jitter threshold.
At −$1.19 common price per commerce, the structure successfully isolates the agent’s structural alpha from microstructural noise. The system hovers close to breakeven after friction — a qualitative part transition from the catastrophic collapse of the baselines.
