Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Abstract—The window between zero-day vulnerability disclosure and weaponized exploitation has narrowed sharply in recent years, limiting the effectiveness of signature-based intrusion detection against novel payloads. Existing machine learning approaches either strip payload contents into coarse flow statistics or inherit the quadratic O(T 2) complexity of Transformer architectures on raw byte streams. This paper presents the Sovereign Byte Firewall, a tokenization-free network anomaly detector that models raw packet bytes as a next-byte prediction problem and converts per-window information-theoretic surprise into calibrated alerts using Extreme Value Theory (EVT) Peak-Over-Threshold bounds. I evaluate two backbones under a fully matched protocol — a 528K-parameter causal Transformer and a 301K-parameter Mamba-2 selective state space model — calibrated on the same benign file and the same 24 attack families, then scored on the same held-out split of 34,043 benign and 7,572 zero-day attack windows. At the EVT operating point the Transformer detects 15.0% of held-out zero-day windows at a 0.038% false-positive rate, while Mamba-2 detects 14.1% at 0.1%; the target-1% rule preserves this ordering. Replacing self-attention with a state space backbone therefore did not improve detection on this data: its advantage is architectural, delivering comparable detection at roughly 57% of the parameter count with linear-time O(T ) processing and constant-memory streaming. Two findings generalize beyond this system: threshold selection moves deployed behaviour more than backbone choice does, and EVT thresholds — fitted on the benign tail alone — are invariant to which attack families an operator happens to have labelled, while discriminative rules such as Youden are not. Limitations are stated explicitly, including modest absolute zero-day recall, blindness to encrypted payloads, and reliance on a single training corpus. Both evaluated checkpoints and the evaluation harness are released so that every figure reported here can be independently re-derived.