Strange Compass Transformers: Geometric Navigation in Semantic Embedding Space

Current approaches to semantic relational computation face two limits: autoregressive models pay sequential generation cost for every transformation, while similarity search can only reach nearby concepts, with no grammar for arbitrary relational navigation. Neither offers precise, interpretable, repeatable operations on meaning.We propose Strange Compass Transformers (SCTs), a non-autoregressive architecture that performs explicit geometric transformations in semantic embedding space through learned 2D rotation planes. Rather than generating text, an SCT acts as a semantic actuator: given a concept and a parameterized query, angle (0°-360°), magnitude (0–1), and context, it returns a transformed embedding by projecting into a per-context plane, rotating, and lifting back.The core innovation is ontological, inspired by Hofstadter and Sander's Surfaces and Essences and its thesis that analogy is the core of cognition. SCT takes its "strange" namesake from the Hofstadterian strange loop and exists to test, empirically, whether integrating analogical navigation as a first-class primitive is useful in ML systems. The compass metaphor and the multi-plane design are the author's contributions, addressing polysemy and the distinctive training challenges of such a model.At the center is a derived compass, not a stipulated one. Angular semantics fall out of the cosine–sine landmark structure of the plane: identity at 0°, opposition at 180°, positive and negative orthogonality at 90°/270°, and graded analogy occupying the intercardinal cone, landmarked at 45°, with the full eight-direction structure (identity, analogy, opposed analogy, anti-analogy, reverse analogy, and the two orthogonalities) forced by three coordinates: grade, polarity, and ontological handedness. Motion from a concept toward a queried relation follows an endpoint-pinned transit law: as magnitude increases, the trajectory withdraws toward the neutral origin before re-emerging at the target, traversing the disc's interior at low commitment rather than dragging meaning through unqueried relational neighborhoods, the geometric expression of a single claim, to reach what a concept is, you must first release what it is not.Instrumentally, SCT projects high-dimensional embeddings onto learnable 2D manifolds where clean rotations occur, then lifts back. Shadow Plane Adapters, per-context learned projections, absorb the mismatch between pretrained embedding structure and rotation semantics, isolating the polysemy that arises when a word lives in many contexts. The architecture targets semantic navigation at ~131M parameters (120k vocabulary, trainable on consumer GPUs) through an 8-head compass (4 cardinal + 4 intercardinal) emitting continuous proximity signals, with involution, endpoint-pinned transit, and monotone magnitude response as guaranteed properties.By externalizing analogical structure as a queryable geometric primitive, SCT provides reusable relational infrastructure for downstream reasoning systems, amortizing the cost of learning semantic relationships across tasks.