Proprioceptive AI: Self-Compounding Behavioral Probes for Autonomous Model Improvement and Probe-Guided Compression

This paper presents Proprioceptive AI, a three-timescale framework for autonomous behavioral measurement, correction, and self-improvement in large language models. At the fast timescale (per-token), lightweight binary classifiers (~1M parameters) operate on hidden state representations extracted at intermediate transformer layers to produce real-time behavioral scores across 20 dimensions (15 suppression, 5 amplification). At the medium timescale (per-session), a self-compounding active learning loop converts model outputs into labeled training data: probe scores above/below confidence thresholds yield automatic labels, while uncertain examples (scores 0.3–0.7) are mined as hard cases and prioritized by proximity to the decision boundary for maximum marginal information. Anchor data preservation (α=0.5 seed weight in the retraining loss) maintains calibration against the original external reference standard, providing a formal non-circularity guarantee absent from pure self-training approaches. At the slow timescale (per-epoch), retrained probes yield sharper decision boundaries, producing more informative hard cases in subsequent cycles. We address the bootstrap problem through contrastive pair generation: an external reference model produces behaviorally degraded variants of 3,582 bilingual paragraphs across 15 dimensions ($19.30 via Claude Haiku) and shallow/deep capability variants across 5 dimensions ($19.30 via Claude Sonnet 4.5), establishing an externally-grounded measurement instrument at one-time cost. Trained probes achieve separation ratios of 125–1,376× across architectures (LLaMA, Qwen, Mistral), with three-layer hidden state concatenation (L/4, L/2, 3L/4) and token-averaged pooling. The framework extends bidirectionally: behavioral suppression probes push hidden states away from pathological subspaces via negatively-weighted LoRA adapter signals, while capability amplification probes (reasoning depth, factual precision, conceptual synthesis, inferential novelty, calibrated confidence) push hidden states toward quality subspaces via positively-weighted signals, with per-probe weights independently tunable at deployment time. We further introduce probe-guided model compression, exploiting the observation that behavioral information concentrates in low-dimensional subspaces (16D fiber projections at 128:1–256:1 compression). Dimensional importance scores derived from probe weight matrix column norms (I(i) = max_j ||W_j[:,i]||₂) provide a semantically grounded criterion for structured pruning, heterogeneous quantization (FP16/INT8/INT4 allocation by importance tier), and targeted distillation where the student matches the teacher only on probe-verified important dimensions. Projected compression ratios of 5–20× preserve ≥95% of behavioral probe scores. Cross-language transfer experiments test the hypothesis that behavioral encoding occupies language-invariant geometric subspaces in multilingual hidden state spaces. The complete pipeline is reproducible on a single RTX 3090 for under $60 in external API costs. Full prompts (15 behavioral × 2 languages + 5 capability × 2 polarities), JSONL schemas, training configurations, and pipeline pseudocode are provided as appendices.