Continuous Sign Language Recognition (CSLR) is sensitive to how motion unfolds over time: short stroke-like segments often carry linguistic content, while transitional movements connect signs and can introduce boundary ambiguity. Standard CSLR systems typically learn these temporal regimes implicitly using a CNN backbone and a temporal module trained with Connectionist Temporal Classification (CTC), which can make decoding prone to deletion/insertion errors near boundaries. In this paper, we present an empirical study of CTC based CSLR on PHOENIX-2014, reporting Word Error Rate (WER) together with deletion/insertion breakdowns for standard backbone-temporal baselines. We additionally introduce a lightweight motion saliency diagnostic based on normalized frame differences to highlight candidate boundary-difficult intervals and support discussion/visual inspection. Finally, we describe a standard self distillation setup that stabilizes training without requiring extra annotations. The overall goal is to characterize boundary-related error patterns and motivate boundary-aware modeling directions for improving CSLR alignment robustness.
Paper
Full text
Motion Cues for Robust Continuous Sign Language Recognition: Baselines and Diagnostics
Semantic Scholar · 2026
Abstract
Continuous Sign Language Recognition (CSLR) is sensitive to how motion unfolds over time: short stroke-like segments often carry linguistic content, while transitional movements connect signs and can introduce boundary ambiguity. Standard CSLR systems typically learn these temporal regimes implicitly using a CNN backbone and a temporal module trained with Connectionist Temporal Classification (CTC), which can make decoding prone to deletion/insertion errors near boundaries. In this paper, we present an empirical study of CTC based CSLR on PHOENIX-2014, reporting Word Error Rate (WER) together with deletion/insertion breakdowns for standard backbone-temporal baselines. We additionally introduce a lightweight motion saliency diagnostic based on normalized frame differences to highlight candidate boundary-difficult intervals and support discussion/visual inspection. Finally, we describe a standard self distillation setup that stabilizes training without requiring extra annotations. The overall goal is to characterize boundary-related error patterns and motivate boundary-aware modeling directions for improving CSLR alignment robustness.