A Real-Time Framework for Visual Feedback of Articulatory Data Using Statistical Shape Models

We present a novel open-source framework for visualizing electromagnetic articulography (EMA) data in real-time, with a modular framework and anatomically accurate tongue and palate models derived by multilinear subspace learning.

Computer Science Engineering Medicine

Source

Paper

References (10)

01A statistical shape space model of the palate surface trained on 3D MRI scans of the vocal tractAlexander Hewer, I. Steiner, Timo Bolkart et al.2015 · International Congress of Phonetic Sciences · 6 citations In Library

02Investigating the effects of posture and noise on speech productionI. Steiner, P. Knopp, Sebastian Musche et al.2014 · 6 citations

03An Audiovisual Talking Head for Augmented Speech Generation: Models and Animations Based on a Real Speaker's Articulatory DataP. Badin, F. Elisei, G. Bailly et al.2008 · Articulated Motion and Deformable Objects · 63 citations

04On the limited memory BFGS method for large scale optimizationDong C. Liu, J. Nocedal1989 · Mathematical programming · 8.9k citations

05Automatic animation of an articulatory tongue model from ultrasound images using Gaussian mixture regressionDiandra Fabre, Thomas Hueber, P. Badin2014 · Interspeech · 13 citations

06Opti-speech: a real-time, 3d visual feedback system for speech trainingW. Katz, T. Campbell, Jun Wang et al.2014 · Interspeech · 44 citations

07Ultrax: Real-time tongue tracking for speech therapy using ultrasound2014 · Ultrax: Real-time tongue tracking for speech therapy using ultrasound

08A biomechanical tongue model for speech production based on MRI live speaker data2011 · http://www.adambaker.org/qmu. php, 2011.

09“The Blender project.”http://blender.org/

10VulintusVulintus

Progress in animation of an EMA-controlled tongue model for acoustic-visual speech synthesisI. Steiner, Slim Ouni2012 · arXiv.org · 5 citations In Library

Predicting Tongue Positions from Acoustics and Facial FeaturesAsterios Toutios, Slim Ouni2011 · Interspeech · 4 citations In Library

Spatial and temporal alignment of multimodal human speech production data: Real time imaging, flesh point tracking and audioJangwon Kim, A. Lammert, P. Ghosh et al.2013 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 8 citations In Library

Contour-based 3d tongue motion visualization using ultrasound image sequencesKele Xu, Yin Yang, Clémence Leboullenger et al.2016 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 8 citations In Library

Interpretable Modeling of Articulatory Temporal Dynamics from Real-Time MRI for Phoneme RecognitionJay J. Park, Hong Anh Nguyen, Sean Foley et al.2026 In Library

Articulatory Representation Learning via Joint Factor Analysis and Neural Matrix FactorizationJiachen Lian, A. Black, Yijing Lu et al.2022 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 23 citations In Library

ARTI-6: Towards Six-dimensional Articulatory Speech EncodingJihwan Lee, Sean Foley, Thanathai Lertpetchpun et al.2025 · IEEE International Conference on Acoustics, Speech, and Signal Processing · 1 citation In Library

Arti-6: Towards six-dimensional Articulatory Speech EncodingJihwan Lee, Sean Foley, Thanathai Lertpetchpun et al.2026 In Library

ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory InversionH. Kim, Byungchan Hwang, H. Kim2026 · 0 citations In Library

Patents near this paper

Wireless Real-Time Tongue Tracking for Speech Impairment Diagnosis, Speech Therapy with Audiovisual Biofeedback, and Silent Speech InterfacesUS 9,911,358 · GEORGIA TECH RESEARCH CORPORATION · 2018 Simplified, Interactive, Real-Time Ultrasound Biofeedback System for Speech RemediationUS 11,688,294 · UNIVERSITY OF CINCINNATI · 2023 METHOD AND APPARATUS FOR PROVIDING VISUAL FEEDBACK OF SPEECH PRODUCTIONUS 6,728,680 · INTERNATIONAL BUSINESS MACHINES CORPORATION · 2004 METHOD FOR GENERATING REALISTIC FACIAL ANIMATION DIRECTLY FROM SPEECH UTILIZING HIDDEN MARKOV MODELSUS 6,735,566 · MITSUBISHI ELECTRIC INFORMATION TECHNOLOGY CENTER AMERICA, INC. · 2004 Method and System for Implementing Three-Dimensional Facial Modeling and Visual Speech SynthesisUS 11,145,100 · THE REGENTS OF THE UNIVERSITY OF COLORADO, A BODY CORPORATE · 2021

Source of record

arXiv

A Real-Time Framework for Visual Feedback of Articulatory Data Using Statistical Shape Models

Paper

References (10)

Similar papers

Patents near this paper

Welcome.

Patents near this paper