Research and Development

Articulatory Synthesis and Recognition

Articulatory Synthesis

Progress in Speech Synthesis edited by Jan P. H. van Santen, Richard Sproat, Joseph Olive and Julia Hirschberg, A Review of Articulatory Speech Synthesis by Pertti Palo, Investigations in Articulatory Synthesis by Athanassios Katsamanis, Pirros Tsiakoulis, Petros Maragos and Alexandros Potamianos, Literature Review on Articulatory Models by Pirros Tsiakoulis, Biomechanics of Speech Sounds by Du Brul and E. Lloyd, Laryngeal Biomechanics by B. Raymond Fink, Principles of Voice Production by Ingo R. Titze, Muscle and Tendon: Properties, Models, Scaling, and Application to Biomechanics and Motor Control by Felix E. Zajac, Computations Underlying the Execution of Movement: A Biological Perspective by Emilio Bizzi, Ferdinando A. Mussa-Ivaldi and Simon Giszter, Training an Articulatory Synthesizer with Continuous Acoustic Data by Santitham Prom-on, Peter Birkholz and Yi Xu, Training a Vocal Tract Synthesizer to Imitate Speech using Distal Supervised Learning by Ian S. Howard and Mark A. Huckvale, Data Driven Articulatory Synthesis with Deep Neural Networks by Sandesh Aryal and Ricardo Gutierrez-Osuna, Simulation of Flow and Acoustics in the Vocal Tract by Peter Birkholz and Dietmar Jackel, A Contribution to Simulating a Three-dimensional Larynx Model Using the Finite Element Method by Rosa de Oliveira, Marcelo José Carlos Pereira, Marcos Grellet and Abeer Alwan, A Finite-element Model of Vocal-fold Vibration by Fariborz Alipour, David A. Berry and Ingo R. Titze, Simulation of Vocal Fold Impact Pressures with a Self-oscillating Finite-element Model by Chao Tao, Jack J. Jiang and Yu Zhang, Aerodynamic Study of Three-dimensional Larynx Models Using Finite Element Methods by Rosa de Oliveira, Marcelo and José Carlos Pereira, A Theoretical Study of the Effects of Various Laryngeal Configurations on the Acoustics of Phonation by Ingo R. Titze and David T. Talkin and Towards Articulatory Speech Synthesis with a Dynamic 3D Finite Element Tongue Model by Kees van den Doel, Florian Vogt, R. Elliot English and Sidney Fels.


Mechanics of Respiration by Wallace O. Fenn, Physical Properties of Human Lungs Measured during Spontaneous Respiration by Jere Mead and James L. Whittenberger, The Respiratory Muscles: Mechanics and Neural Control by Edward J. M. Campbell, Emilio Agostoni and J. Newsom Davis, The Relationship between Prosody and Breathing in Spontaneous Discourse by Kathryn Hird and Kim Kirsner, The Effects of Breath Sounds on the Perception of Synthetic Speech by D. H. Whalen, Charles E. Hoequist and Sonya M. Sheffert, Temporal Patterns of Cognitive Activity and Breath Control in Speech by Alan Henderson, Frieda Goldman-Eisler and Andrew Skarbek, Cognitive-linguistic Demands and Speech Breathing by Heather L. Mitchell, Jeannette D. Hoit and Peter J. Watson, Breathing Patterns during Spontaneous Speech by Alison L. Winkworth, Pamela J. Davis, Roger D. Adams and Elizabeth Ellis, Pauses and Respiratory Markers of the Structure of Book Reading by Gérard Bailly and Cécilia Gouvernayre, Adaptation of Respiratory Patterns in Collaborative Reading by Gérard Bailly, Amélie Rochet-Capellan and Coriandre Vilain, Respiratory Markers of Conversational Interaction by David H. McFarland, Audible Aspects of Speech Preparation by James M. Scobbie, Sonja Schaeffler and Ineke Mennen and Measuring Reaction Times: Vocalisation vs. Articulation by Sonja Schaeffler, James M. Scobbie and Felix Schaeffler.

Facial Biomechanics

A Model of Facial Biomechanics for Speech Production by Jorge C. Lucero and Kevin G. Munhall, Quantitative Association of Vocal-tract and Facial Behavior by Hani Yehia, Philip Rubin and Eric Vatikiotis-Bateson, See Me, Hear Me: Integrating Automatic Speech Recognition and Lip-reading by Paul Duchnowski, Uwe Meier and Alex Waibel, Toward Movement-invariant Automatic Lip-reading and Speech Recognition by Paul Duchnowski, Martin Hunke, Dietrich Büsching, Uwe Meier and Alex Waibel, Hearing by Eye: The Psychology of Lip-reading edited by Barbara Dodd and Ruth Campbell, Computer Lipreading for Improved Accuracy in Automatic Speech Recognition by Peter L. Silsbee and Alan C. Bovik, Resynthesis of 3D Tongue Movements from Facial Data by Olov Engwall and Jonas Beskow, On the Correlation between Facial Movements, Tongue Movements and Speech Acoustics by Jintao Jiang, Abeer Alwan, Lynne E. Bernstein, Patricia Keating and Ed Auer and Tracking of Multiple Faces for Human-computer Interfaces and Virtual Environments by Fu Jie Huang and Tsuhan Chen.

Motor Cognition

Learning to Speak: Sensori-motor Control of Speech Movements by Gérard Bailly, Sensory-motor Control of Speech Movements by Gérard Bailly, Simulation of Action as a Unifying Concept for Motor Cognition by Marc Jeannerod and S. H. Johnson-Frey and Toward Understanding Speech Planning by Observing its Execution – Representations, Modeling and Analysis by Vikram Ramanarayanan.


Construction and Control of a Three-dimensional Vocal Tract Model by Peter Birkholz, Dietmar Jackèl and Bernd J. Kröger, Control Concepts for Articulatory Speech Synthesis by Peter Birkholz, Ingmar M. A. Steiner and Stefan Breuer, Control of an Articulatory Speech Synthesizer Based on Dynamic Approximation of Spatial Articulatory Targets by Peter Birkholz, Observations on the Dynamic Control of an Articulatory Synthesizer Using Speech Production Data by Ingmar M. A. Steiner and The Organization of a Neurocomputational Control Model for Articulatory Speech Synthesis by Bernd J. Kröger, Anja Lowit and Ralph Schnitker.

Articulatory Recognition

Adaptive Computation of Articulatory Parameters from the Speech Signal by S. E. Levinson and C. E. Schmidt, Estimating Articulatory Motion from Speech Wave by Katsuhiko Shirai and Tetsunori Kobayashi, Estimating Articulatory Parameters from the Acoustic Speech Signal by Korin Richmond, Estimation of Articulatory Parameters from the Acoustic Speech Signal by Alexander Sepulveda, Estimation of Articulatory Movements from Speech Acoustics Using an HMM-based Speech Production Model by Sadao Hiroya and Masaaki Honda, Formant Trajectories for Acoustic-to-articulatory Inversion by İ. Yücel Özbek, Mark Hasegawa-Johnson and Mübeccel Demirekler, From Acoustics to Vocal Tract Time Functions by Vikramjit Mitra, İ. Yücel Özbek, Hosung Nam, Xinhui Zhou and Carol Y. Espy-Wilson, Acoustic-to-articulatory Inversion: Methods and Acquisition of Articulatory Data by Shinji Maeda, Marie-Odile Berger, Olov Engwall, Yves Laprie, Petros Maragos, Blaise Potard and Jean Schoentgen, An Analysis-by-synthesis Approach to Vocal Tract Modeling for Robust Speech Recognition by Ziad A. Al Bawab, Speech Production Knowledge in Automatic Speech Recognition by Simon King, Joe Frankel, Karen Livescu, Erik McDermott, Korin Richmond and Mirjam Wester, Adding Articulatory Features to Acoustic Features for Automatic Speech Recognition by Igor Zlokarnik, A Subject-independent Acoustic-to-articulatory Inversion by Prasanta K. Ghosh and Shrikanth S. Narayanan, Improved Subject-independent Acoustic-to-articulatory Inversion by Amber Afshan and Prasanta K. Ghosh, Modelling the Uncertainty in Recovering Articulation from Acoustics by Korin Richmond, Simon King and Paul Taylor, Noise Robustness of Tract Variables and Their Application to Speech Recognition by Vikramjit Mitra, Hosung Nam, Carol Y. Espy-Wilson, Elliot Saltzman and Louis Goldstein, Articulatory Information for Noise Robust Speech Recognition by Vikramjit Mitra, Hosung Nam, Carol Y. Espy-Wilson, Elliot Saltzman and Louis Goldstein, Statistical Mapping between Articulatory Movements and Acoustic Spectrum Using a Gaussian Mixture Model by Tomoki Toda, Alan W. Black and Keiichi Tokuda, An Empirical Investigation of the Nonuniqueness in the Acoustic-to-articulatory Mapping by Chao Qin and Á. Miguel, A Generalized Smoothness Criterion for Acoustic-to-articulatory Inversion by Prasanta K. Ghosh and Shrikanth Narayanan, A Comparison of Acoustic Features for Articulatory Inversion by Chao Qin and Á. Miguel, Information Theoretic Acoustic Feature Selection for Acoustic-to-articulatory Inversion by Prasanta K. Ghosh and Shrikanth Narayanan, Relevance-weighted-reconstruction of Articulatory Features in Deep-neural-network-based Acoustic-to-articulatory Mapping by Claudia Canevari, Leonardo Badino, Luciano Fadiga and Giorgio Metta, A Deep Neural Network for Acoustic-articulatory Speech Inversion by Benigno Uria, Steve Renals and Korin Richmond, Deep Architectures for Articulatory Inversion by Benigno Uria, Iain Murray, Steve Renals and Korin Richmond, Retrieving Tract Variables from Acoustics: A Comparison of Different Machine Learning Strategies by Vikramjit Mitra, Hosung Nam, Carol Y. Espy-Wilson, Elliot Saltzman and Louis Goldstein, Introducing Visual Cues in Acoustic-to-articulatory Inversion by Olov Engwall, Analysis, Synthesis, and Perception of Visible Articulatory Movements by N. M. Brooke and Quentin Summerfield and Audiovisual-to-articulatory Inversion by Hedvig Kjellström and Olov Engwall.


Speaker Verification Based on Fusion of Acoustic and Articulatory Information by Ming Li, Jangwon Kim, Prasanta K. Ghosh, Vikram Ramanarayanan and Shrikanth Narayanan, Speaker Verification Based on the Fusion of Speech Acoustics and Inverted Articulatory Signals by Ming Li, Jangwon Kim, Adam Lammert, Prasanta K. Ghosh, Vikram Ramanarayanan and Shrikanth Narayanan, Use of Language as a Cognitive Biometric Trait by Neeti Pokhriyal, Ifeoma Nwogu and Vengatesan Govindaraju and Multimodal Cognitive Biometrics by Lurdes Faria, Vitor J. Sa and Sérgio Tenreiro de Magalhaes.

Gesture Recognition

Gesture Recognition: A Survey by Sushmita Mitra and Tinku Acharya, Continuous Realtime Gesture Following and Recognition by Frédéric Bevilacqua, Bruno Zamborlin, Anthony Sypniewski, Norbert Schnell, Fabrice Guédy and Nicolas Rasamimanana, A Procedure for Estimating Gestural Scores from Natural Speech by Hosung Nam, Vikramjit Mitra, Mark Tiede, Elliot Saltzman, Louis Goldstein, Carol Y Espy-Wilson and Mark Hasegawa-Johnson, Mapping between Acoustic and Articulatory Gestures by Gopal Ananthakrishnan and Olov Engwall and Recognizing Articulatory Gestures from Speech for Robust Speech Recognition by Vikramjit Mitra, Hosung Nam, Carol Espy-Wilson, Elliot Saltzman and Louis Goldstein.


The Importance of Studying Prosody in the Comprehension of Spontaneous Spoken Discourse by Jazmín Cevasco and Fernando Marmolejo Ramos, The Prosodic Investigation of Information Structure by Aoju Chen, Information, Prosody and Modeling – With Emphasis on Tonal Features of Speech by Hiroya Fujisaki, Using Prosodic and Voice Quality Features for Paralinguistic Information Extraction by C. Ishi, Hiroshi Ishiguro and Norihiro Hagita, Paralinguistics in Speech and Language – State-of-the-art and the Challenge by Björn Schuller, Stefan Steidl, Anton Batliner, Felix Burkhardt, Laurence Devillers, Christian Müller and Shrikanth Narayanan, Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing by Björn Schuller and Anton Batliner, Ten Recent Trends in Computational Paralinguistics by Björn Schuller and Felix Weninger, Universal and Language-specific Perception of Paralinguistic Intonational Meaning by Aoju Chen, Language-specificity in the Perception of Paralinguistic Intonational Meaning by Aoju Chen, Carlos Gussenhoven and Toni Rietveld, Speech Prosody: A Methodological Review by Yi Xu, Prosodic Phrasing is Central to Language Comprehension by Lyn Frazier, Katy Carlson and Charles Clifton, Expressive Speech Synthesis: A Review by D. Govind and S. R. Mahadeva Prasanna, Experimental and Theoretical Advances in Prosody: A Review by Michael Wagner and Duane G. Watson, Sentence Prosody: Intonation, Stress, and Phrasing by Elisabeth Selkirk, Co-training Succeeds in Computational Paralinguistics by Zixing Zhang, Jun Deng and Bjorn Schuller, The Prosody of Focus: A Case-study with Cross-linguistic Implications by Sónia Frota, Topic Change Detection Based on Prosodic Cues in Unimodal Setting by G. Esfandiari Baiat and I. Szekrenyes, Evaluating Prosodic Cues as a Means to Disambiguate Algebraic Expressions: An Empirical Study by Ed Gellenbeck and Andreas Stefik and Multi-modal Mathematics: Conveying Math using Synthetic Speech and Speech Recognition by Donal Fitzpatrick and Arthur I. Karshmer.


Toward the Simulation of Emotion in Synthetic Speech: A Review of the Literature on Human Vocal Emotion by Iain R. Murray and John L. Arnott, Vocal Affect Expression: A Review and a Model for Future Research by Klaus R. Scherer, Voice and Emotional States by Gardruir Klesmeyar, Emotions and Speech: Some Acoustical Correlates by Carl E. Williams and Kenneth N. Stevens, Determinism in Speech Pitch Relation to Emotion by Ahmed M. Mahmoud and Wan H. Hassan, Communicating Emotion: The Role of Prosodic Features by Robert W. Frick, Voice Stress Analysis by Leon J. M. Rothkrantz, Pascal Wiggers, Jan-Willem A. van Wees and Robert J. van Vark, The Effects of Different Levels of Task Complexity on Three Vocal Measures by G. R. Griffin and C. E. Williams, Speech Under Stress: Analysis, Modeling and Recognition by John H. L. Hansen and Sanjay Patil, Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications by Rafael Calvo and Sidney D’Mello, Automatic Detection of Learner’s Affect from Conversational Cues by Sidney K. D’Mello, Scotty D. Craig, Amy Witherspoon, Bethany Mcdaniel and Arthur Graesser, Multimodal Semi-automated Affect Detection from Conversational Cues, Gross Body Language, and Facial Features by Sidney K. D’Mello and Arthur Graesser and Toward a Machine Learning Framework for Understanding Affective Tutorial Interaction by Joseph F. Grafsgaard, Kristy E. Boyer and James C. Lester.


Speaking Style Variation and Speaker Personality by Akiko Mokhtari and Nick Campbell, From Speech to Personality: Mapping Voice Quality and Intonation into Personality Differences by Gelareh Mohammadi, Antonio Origlia, Maurizio Filippone and Alessandro Vinciarelli, On Speaker-independent Personality Perception and Prediction from Speech by Tim Polzehl, Katrin Schoenenberg, Sebastian Moller, Florian Metze, Gelareh Mohammadi and Alessandro Vinciarelli, Automatic Personality Perception: Prediction of Trait Attribution Based on Prosodic Features by Gelareh Mohammadi and Alessandro Vinciarelli and Voice and Identity: A Contrastive Study of Identity Perception in Voice by Begoña P. Herrero.


Expressive Speech: Production, Perception and Application to Speech Synthesis by Donna Erickson, Improving Voice and Articulation by Hilda B. Fisher, The Professional Voice by Robert T. Sataloff, The Use and Training of the Human Voice: A Practical Approach to Speech and Voice Dynamics by Arthur Lessac, Training the Speaking Voice by Virgil A. Anderson, Developing Vocal Skills by Theodore D. Hanley and Wayne L. Thurman, Training Actors’ Voices by Tara McAllister-Viel, Voice and the Actor by Cicely Berry, Voice Modifications of Stage Actors: Acoustic Analyses by Bonnie N. Raphael and Ronald C. Scherer, Voice in Modern Theatre by Jacqueline Martin, Vocal Direction for the Theatre: From Script Analysis to Opening Night by Nan Withers-Wilson, Voice and Speech in the Theatre by James C. Turner and Malcolm Morrison, Voice and Identity: A Contrastive Study of Identity Perception in Voice by Begoña P. Herrero, The Science of Singing Voice by Johan Sundberg and Thomas D. Rossing, Dynamics of the Singing Voice by Meribeth Bunch, The Structure of Singing: System and Art in Vocal Technique by Richard Miller, Synthesis and Processing of the Singing Voice by Xavier Rodet, Articulatory Synthesis of Singing by Peter Birkholz and Articulatory Synthesis of Speech and Singing: State of the Art and Suggestions for Future Research by Bernd J. Kröger and Peter Birkholz.