Search Results for author:
Found 15 papers, 9 papers with code
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings
The effect of the interlocutor is even more subtle, with submitted systems at best performing barely above chance.
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022
For each tier, we evaluated both the human-likeness of the gesture motion and its appropriateness for the specific speech signal.
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation
Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human-like motion; grounding the gesture in the co-occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications.
The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation
On the other hand, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings.
Multimodal analysis of the predictability of hand-gesture properties
Embodied conversational agents benefit from being able to accompany their speech with gestures.
Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech
We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures.
A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents
To date, recent end-to-end gesture generation methods have not been evaluated in a real-time interaction with users.
Generating coherent spontaneous speech and gesture from text
In contrast to previous approaches for joint speech-and-gesture generation, we generate full-body gestures from speech synthesis trained on recordings of spontaneous speech from the same person as the motion-capture data.
Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation
We provide an analysis of different representations for the input (speech) and the output (motion) of the network by both objective and subjective evaluations.
Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings
Our contributions are: a) a method for feature extraction from multi-party video and speech recordings, resulting in a representation that allows for independent control and manipulation of expression and speech articulation in a 3D avatar; b) an extension to MoGlow, a recent motion-synthesis method based on normalizing flows, to also take multi-modal signals from the interlocutor as input and subsequently output interlocutor-aware facial gestures; and c) a subjective evaluation assessing the use and relative importance of the input modalities.
Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows
In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters.
Gesticulator: A framework for semantically-aware speech-driven gesture generation
During speech, people spontaneously gesticulate, which plays a key role in conveying information.
Analyzing Input and Output Representations for Speech-Driven Gesture Generation
We evaluate different representation sizes in order to find the most effective dimensionality for the representation.
Gesture Generation
Human-Computer Interaction
I.2.6; I.5.1; J.4
A Neural Network Approach to Missing Marker Reconstruction in Human Motion Capture
Optical motion capture systems have become a widely used technology in various fields, such as augmented reality, robotics, movie production, etc.
Machine Learning and Social Robotics for Detecting Early Signs of Dementia
This paper presents the EACare project, an ambitious multi-disciplinary collaboration with the aim to develop an embodied system, capable of carrying out neuropsychological tests to detect early signs of dementia, e. g., due to Alzheimer's disease.
