Search Results for author:
Found 13 papers, 6 papers with code
Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents
This paper focuses on enhancing human-agent communication by integrating spatial context into virtual agents' non-verbal behaviors, specifically gestures.
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis
Specifically, we use unimodal synthesis models trained on large datasets to create multimodal (but synthetic) parallel training data, and then pre-train a joint synthesis model on that material.
Unified speech and gesture synthesis using flow matching
As text-to-speech technologies achieve remarkable naturalness in read-aloud tasks, there is growing interest in multimodal synthesis of verbal and non-verbal communicative behaviour, such as spontaneous speech and associated body gestures.
Ranked #1 on
Motion Synthesis
on Trinity Speech-Gesture Dataset
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation
The output of the CSMP module is used as a conditioning signal in the diffusion-based gesture synthesis model in order to achieve semantically-aware co-speech gesture generation.
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis
With read-aloud speech synthesis achieving high naturalness scores, there is a growing research interest in synthesising spontaneous speech.
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Diffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models.
Integrated Speech and Gesture Synthesis
Text-to-speech and co-speech gesture synthesis have until now been treated as separate areas by two different research communities, and applications merely stack the two technologies using a simple system-level pipeline.
Transflower: probabilistic autoregressive dance generation with multimodal attention
First, we present a novel probabilistic autoregressive architecture that models the distribution over future poses with a normalizing flow conditioned on previous poses as well as music context, using a multimodal transformer encoder.
Generating coherent spontaneous speech and gesture from text
In contrast to previous approaches for joint speech-and-gesture generation, we generate full-body gestures from speech synthesis trained on recordings of spontaneous speech from the same person as the motion-capture data.
Robust model training and generalisation with Studentising flows
Normalising flows are tractable probabilistic models that leverage the power of deep learning to describe a wide parametric family of distributions, all while remaining trainable using maximum likelihood.
Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows
In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters.
Gesticulator: A framework for semantically-aware speech-driven gesture generation
During speech, people spontaneously gesticulate, which plays a key role in conveying information.
MoGlow: Probabilistic and controllable motion synthesis using normalising flows
Data-driven modelling and synthesis of motion is an active research area with applications that include animation, games, and social robotics.
