Search Results for author:
Found 49 papers, 8 papers with code
C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model
Co-speech gesture generation is crucial for automatic digital avatar animation.
Argument Attribution Explanations in Quantitative Bipolar Argumentation Frameworks (Technical Report)
Argumentative explainable AI has been advocated by several in recent years, with an increasing interest on explaining the reasoning outcomes of Argumentation Frameworks (AFs).
Efficient STL Control Synthesis under Asynchronous Temporal Robustness Constraints
Given an ATR bound, we compute a sequence of control inputs so that the specification is satisfied by the system as long as each sub-trajectory is shifted not more than the ATR bound.
Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts
In this paper, we introduce Mega-TTS 2, a generic zero-shot multispeaker TTS model that is capable of synthesizing speech for unseen speakers with arbitrary-length prompts.
GenerTTS: Pronunciation Disentanglement for Timbre and Style Generalization in Cross-Lingual Text-to-Speech
Cross-lingual timbre and style generalizable text-to-speech (TTS) aims to synthesize speech with a specific reference timbre or style that is never trained in the target language.
Towards Building Voice-based Conversational Recommender Systems: Datasets, Potential Solutions, and Prospects
Conversational recommender systems (CRSs) have become crucial emerging research topics in the field of RSs, thanks to their natural advantages of explicitly acquiring user preferences via interactive conversations and revealing the reasons behind recommendations.
Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis
We are interested in a novel task, namely low-resource text-to-talking avatar.
Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
3) We further use a VQGAN-based acoustic model to generate the spectrogram and a latent code language model to fit the distribution of prosody, since prosody changes quickly over time in a sentence, and language models can capture both local and long-range dependencies.
Detector Guidance for Multi-Object Text-to-Image Generation
Previous works identify the problem of information mixing in the CLIP text encoder and introduce the T5 text encoder or incorporate strong prior knowledge to assist with the alignment.
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation
Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data.
Ranked #4 on
Audio Generation
on AudioCaps
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation
Direct speech-to-speech translation (S2ST) aims to convert speech from one language into another, and has demonstrated significant progress to date.
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation
Recently, neural radiance field (NeRF) has become a popular rendering technique in this field since it could achieve high-fidelity and 3D-consistent talking face generation with a few-minute-long training video.
Data-Driven Safe Controller Synthesis for Deterministic Systems: A Posteriori Method With Validation Tests
We first formulate the safety synthesis problem as a robust convex program (RCP) based on notion of control barrier function.
LiteG2P: A fast, light and high accuracy model for grapheme-to-phoneme conversion
As a key component of automated speech recognition (ASR) and the front-end in text-to-speech (TTS), grapheme-to-phoneme (G2P) plays the role of converting letters to their corresponding pronunciations.
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models
Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data.
Ranked #7 on
Audio Generation
on AudioCaps
Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features
Speech-to-speech translation directly translates a speech utterance to another between different languages, and has great potential in tasks such as simultaneous interpretation.
Markov decision processes with maximum entropy rate for Surveillance Tasks
We consider the problem of synthesizing optimal policies for Markov decision processes (MDP) for both utility objective and security constraint.
You Don't Know When I Will Arrive: Unpredictable Controller Synthesis for Temporal Logic Tasks
This problem is particularly challenging since future information is involved in the synthesis process.
Explaining Random Forests using Bipolar Argumentation and Markov Networks (Technical Report)
Random forests are decision tree ensembles that can be used to solve a variety of machine learning problems.
Model Predictive Control for Signal Temporal Logic Specifications with Time Interval Decomposition
However, instead of applying MPC directly for the entire task horizon, we decompose the STL formula into several sub-formulae with disjoint time horizons, and shrinking horizon MPC is applied for each short-horizon sub-formula iteratively.
Abstraction-Based Verification of Approximate Pre-Opacity for Control Systems
In this paper, we first introduce a concept of approximate pre-opacity by capturing the security level of control systems with respect to the measurement precision of the intruder.
Model Predictive Monitoring of Dynamic Systems for Signal Temporal Logic Specifications
To this end, effective approaches for the computation of feasible sets of STL formulae are provided.
Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective
Specifically, we consider the generation of cross-domain videos from two sets of latent factors, one encoding the static information and another encoding the dynamic information.
A Novel Chinese Dialect TTS Frontend with Non-Autoregressive Neural Machine Translation
In this paper, we propose a novel Chinese dialect TTS frontend with a translation module, which converts Mandarin text into dialectic expressions to improve the intelligibility and naturalness of synthesized speech.
Towards a Theory of Faithfulness: Faithful Explanations of Differentiable Classifiers over Continuous Data
There is broad agreement in the literature that explanation methods should be faithful to the model that they explain, but faithfulness remains a rather vague term.
A Unified Framework for Verification of Observational Properties for Partially-Observed Discrete-Event Systems
However, in contrast to existing results, where different verification procedures are developed for different properties case-by-case, in this work, we provide a unified framework for verifying all these properties by reducing each of them as an instance of HyperLTL model checking.
A Uniform Framework for Diagnosis of Discrete-Event Systems with Unreliable Sensors using Linear Temporal Logic
In this work, we propose a novel \emph{uniform framework} for diagnosability of DES subject to, not only sensor failures, but also a very general class of unreliable sensors.
Fault Diagnosis of Discrete-Event Systems under Non-Deterministic Observations with Output Fairness
Non-deterministic observation is a general observation model that includes the case of intermittent loss of observations.
To Explore or Not to Explore: Regret-Based LTL Planning in Partially-Known Environments
In contrast to the standard game-based approach that optimizes the worst-case cost, in the paper, we propose to use regret as a new metric for planning in such a partially-known environment.
You Don't Know What I Know: On Notion of High-Order Opacity in Discrete-Event Systems
In this paper, we consider a new type of secret related to the knowledge of the system user.
Sensor Deception Attacks Against Initial-State Privacy in Supervisory Control Systems
Specifically, we consider an active attacker that can tamper with the observations received by the supervisor by, e. g., hacking on the communication channel between the sensors and the supervisor.
Online Monitoring of Dynamic Systems for Signal Temporal Logic Specifications with Model Information
We show that, by explicitly utilizing the model information of the dynamic system, the proposed online monitoring algorithm can falsify or certify of the specification in advance compared with existing algorithms, where no model information is used.
Towards Realistic Visual Dubbing with Heterogeneous Sources
The task of few-shot visual dubbing focuses on synchronizing the lip movements with arbitrary speech input for any talking head video.
Data privacy protection in microscopic image analysis for material data mining
The core contributions are as follows: 1) the federated learning algorithm is introduced into the polycrystalline microstructure image segmentation task to make full use of different user data to carry out machine learning, break the data island and improve the model generalization ability under the condition of ensuring the privacy and security of user data; 2) A data sharing strategy based on style transfer is proposed.
Towards Using Clothes Style Transfer for Scenario-aware Person Video Generation
To tackle this problem, most recent AdaIN-based architectures are proposed to extract clothes and scenario features for generation.
Towards High-fidelity Singing Voice Conversion with Acoustic Reference and Contrastive Predictive Coding
Experiments show that, compared with the baseline models, our proposed model can significantly improve the naturalness of converted singing voices and the similarity with the target singer.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech
In expressive speech synthesis, there are high requirements for emotion interpretation.
Personalized Heterogeneous Federated Learning with Gradient Similarity
In SPFL, the server uses the Softmax Normalized Gradient Similarity (SNGS) to weight the relationship between clients, and sends the personalized global model to each client.
Cloud-Assisted Nonlinear Model Predictive Control for Finite-Duration Tasks
In this paper, we propose a novel cloud-assisted model predictive control (MPC) framework in which we systematically fuse a cloud MPC that uses a high-fidelity nonlinear model but is subject to communication delays with a local MPC that exploits simplified dynamics (due to limited computation) but has timely feedback.
Optimal Synthesis of Opacity-Enforcing Supervisors for Qualitative and Quantitative Specifications
We assume that the system has a "secret" that does not want to be revealed to the intruder.
A Framework for Current-State Opacity under Dynamic Information Release Mechanism
In this paper, we investigate the verification of current-state opacity for discrete-event systems under Orwellian-type observations, i. e., the system is allowed to re-interpret the observation of an event based on its future suffix.
PPG-based singing voice conversion with adversarial representation learning
Singing voice conversion (SVC) aims to convert the voice of one singer to that of other singers while keeping the singing content and melody.
Opacity Enforcing Supervisory Control using Non-deterministic Supervisors
Compared with the standard deterministic control mechanism, such a non-deterministic control mechanism can enhance the plausible deniability of the controlled system as the online control decision is a random realization and cannot be implicitly inferred from the control policy.
End-to-End Learning for Simultaneously Generating Decision Map and Multi-Focus Image Fusion Result
In this work, to handle the requirements of both output image quality and comprehensive simplicity of structure implementation, we propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure.
Xiaomingbot: A Multilingual Robot News Reporter
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation.
Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech
Accent conversion (AC) transforms a non-native speaker's accent into a native accent while maintaining the speaker's voice timbre.
ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders
This paper presents ByteSing, a Chinese singing voice synthesis (SVS) system based on duration allocated Tacotron-like acoustic models and WaveRNN neural vocoders.
A hybrid text normalization system using multi-head self-attention for mandarin
In this paper, we propose a hybrid text normalization system using multi-head self-attention.
A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis
In Mandarin text-to-speech (TTS) system, the front-end text processing module significantly influences the intelligibility and naturalness of synthesized speech.
