Search Results for author:
Found 132 papers, 52 papers with code
Simple and Effective Graph-to-Graph Annotation Conversion
Annotation conversion is an effective way to construct datasets under new annotation guidelines based on existing datasets with little human labour.
Sounding that Object: Interactive Object-Aware Image to Audio Generation
Generating accurate sounds for complex audio-visual scenes is challenging, especially in the presence of multiple objects and sound sources.
SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
Foundation models have achieved transformative success across biomedical domains by enabling holistic understanding of multimodal data.
MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation
However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens.
Discrete Markov Bridge
To address these limitations, we propose Discrete Markov Bridge, a novel framework specifically designed for discrete representation learning.
Towards Reliable Large Audio Language Model
Recent advancements in large audio language models (LALMs) have demonstrated impressive results and promising prospects in universal understanding and reasoning across speech, music, and general sound.
CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training
Despite these advancements, CosyVoice 2 exhibits limitations in language coverage, domain diversity, data volume, text formats, and post-training techniques.
AudioMorphix: Training-free audio editing with diffusion probabilistic models
Editing sound with precision is a crucial yet underexplored challenge in audio content creation.
Leveraging Large Language Models for Command Injection Vulnerability Analysis in Python: An Empirical Study on Popular Open-Source Projects
This study evaluates the potential of large language models (LLMs), such as GPT-4, as an alternative approach for automated testing for vulnerability detection.
Personalize Your Gaussian: Consistent 3D Scene Personalization from a Single Image
Personalizing 3D scenes from a single reference image enables intuitive user-guided editing, which requires achieving both multi-view consistency across perspectives and referential consistency with the input image.
MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Each item in the benchmark demands multi-step deep reasoning beyond surface-level understanding.
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space
We introduce LatentSeek, a novel framework that enhances LLM reasoning through Test-Time Instance-level Adaptation (TTIA) within the model's latent space.
JAEGER: Dual-Level Humanoid Whole-Body Controller
This paper presents JAEGER, a dual-level whole-body controller for humanoid robots that addresses the challenges of training a more robust and versatile policy.
Probing and Inducing Combinational Creativity in Vision-Language Models
The ability to combine existing concepts into novel ideas stands as a fundamental hallmark of human intelligence.
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model
This technical report highlights the key design decisions that enhance the performance of the medium-sized diffusion model.
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training
Multimodal large language models (MLLMs), such as GPT-4o, are garnering significant attention.
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts
The rapid advancement of multi-modal language models (MLLMs) like GPT-4o has propelled the development of Omni language models, designed to process and proactively respond to continuous streams of multi-modal data.
QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
To bridge this gap, we introduce QualiSpeech, a comprehensive low-level speech quality assessment dataset encompassing 11 key aspects and detailed natural language comments that include reasoning and contextual insights.
Solla: Towards a Speech-Oriented LLM That Hears Acoustic Context
Large Language Models (LLMs) have recently shown remarkable ability to process not only text but also multimodal inputs such as speech and audio.
A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Control
In this study, we introduce a parallel hybrid action space reinforcement learning model (PH-DDPG) that optimizes traffic signal phase and duration of traffic signals simultaneously, eliminating the need for sequential decision-making seen in traditional two-stage models.
PBR3DGen: A VLM-guided Mesh Generation with High-quality PBR Texture
In this work, we present PBR3DGen, a two-stage mesh generation method with high-quality PBR materials that integrates the novel multi-view PBR material estimation model and a 3D PBR mesh reconstruction model.
From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation up to 100K Tokens
While traditional speculative decoding methods exist, simply extending their generation limits fails to accelerate the process and can be detrimental.
The establishment of static digital humans and the integration with spinal models
Next, we align the real spine model reconstructed from CT images with the standard skeletal model.
DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation
Several recent studies have attempted to autoregressively generate continuous speech representations without discrete speech tokens by combining diffusion and autoregressive models, yet they often face challenges with excessive computational loads or suboptimal outcomes.
Nautilus: Locality-aware Autoencoder for Scalable Mesh Generation
To address these challenges, we propose Nautilus, a locality-aware autoencoder for artist-like mesh generation that leverages the local properties of manifold meshes to achieve structural fidelity and efficient representation.
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Previous models for voice interactions are categorized as native and aligned.
Reasoning Mamba: Hypergraph-Guided Region Relation Calculating for Weakly Supervised Affordance Grounding
This paper pays attention to Weakly Supervised Affordance Grounding (WSAG) task that aims to train model to identify affordance regions using human-object interaction images and egocentric images without the need for costly pixel-level annotations.
Friends-MMC: A Dataset for Multi-modal Multi-party Conversation Understanding
Based on this Friends-MMC dataset, we further study two fundamental MMC tasks: conversation speaker identification and conversation response prediction, both of which have the multi-party nature with the video or image as visual context.
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode.
Pushing Rendering Boundaries: Hard Gaussian Splatting
To address this problem, we propose Hard Gaussian Splatting, dubbed HGS, which considers multi-view significant positional gradients and rendering errors to grow hard Gaussians that fill the gaps of classical Gaussian Splatting on 3D scenes, thus achieving superior NVS results.
SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
Unlike traditional modularised conversational AI systems, which separate speech recognition, understanding, and text-to-speech generation into distinct components, multimodal LLMs operate as single end-to-end models.
VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format
We construct MMDuetIT, a video-text training dataset designed to adapt VideoLLMs to video-text duet interaction format.
Vehicles, Pedestrians, and E-bikes: a Three-party Game at Right-turn-on-red Crossroads Revealing the Dual and Irrational Role of E-bikes that Risks Traffic Safety
The widespread use of e-bikes has facilitated short-distance travel yet led to confusion and safety problems in road traffic.
IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilities
Current methods of building LLMs with voice interaction capabilities rely heavily on explicit text autoregressive generation before or during speech response generation to maintain content quality, which unfortunately brings computational overhead and increases latency in multi-turn interactions.
Metadata Matters for Time Series: Informative Forecasting with Transformers
Further, a Transformer encoder is employed to communicate series and metadata tokens, which can extend series representations by metadata information for more accurate forecasting.
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
The results demonstrate that auditory LLMs achieve competitive performance compared to state-of-the-art task-specific small models in predicting MOS and SIM, while also delivering promising results in A/B testing and natural language descriptions.
NEST-RQ: Next Token Prediction for Speech Self-Supervised Pre-Training
NEST-RQ employs causal encoders with only left context and uses next token prediction (NTP) as the training task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
VideoLLaMB: Long-context Video Understanding with Recurrent Memory Bridges
Recent advancements in large-scale video-language models have shown significant potential for real-time planning and detailed interactions.
Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
To analyze image representations while completely avoiding the influence of all other factors other than the image representation itself, we propose a parametric-free representation alignment metric (Pfram) that can measure the similarities between any two representation systems without requiring additional training parameters.
ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning
Experimental results on two representative VL programming methods showcase consistent improvements on five compositional reasoning tasks on standard benchmarks.
Deep Time Series Models: A Comprehensive Survey and Benchmark
Further, we develop and release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks, which implements 24 mainstream models, covers 30 datasets from different domains, and supports five prevalent analysis tasks.
Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios.
Ranked #2 on
Speech Recognition
on AISHELL-1
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs).
CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation
Our in-depth category-level analysis reveals a lack of Chinese cultural knowledge in existing VLMs.
A Comprehensive Solution to Connect Speech Encoder and Large Language Model for ASR
Finally, we explore training and inference methods to mitigate high insertion errors.
VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models
Recent advancements in Multimodal Large Language Models (MLLMs) have extended their capabilities to video understanding.
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
To obtain fine-grained temporal information required by speech understanding, while keeping efficient for other video elements, this paper proposes a novel multi-resolution causal Q-Former (MRC Q-Former) structure to connect pre-trained audio-visual encoders and the backbone large language model.
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words
We argue that this is due to the lack of principles on task definition and model development, which requires open-source datasets and metrics suitable for model evaluation.
Bayesian Intervention Optimization for Causal Discovery
We demonstrate the effectiveness of our method through various experiments.
Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR
This paper addresses challenges in integrating new languages into a pre-trained multilingual automatic speech recognition (mASR) system, particularly in scenarios where training data for existing languages is limited or unavailable.
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
Seed-TTS offers superior controllability over various speech attributes such as emotion and is capable of generating highly expressive and diverse speech for speakers in the wild.
Progressive Confident Masking Attention Network for Audio-Visual Segmentation
Audio and visual signals typically occur simultaneously, and humans possess an innate ability to correlate and synchronize information from these two modalities.
A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategies
1) For the tuning method, we design a model assembly strategy to reconstruct a lightweight model while preserving performance through distillation.
Human-Centered LLM-Agent User Interface: A Position Paper
Large Language Model (LLM) -in-the-loop applications have been shown to effectively interpret the human user's commands, make plans, and operate external tools/systems accordingly.
Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges
As a result, an overview of the categories, methods, and evaluation of medical dialogue systems remain limited and underspecified, hindering the further improvement of this area.
FIMP-HGA: A Novel Approach to Addressing the Partitioning Min-Max Weighted Matching Problem
Similar to MP$_{\text{LS}}$, FIMP-HGA divides the solving into match and partition stages, iteratively refining the solution.
VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing
We present VoiceShop, a novel speech-to-speech framework that can modify multiple attributes of speech, such as age, gender, accent, and speech style, in a single forward pass while preserving the input speaker's timbre.
Predicate Debiasing in Vision-Language Models Integration for Scene Graph Generation Enhancement
Scene Graph Generation (SGG) provides basic language representation of visual scenes, requiring models to grasp complex and diverse semantics between objects.
View-Consistent 3D Editing with Gaussian Splatting
However, this approach faces a critical issue of multi-view inconsistency, where the guidance images exhibit significant discrepancies across views, leading to mode collapse and visual artifacts of 3DGS.
HawkEye: Training Video-Text LLMs for Grounding Text in Videos
Video-text Large Language Models (video-text LLMs) have shown remarkable performance in answering questions and holding conversations on simple videos.
Ranked #13 on
Video Question Answering
on MVBench
TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
We propose a novel approach, TimeXer, to ingest external information to enhance the forecasting of endogenous variables.
Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge
Our framework significantly enhances the temporal capabilities of current MLLMs through three key innovations: an efficient multi-span temporal grounding algorithm applied to low-dimension temporal features projected from flow; a multimodal length extrapolation training paradigm that utilizes low-dimension temporal features to extend the training context window size; and a bootstrapping framework that bridges our model with pluggable MLLMs without requiring annotation.
Ranked #29 on
Video Question Answering
on NExT-QA
TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling
To emphasize temporal correlation modeling, this paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering
However, most models can only handle simple videos in terms of temporal reasoning, and their performance tends to drop when answering temporal-reasoning questions on long and informative videos.
A-SDM: Accelerating Stable Diffusion through Redundancy Removal and Performance Optimization
The Stable Diffusion Model (SDM) is a popular and efficient text-to-image (t2i) generation and image-to-image (i2i) generation model.
Audio Prompt Tuning for Universal Sound Separation
Specifically, APT improves the separation performance of specific sources through training a small number of prompt parameters with limited audio samples, while maintaining the generalization of the USS model by keeping its parameters frozen.
LLaMA Rider: Spurring Large Language Models to Explore the Open World
Recently, various studies have leveraged Large Language Models (LLMs) to help decision-making and planning in environments, and try to align the LLMs' knowledge with the world conditions.
Teaching Text-to-Image Models to Communicate in Dialog
In this paper, we focus on the innovative dialog-to-image generation task, where the model synthesizes a high-resolution image aligned with the given dialog context as a response.
InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models
We tested InstructME in instrument-editing, remixing, and multi-round editing.
Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP).
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Any audio can be translated into LOA based on AudioMAE, a self-supervised pre-trained representation learning model.
Ranked #4 on
Audio Generation
on AudioCaps
(FAD metric)
Separate Anything You Describe
In this work, we introduce AudioSep, a foundation model for open-domain audio source separation with natural language queries.
Query Encoder Distillation via Embedding Alignment is a Strong Baseline Method to Boost Dense Retriever Online Efficiency
The information retrieval community has made significant progress in improving the efficiency of Dual Encoder (DE) dense passage retrieval systems, making them suitable for latency-sensitive settings.
PolyVoice: Language Models for Speech to Speech Translation
For the speech synthesis part, we adopt the existing VALL-E X approach and build a unit-based audio language model.
MoviePuzzle: Visual Narrative Reasoning through Multimodal Order Learning
We introduce MoviePuzzle, a novel challenge that targets visual narrative reasoning and holistic movie understanding.
Shuo Wen Jie Zi: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training
We introduce CDBERT, a new learning paradigm that enhances the semantics understanding ability of the Chinese PLMs with dictionary knowledge and structure of Chinese characters.
VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions
Video-grounded dialogue understanding is a challenging problem that requires machine to perceive, parse and reason over situated semantics extracted from weakly aligned video and dialogues.
Language-universal phonetic encoder for low-resource speech recognition
Our main approach and adaptation are effective on extremely low-resource languages, even within domain- and language-mismatched scenarios.
Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition
Moreover, on 3 of the 4 languages, comparing to the standard HuBERT, the approach performs better, meanwhile is able to save supervised training data by 1. 5k hours (75%) at most.
A unified front-end framework for English text-to-speech synthesis
The front-end is a critical component of English text-to-speech (TTS) systems, responsible for extracting linguistic features that are essential for a text-to-speech model to synthesize speech, such as prosodies and phonemes.
Memory Augmented Lookup Dictionary based Language Modeling for Automatic Speech Recognition
Recent studies have shown that using an external Language Model (LM) benefits the end-to-end Automatic Speech Recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Zero-Shot Accent Conversion using Pseudo Siamese Disentanglement Network
The goal of accent conversion (AC) is to convert the accent of speech into the target accent while preserving the content and speaker identity.
Interactive Context-Aware Network for RGB-T Salient Object Detection
However, most related works are based on RGB images, which lose massive useful information.
Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidance
In this paper, we propose to use intermediate bottleneck features (IBFs) to replace PPGs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue Generation
We study video-grounded dialogue generation, where a response is generated based on the dialogue context and the associated video.
How Many Grid-Forming Converters Do We Need? A Perspective From Small Signal Stability and Power Grid Strength
Based on our analysis, we further study the problem of how to configure GFM converters in the grid and how many GFM converters we will need.
Network-Level Adversaries in Federated Learning
Federated learning is a popular strategy for training models on distributed, sensitive data, while preserving data privacy.
Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task
However, CL on VQA involves not only the expansion of label sets (new Answer sets).
SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation
Adapting to a continuously evolving environment is a safety-critical challenge inevitably faced by all autonomous driving systems.
VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration
Speech restoration aims to remove distortions in speech signals.
GEB+: A Benchmark for Generic Event Boundary Captioning, Grounding and Retrieval
In this paper, we introduce a new dataset called Kinetic-GEB+.
Ranked #1 on
Boundary Captioning
on Kinetics-GEB+
The USTC-Ximalaya system for the ICASSP 2022 multi-channel multi-party meeting transcription (M2MeT) challenge
We propose two improvements to target-speaker voice activity detection (TS-VAD), the core component in our proposed speaker diarization system that was submitted to the 2022 Multi-Channel Multi-Party Meeting Transcription (M2MeT) challenge.
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant
In contrast, we present a new task called Task-oriented Question-driven Video Segment Retrieval (TQVSR).
Neural Dubber: Dubbing for Videos According to Scripts
Neural Dubber is a multi-modal text-to-speech (TTS) model that utilizes the lip movement in the video to control the prosody of the generated speech.
Deep Superpixel-based Network for Blind Image Quality Assessment
In order to fill this gap, we propose a deep adaptive superpixel-based network, namely DSN-IQA, to assess the quality of image based on multi-scale and superpixel segmentation.
Cloning one's voice using very limited data in the wild
(2) How to clone a person's voice while controlling the style and prosody.
Audiovisual Singing Voice Separation
Separating a song into vocal and accompaniment components is an active research topic, and recent years witnessed an increased performance from supervised training using deep learning techniques.
VeniBot: Towards Autonomous Venipuncture with Semi-supervised Vein Segmentation from Ultrasound Images
In the modern medical care, venipuncture is an indispensable procedure for both diagnosis and treatment.
VeniBot: Towards Autonomous Venipuncture with Automatic Puncture Area and Angle Regression from NIR Images
For the automation, we focus on the positioning part and propose a Dual-In-Dual-Out network based on two-step learning and two-task learning, which can achieve fully automatic regression of the suitable puncture area and angle from near-infrared(NIR) images.
Supervised Chorus Detection for Popular Music Using Convolutional Neural Network and Multi-task Learning
This paper presents a novel supervised approach to detecting the chorus segments in popular music.
Modeling the Compatibility of Stem Tracks to Generate Music Mashups
A music mashup combines audio elements from two or more songs to create a new work.
USTC-NELSLIP System Description for DIHARD-III Challenge
This system description describes our submission system to the Third DIHARD Speech Diarization Challenge.
Listen, Read, and Identify: Multimodal Singing Language Identification of Music
Optionally, LRID-Net is facilitated with modality dropouts to handle a missing modality.
Large-Scale MIDI-based Composer Classification
Music classification is a task to classify a music piece into labels such as genres or composers.
GiantMIDI-Piano: A large-scale MIDI dataset for classical piano music
In this article, we create a GiantMIDI-Piano (GP) dataset containing 38, 700, 838 transcribed notes and 10, 855 unique solo piano works composed by 2, 786 composers.
High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times
In addition, previous AMT systems are sensitive to the misaligned onset and offset labels of audio recordings.
Ranked #5 on
Music Transcription
on MAESTRO
Music Transcription
Sound
Audio and Speech Processing
Xiaomingbot: A Multilingual Robot News Reporter
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation.
Noise Robust TTS for Low Resource Speakers using Pre-trained Model and Speech Enhancement
Firstly, the speech synthesis model is pre-trained with both multi-speaker clean data and noisy augmented data; then the pre-trained model is adapted on noisy low-resource new speaker data; finally, by setting the clean speech condition, the model can synthesize the new speaker's clean voice.
Improving Accent Conversion with Reference Encoder and End-To-End Text-To-Speech
Accent conversion (AC) transforms a non-native speaker's accent into a native accent while maintaining the speaker's voice timbre.
Review of Text Style Transfer Based on Deep Learning
In the traditional text style transfer model, the text style is generally relied on by experts knowledge and hand-designed rules, but with the application of deep learning in the field of natural language processing, the text style transfer method based on deep learning Started to be heavily researched.
Adversarial Feature Learning and Unsupervised Clustering based Speech Synthesis for Found Data with Acoustic and Textual Noise
As for the speech-side noise, we propose to learn a noise-independent feature in the auto-regressive decoder through adversarial training and data augmentation, which does not need an extra speech enhancement model.
ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders
This paper presents ByteSing, a Chinese singing voice synthesis (SVS) system based on duration allocated Tacotron-like acoustic models and WaveRNN neural vocoders.
Convolutional Embedding for Edit Distance
Edit-distance-based string similarity search has many applications such as spell correction, data de-duplication, and sequence alignment.
A unified sequence-to-sequence front-end model for Mandarin text-to-speech synthesis
In Mandarin text-to-speech (TTS) system, the front-end text processing module significantly influences the intelligibility and naturalness of synthesized speech.
A hybrid text normalization system using multi-head self-attention for mandarin
In this paper, we propose a hybrid text normalization system using multi-head self-attention.
HIT-SCIR at MRP 2019: A Unified Pipeline for Meaning Representation Parsing via Efficient Training and Effective Encoding
This paper describes our system (HIT-SCIR) for CoNLL 2019 shared task: Cross-Framework Meaning Representation Parsing.
Ranked #1 on
UCCA Parsing
on CoNLL 2019
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
In this approach, a linear transformation is learned from contextual word alignments to align the contextualized embeddings independently trained in different languages.
Hierarchical Generative Modeling for Controllable Speech Synthesis
This paper proposes a neural sequence-to-sequence text-to-speech (TTS) model which can control latent attributes in the generated speech that are rarely annotated in the training data, such as speaking style, accent, background noise, and recording conditions.
Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis
We demonstrate that the proposed framework enables Tacotron to generate intelligible speech using less than half an hour of paired training data.
Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis
GSTs can be used within Tacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to uncover expressive factors of variation in speaking style.
Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation
This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies.
Ranked #3 on
Dependency Parsing
on Universal Dependencies
Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
We present an extension to the Tacotron speech synthesis architecture that learns a latent embedding space of prosody, derived from a reference acoustic representation containing the desired prosody.
Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
In this work, we propose "global style tokens" (GSTs), a bank of embeddings that are jointly trained within Tacotron, a state-of-the-art end-to-end speech synthesis system.
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text.
Ranked #2 on
Speech Synthesis
on North American English
Uncovering Latent Style Factors for Expressive Speech Synthesis
Prosodic modeling is a core problem in speech synthesis.
The HIT-SCIR System for End-to-End Parsing of Universal Dependencies
Our system includes three pipelined components: \textit{tokenization}, \textit{Part-of-Speech} (POS) \textit{tagging} and \textit{dependency parsing}.
Tacotron: Towards End-to-End Speech Synthesis
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.
Ranked #5 on
Speech Synthesis
on North American English
Trainable Frontend For Robust and Far-Field Keyword Spotting
Robust and far-field speech recognition is critical to enable true hands-free communication.
Cocktail Party Processing via Structured Prediction
While human listeners excel at selectively attending to a conversation in a cocktail party, machine performance is still far inferior by comparison.
