Search Results for author:
Found 90 papers, 48 papers with code
Eyeglasses 3D shape reconstruction from a single face image
A complete 3D face reconstruction requires to explicitly model the eyeglasses on the face, which is less investigated in the literature.
EmRel: Joint Representation of Entities and Embedded Relations for Multi-triple Extraction
Multi-triple extraction is a challenging task due to the existence of informative inter-triple correlations, and consequently rich interactions across the constituent entities and relations. While existing works only explore entity representations, we propose to explicitly introduce relation representation, jointly represent it with entities, and novelly align them to identify valid triples. We perform comprehensive experiments on document-level relation extraction and joint entity and relation extraction along with ablations to demonstrate the advantage of the proposed method.
Document-level Relation Extraction
Joint Entity and Relation Extraction
BDKG at MEDIQA 2021: System Report for the Radiology Report Summarization Task
This paper presents our winning system at the Radiology Report Summarization track of the MEDIQA 2021 shared task.
Learn and Review: Enhancing Continual Named Entity Recognition via Reviewing Synthetic Samples
However, the existing method depends on the relevance between tasks and is prone to inter-type confusion. In this paper, we propose a novel two-stage framework Learn-and-Review (L&R) for continual NER under the type-incremental setting to alleviate the above issues. Specifically, for the learning stage, we distill the old knowledge from teacher to a student on the current dataset.
Continual Named Entity Recognition
named-entity-recognition
+2
Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network
Whether it is the conventional modularized approach or the more recent end-to-end neural diarization (EEND), an additional automatic speech recognition (ASR) model and an orchestration algorithm are required to associate the speaker labels with recognized words.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Models
We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages.
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts.
Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation
Existing autoregressive models follow the two-stage generation paradigm that first learns a codebook in the latent space for image reconstruction and then completes the image generation autoregressively based on the learned codebook.
Inductive Relation Prediction from Relational Paths and Context with Hierarchical Transformers
Relation prediction on knowledge graphs (KGs) is a key research topic.
$k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
In-Context Learning (ICL), which formulates target tasks as prompt completion conditioned on in-context demonstrations, has become the prevailing utilization of LLMs.
CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Augmenting Transformer-Transducer Based Speaker Change Detection With Token-Level Training Loss
Due to the sparsity of the speaker changes in the training data, the conventional T-T based SCD model loss leads to sub-optimal detection accuracy.
Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting
In this paper, we present a novel approach to adapt a sequence-to-sequence Transformer-Transducer ASR system to the keyword spotting (KWS) task.
Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clustering
While recent research advances in speaker diarization mostly focus on improving the quality of diarization results, there is also an increasing interest in improving the efficiency of diarization systems.
Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity
In this paper, we consider introducing an auxiliary task of Chinese pronunciation prediction (CPP) to improve CSC, and, for the first time, systematically discuss the adaptivity and granularity of this auxiliary task.
A Universally-Deployable ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement, and Voice Separation
The joint model uses contextual information, such as a reference of the playback audio, noise context, and speaker embedding.
Compact and Robust Deep Learning Architecture for Fluorescence Lifetime Imaging and FPGA Implementation
FLAN+LS on hardware achieves the highest computing efficiency compared to 1-D CNN and FLAN.
DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation
Text-to-image generation aims at generating realistic images which are semantically consistent with the given text.
MFAN: Multi-modal Feature-enhanced Attention Networks for Rumor Detection
Rumor spreaders are increasingly taking advantage of multimedia content to attract and mislead news consumers on social media.
Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation
Previous works on morphable models mostly focus on large-scale facial geometry but ignore facial details.
Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Personal VAD 2.0: Optimizing Personal Voice Activity Detection for On-Device Speech Recognition
Personalization of on-device speech recognition (ASR) has seen explosive growth in recent years, largely due to the increasing popularity of personal assistant features on mobile devices and smart home speakers.
Fast fluorescence lifetime imaging analysis via extreme learning machine
As there is no back-propagation process for ELM during the training phase, the training speed is much higher than existing neural network approaches.
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Ranked #4 on
Motion Synthesis
on AIST++
Parameter-Free Attentive Scoring for Speaker Verification
This paper presents a novel study of parameter-free attentive scoring for speaker verification.
Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech
In this paper, we introduce a novel language identification system based on conformer layers.
Closing the Gap between Single-User and Multi-User VoiceFilter-Lite
However, one limitation of VoiceFilter-Lite, and other speaker-conditioned speech models in general, is that these models are usually limited to a single target speaker.
Microservice Deployment in Edge Computing Based on Deep Q Learning
Then, we propose a multi-objective microservice deployment problem (MMDP) in edge computing.
CVSS Corpus and Massively Multilingual Speech-to-Speech Translation
In addition, CVSS provides normalized translation text which matches the pronunciation in the translation speech.
Negative-Aware Attention Framework for Image-Text Matching
Image-text matching, as a fundamental task, bridges the gap between vision and language.
A Conformer-based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation
Compared to the noisy baseline, the joint model reduces the word error rate in low signal-to-noise ratio conditions by at least 71% on our echo cancellation dataset, 10% on our noisy dataset, and 26% on our multi-speaker dataset.
Cross-attention conformer for context modeling in speech enhancement for ASR
This work introduces \emph{cross-attention conformer}, an attention-based architecture for context modeling in speech enhancement.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination
In this work we introduce eHealth, a Chinese biomedical PLM built from scratch with a new pre-training framework.
Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection
In this paper, we present a novel speaker diarization system for streaming on-device applications.
Learning Oculomotor Behaviors from Scanpath
Identifying oculomotor behaviors relevant for eye-tracking applications is a critical but often challenging task.
Multi-user VoiceFilter-Lite via Attentive Speaker Embedding
In this paper, we propose a solution to allow speaker conditioned speech models, such as VoiceFilter-Lite, to support an arbitrary number of enrolled users in a single pass.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Link Prediction on N-ary Relational Facts: A Graph-based Approach
The key to our approach is to represent the n-ary structure of a fact as a small heterogeneous graph, and model this graph with edge-biased fully-connected attention.
Inverting Generative Adversarial Renderer for Face Reconstruction
Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization.
Personalized Keyphrase Detection using Speaker and Environment Information
In this paper, we introduce a streaming keyphrase detection system that can be easily customized to accurately detect any phrase composed of words from a large vocabulary.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SpeakerStew: Scaling to Many Languages with a Triaged Multilingual Text-Dependent and Text-Independent Speaker Verification System
To the best of our knowledge, this is the first study of speaker verification systems at the scale of 46 languages.
Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition
In this work we propose scoring these representations in a way that can capture uncertainty, enroll/test asymmetry and additional non-linear information.
Entity Structure Within and Throughout: Modeling Mention Dependencies for Document-Level Relation Extraction
Our experiments demonstrate the usefulness of the proposed entity structure and the effectiveness of SSAN.
Ranked #3 on
Relation Extraction
on DocRED
Synth2Aug: Cross-domain speaker recognition with TTS synthesized speech
In recent years, Text-To-Speech (TTS) has been used as a data augmentation technique for speech recognition to help complement inadequacies in the training data.
Event Extraction as Multi-turn Question Answering
Most traditional approaches formulate this task as classification problems, with event types or argument roles taken as golden labels.
VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition
We introduce VoiceFilter-Lite, a single-channel source separation model that runs on the device to preserve only the speech signals from a target user, as part of a streaming speech recognition system.
Textual Echo Cancellation
In this paper, we propose Textual Echo Cancellation (TEC) - a framework for cancelling the text-to-speech (TTS) playback echo from overlapping speech recordings.
Version Control of Speaker Recognition Systems
In this paper, we describe different version control strategies for speaker recognition systems that had been carefully studied at Google from years of engineering practice.
A Comparative Study on Polyp Classification using Convolutional Neural Networks
In this work, we compare the performance of the state-of-the-art general object classification models for polyp classification.
Curriculum Learning for Natural Language Understanding
With the great success of pre-trained language models, the pretrain-finetune paradigm now becomes the undoubtedly dominant solution for natural language understanding (NLU) tasks.
Fast and Accurate: Structure Coherence Component for Face Alignment
In this paper, we propose a fast and accurate coordinate regression method for face alignment.
Ranked #14 on
Face Alignment
on COFW
Interpretable and Efficient Heterogeneous Graph Convolutional Network
However, regarding Heterogeneous Information Network (HIN), existing HIN-oriented GCN methods still suffer from two deficiencies: (1) they cannot flexibly explore all possible meta-paths and extract the most useful ones for a target object, which hinders both effectiveness and interpretability; (2) they often need to generate intermediate meta-path based dense graphs, which leads to high computational complexity.
CoKE: Contextualized Knowledge Graph Embedding
This work presents Contextualized Knowledge Graph Embedding (CoKE), a novel paradigm that takes into account such contextual nature, and learns dynamic, flexible, and fully contextualized entity and relation embeddings.
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Spoofing attacks within a logical access (LA) scenario are generated with the latest speech synthesis and voice conversion technologies, including state-of-the-art neural acoustic and waveform model techniques.
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos
A structure predictor is proposed to predict the missing face structural information temporally, which serves as a geometry prior.
Make a Face: Towards Arbitrary High Fidelity Face Manipulation
Recent studies have shown remarkable success in face manipulation task with the advance of GANs and VAEs paradigms, but the outputs are sometimes limited to low-resolution and lack of diversity.
Personal VAD: Speaker-Conditioned Voice Activity Detection
In this paper, we propose "personal VAD", a system to detect the voice activity of a target speaker at the frame level.
Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension
In this work, we investigate the potential of leveraging external knowledge bases (KBs) to further improve BERT for MRC.
Adaptive Convolution for Multi-Relational Learning
We consider the problem of learning distributed representations for entities and relations of multi-relational data so as to predict missing links therein.
Ranked #10 on
Link Prediction
on WN18
Tuplemax Loss for Language Identification
In many scenarios of a language identification task, the user will specify a small set of languages which he/she can speak instead of a large set of all possible languages.
VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking
In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker.
Fully Supervised Speaker Diarization
In this paper, we propose a fully supervised speaker diarization approach, named unbounded interleaved-state recurrent neural networks (UIS-RNN).
Ranked #1 on
Speaker Diarization
on Hub5'00 CallHome
Sample Efficient Adaptive Text-to-Speech
Instead, the aim is to produce a network that requires few data at deployment time to rapidly adapt to new speakers.
An Efficient Approach for Polyps Detection in Endoscopic Videos Based on Faster R-CNN
Polyp has long been considered as one of the major etiologies to colorectal cancer which is a fatal disease around the world, thus early detection and recognition of polyps plays a crucial role in clinical routines.
Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
Clone a voice in 5 seconds to generate arbitrary speech in real-time
Look at Boundary: A Boundary-Aware Face Alignment Algorithm
By utilising boundary information of 300-W dataset, our method achieves 3. 92% mean error with 0. 39% failure rate on COFW dataset, and 1. 25% mean error on AFLW-Full dataset.
Ranked #3 on
Face Alignment
on AFLW-19
(using extra training data)
Improving Knowledge Graph Embedding Using Simple Constraints
We examine non-negativity constraints on entity representations and approximate entailment constraints on relation representations.
Links: A High-Dimensional Online Clustering Method
We present a novel algorithm, called Links, designed to perform online clustering on unit vectors in a high-dimensional Euclidean space.
Wavenet based low rate speech coding
Traditional parametric coding of speech facilitates low rate but provides poor reconstruction quality because of the inadequacy of the model used.
Knowledge Graph Embedding with Iterative Guidance from Soft Rules
In this paper, we propose Rule-Guided Embedding (RUGE), a novel paradigm of KG embedding with iterative guidance from soft rules.
Ranked #2 on
Link Prediction
on YAGO37
Generalized End-to-End Loss for Speaker Verification
In this paper, we propose a new loss function called generalized end-to-end (GE2E) loss, which makes the training of speaker verification models more efficient than our previous tuple-based end-to-end (TE2E) loss function.
Ranked #1 on
Speaker Verification
on CALLHOME
Attention-Based Models for Text-Dependent Speaker Verification
Attention-based models have recently shown great performance on a range of tasks, such as speech recognition, machine translation, and image captioning due to their ability to summarize relevant information that expands through the entire length of an input sequence.
Speaker Diarization with LSTM
For many years, i-vector based audio embedding techniques were the dominant approach for speaker verification and speaker diarization applications.
Ranked #1 on
Speaker Diarization
on CALLHOME-109
Relation Extraction with Multi-instance Multi-label Convolutional Neural Networks
In this paper, we propose a multi-instance multi-label convolutional neural network for distantly supervised RE.
Label Consistent Fisher Vectors for Supervised Feature Aggregation
In this paper, we present a simple and efficient way to add supervised information into Fisher vectors, which has become a popular image representation method for image classification and retrieval purposes in recent years.
Semantic Context Forests for Learning-Based Knee Cartilage Segmentation in 3D MR Images
The automatic segmentation of human knee cartilage from 3D MR images is a useful yet challenging task due to the thin sheet structure of the cartilage with diffuse boundaries and inhomogeneous intensities.
Feature Learning by Multidimensional Scaling and its Applications in Object Recognition
The aspects of the images that are captured by the learned features, which we call MDS features, completely depend on what kind of image distance measurement is employed.
GMM-Based Hidden Markov Random Field for Color Image and 3D Volume Segmentation
In this project, we first study the Gaussian-based hidden Markov random field (HMRF) model and its expectation-maximization (EM) algorithm.
The active geometric shape model: A new robust deformable shape model and its applications
Similar to active shape models and active contours, a force field is used in our approach.
Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models
Principal component analysis (PCA) is a popular tool for linear dimensionality reduction and feature extraction.
HMRF-EM-image: Implementation of the Hidden Markov Random Field Model and its Expectation-Maximization Algorithm
In this project, we study the hidden Markov random field (HMRF) model and its expectation-maximization (EM) algorithm.
Tracking Tetrahymena Pyriformis Cells using Decision Trees
Matching cells over time has long been the most difficult step in cell tracking.
