Search Results for author:
Found 149 papers, 65 papers with code
SOWing Information: Cultivating Contextual Coherence with MLLMs in Image Generation
Originating from the diffusion phenomenon in physics, which describes the random movement and collisions of particles, diffusion generative models simulate a random walk in the data space along the denoising trajectory.
Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation
Language-conditioned robotic learning has significantly enhanced robot adaptability by enabling a single model to execute diverse tasks in response to verbal commands.
EAPCR: A Universal Feature Extractor for Scientific Data without Explicit Feature Relation Patterns
The absence of explicit Feature Relation Patterns (FRPs) presents a significant challenge for deep learning techniques in scientific applications that are not image, text, and graph-based.
OCMDP: Observation-Constrained Markov Decision Process
In many practical applications, decision-making processes must balance the costs of acquiring information with the benefits it provides.
ROBIN: Robust and Invisible Watermarks for Diffusion Models with Adversarial Optimization
They empirically inject a watermark that is both invisible and robust and passively achieve concealment by limiting the strength of the watermark, thus reducing the robustness.
When to Trust Your Data: Enhancing Dyna-Style Model-Based Reinforcement Learning With Data Filter
To tackle this issue, we introduce an out-of-distribution (OOD) data filter that removes simulated data from the estimated model that significantly diverges from data collected in the real environment.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast
However, existing CL methods often introduce false negative pairs (FNPs) by neglecting inherent characteristics and then randomly selecting distinct segments as dissimilar pairs, leading to erroneous representation learning, reduced model performance, and overall inefficiency.
Learning To Help: Training Models to Assist Legacy Devices
We cast this problem in the framework of learning with abstention (LWA) in which the expert (edge) must be trained to assist the client (device).
WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration
LLM-based autonomous agents often fail to execute complex web tasks that require dynamic interaction due to the inherent uncertainty and complexity of these environments.
An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Grounding
In this paper, we propose an efficient and effective multi-task visual grounding (EEVG) framework based on Transformer Decoder to address this issue, which reduces the cost in both language and visual aspects.
Iterative Ensemble Training with Anti-Gradient Control for Mitigating Memorization in Diffusion Models
In this paper, we propose a novel training framework for diffusion models from the perspective of visual modality, which is more generic and fundamental for mitigating memorization.
Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking
Our pretrained models demonstrate superior performance (against existing acoustic models pretrained with general audio on 16 out of 19 tasks) and generalizability (to unseen datasets and new respiratory audio modalities).
MAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to learn semantic primitives (attributes and objects) from seen compositions and recognize unseen attribute-object compositions.
LAIP: Learning Local Alignment from Image-Phrase Modeling for Text-based Person Search
This paper proposes the Local Alignment from Image-Phrase modeling (LAIP) framework, with Bidirectional Attention-weighted local alignment (BidirAtt) and Mask Phrase Modeling (MPM) module. BidirAtt goes beyond the typical forward attention by considering the gradient of the transformer as backward attention, utilizing two-sided information for local alignment.
CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation
To address this gap, we introduce CoMM, a high-quality Coherent interleaved image-text MultiModal dataset designed to enhance the coherence, consistency, and alignment of generated multimodal content.
D$^3$: Scaling Up Deepfake Detection by Learning from Discrepancy
The boom of Generative AI brings opportunities entangled with risks and concerns.
Improving Bird's Eye View Semantic Segmentation by Task Decomposition
In the first stage, we train a BEV autoencoder to reconstruct the BEV segmentation maps given corrupted noisy latent representation, which urges the decoder to learn fundamental knowledge of typical BEV patterns.
Learning by Correction: Efficient Tuning Task for Zero-Shot Generative Vision-Language Reasoning
Generative vision-language models (VLMs) have shown impressive performance in zero-shot vision-language tasks like image captioning and visual question answering.
Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks.
Omni-Q: Omni-Directional Scene Understanding for Unsupervised Visual Grounding
Unsupervised visual grounding methods alleviate the issue of expensive manual annotation of image-query pairs by generating pseudo-queries.
Joint Trading and Scheduling among Coupled Carbon-Electricity-Heat-Gas Industrial Clusters
This paper presents a carbon-energy coupling management framework for an industrial park, where the carbon flow model accompanying multi-energy flows is adopted to track and suppress carbon emissions on the user side.
DVIS++: Improved Decoupled Framework for Universal Video Segmentation
We present the \textbf{D}ecoupled \textbf{VI}deo \textbf{S}egmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS).
Ranked #1 on
Video Semantic Segmentation
on VSPW
DETER: Detecting Edited Regions for Deterring Generative Manipulations
While face swapping and attribute editing are performed on similar face regions such as eyes and nose, the inpainting operation can be performed on random image regions, removing the spurious correlations of previous datasets.
A Survey on Trustworthy Edge Intelligence: From Security and Reliability To Transparency and Sustainability
Edge Intelligence (EI) integrates Edge Computing (EC) and Artificial Intelligence (AI) to push the capabilities of AI to the network edge for real-time, efficient and secure intelligent decision-making and computation.
SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes
Implicit neural representation has opened up new possibilities for inverse rendering.
Discovery and Expansion of New Domains within Diffusion Models
In this work, we study the generalization properties of diffusion models in a few-shot setup, introduce a novel tuning-free paradigm to synthesize the target out-of-domain (OOD) data, and demonstrate its advantages compared to existing methods in data-sparse scenarios with large domain gaps.
1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation
Video instance segmentation is a challenging task that serves as the cornerstone of numerous downstream applications, including video editing and autonomous driving.
Vision-Based Human Pose Estimation via Deep Learning: A Survey
In this article, we provide an up-to-date and in-depth overview of the deep learning approaches in vision-based HPE.
WavMark: Watermarking for Audio Generation
Recent breakthroughs in zero-shot voice synthesis have enabled imitating a speaker's voice using just a few seconds of recording while maintaining a high level of realism.
Grounded Image Text Matching with Mismatched Relation Reasoning
This paper introduces Grounded Image Text Matching with Mismatched Relation (GITM-MR), a novel visual-linguistic joint task that evaluates the relation understanding capabilities of transformer-based pre-trained models.
UDAMA: Unsupervised Domain Adaptation through Multi-discriminator Adversarial Training with Noisy Labels Improves Cardio-fitness Prediction
However, most healthcare datasets with high-quality (gold-standard) labels are small-scale, as directly collecting ground truth is often costly and time-consuming.
On decoder-only architecture for speech-to-text and large language model integration
Large language models (LLMs) have achieved remarkable success in the field of natural language processing, enabling better human-computer interaction using natural language.
Accelerating Transducers through Adjacent Token Merging
Recent end-to-end automatic speech recognition (ASR) systems often utilize a Transformer-based acoustic encoder that generates embedding at a high frame rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognition
Different from these methods, in this work, with only a domain-specific text prompt, we propose two zero-shot ASR domain adaptation methods using LLaMA, a 7-billion-parameter large language model (LLM).
Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation
In this work, we investigate the diffusion (physics) in diffusion (machine learning) properties and propose our Cyclic One-Way Diffusion (COW) method to control the direction of diffusion phenomenon given a pre-trained frozen diffusion model for versatile customization application scenarios, where the low-level pixel information from the conditioning needs to be preserved.
1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation
In this report, we successfully validated the effectiveness of the decoupling strategy in video panoptic segmentation.
DVIS: Decoupled Video Instance Segmentation Framework
The efficacy of the decoupling strategy relies on two crucial elements: 1) attaining precise long-term alignment outcomes via frame-by-frame association during tracking, and 2) the effective utilization of temporal information predicated on the aforementioned accurate alignment outcomes during refinement.
Ranked #4 on
Video Panoptic Segmentation
on VIPSeg
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
In this paper, we propose a novel compression strategy that leverages structured pruning and knowledge distillation to reduce the model size and inference cost of the Conformer model while preserving high recognition performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translation
Recent research shows a big convergence in model architecture, training objectives, and inference methods across various tasks for different modalities.
Click-Feedback Retrieval
In this work, we study a setting where the feedback is provided through users clicking liked and disliked searching results.
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World
Based on that, we further introduce a novel Entangled cross-modal prompt approach for open-world predicate scene graph generation (Epic), where models can generalize to unseen predicates in a zero-shot manner.
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
We propose a cross-lingual neural codec language model, VALL-E X, for cross-lingual speech synthesis.
Boundary Guided Learning-Free Semantic Control with Diffusion Models
Applying pre-trained generative denoising diffusion models (DDMs) for downstream tasks such as image semantic editing usually requires either fine-tuning DDMs or learning auxiliary editing networks in the existing literature.
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
In addition, we find Vall-E could preserve the speaker's emotion and acoustic environment of the acoustic prompt in synthesis.
Learning To Segment Every Referring Object Point by Point
Extensive experiments show that our model achieves 52. 06% in terms of accuracy (versus 58. 93% in fully supervised setting) on RefCOCO+@testA, when only using 1% of the mask annotations.
Good Is Bad: Causality Inspired Cloth-Debiasing for Cloth-Changing Person Re-Identification
Entangled representation of clothing and identity (ID)-intrinsic clues are potentially concomitant in conventional person Re-IDentification (ReID).
Generative Graph Neural Networks for Link Prediction
Instead of sampling positive and negative links and heuristically computing the features of their enclosing subgraphs, GraphLP utilizes the feature learning ability of deep-learning models to automatically extract the structural patterns of graphs for link prediction under the assumption that real-world graphs are not locally isolated.
How to Share: Balancing Layer and Chain Sharing in Industrial Microservice Deployment
However, due to the limited resources of edge servers, it is difficult to meet the optimization goals of the two methods at the same time.
BEATs: Audio Pre-Training with Acoustic Tokenizers
In the first iteration, we use random projection as the acoustic tokenizer to train an audio SSL model in a mask and label prediction manner.
Ranked #2 on
Audio Classification
on Balanced Audio Set
Artificial Intelligence Security Competition (AISC)
The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems.
Turning Silver into Gold: Domain Adaptation with Noisy Labels for Wearable Cardio-Respiratory Fitness Prediction
Deep learning models have shown great promise in various healthcare applications.
Exploring WavLM on Speech Enhancement
There is a surge in interest in self-supervised learning approaches for end-to-end speech encoding in recent years as they have achieved great success.
LongFNT: Long-form Speech Recognition with Factorized Neural Transducer
This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Speech separation with large-scale self-supervised learning
Compared with a supervised baseline and the WavLM-based SS model using feature embeddings obtained with the previously released 94K hours trained WavLM, our proposed model obtains 15. 9% and 11. 2% of relative word error rate (WER) reductions, respectively, for a simulated far-field speech mixture test set.
LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers
In this paper, we propose LAMASSU, a streaming language-agnostic multilingual speech recognition and translation model using neural transducers.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Two-Stream Network for Sign Language Recognition and Translation
RGB videos, however, are raw signals with substantial visual redundancy, leading the encoder to overlook the key information for sign language understanding.
Ranked #1 on
Sign Language Translation
on CSL-Daily
Real-time Speech Interruption Analysis: From Cloud to Client Deployment
Meetings are an essential form of communication for all types of organizations, and remote collaboration systems have been much more widely used since the COVID-19 pandemic.
STAR: Zero-Shot Chinese Character Recognition with Stroke- and Radical-Level Decompositions
The proposed method consists of a training stage and an inference stage.
Foundation Transformers
A big convergence of model architectures across language, vision, speech, and multimodal is emerging.
Vision+X: A Survey on Multimodal Learning in the Light of Data
We are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed and interpreted by separate parts of the human brain to constitute a complex, yet harmonious and unified sensing system.
SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Data
In this paper, we propose a cross-modal Speech and Language Model (SpeechLM) to explicitly align speech and text pre-training with a pre-defined unified discrete representation.
SiRi: A Simple Selective Retraining Mechanism for Transformer-based Visual Grounding
Particularly, SiRi conveys a significant principle to the research of visual grounding, i. e., a better initialized vision-language encoder would help the model converge to a better local minimum, advancing the performance accordingly.
Provably Efficient Reinforcement Learning for Online Adaptive Influence Maximization
Online influence maximization aims to maximize the influence spread of a content in a social network with unknown network model by selecting a few seed nodes.
Model-based Reinforcement Learning
reinforcement-learning
+2
Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training
Recently, masked prediction pre-training has seen remarkable progress in self-supervised learning (SSL) for speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
Diffusion probabilistic models (DPMs) have become a popular approach to conditional generation, due to their promising results and support for cross-modal synthesis.
Longitudinal cardio-respiratory fitness prediction through wearables in free-living environments
Cardiorespiratory fitness is an established predictor of metabolic disease and mortality.
Ultra Fast Speech Separation Model with Teacher Student Learning
In this paper, an ultra fast speech separation Transformer model is proposed to achieve both better performance and efficiency with teacher student learning (T-S learning).
Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?
Recently, self-supervised learning (SSL) has demonstrated strong performance in speaker recognition, even if the pre-training objective is designed for speech recognition.
A collaborative decomposition-based evolutionary algorithm integrating normal and penalty-based boundary intersection for many-objective optimization
On the other hand, the parallel reference lines of the parallel decomposition methods including the normal boundary intersection (NBI) might result in poor diversity because of under-sampling near the boundaries for MaOPs with concave frontiers.
Quantized GAN for Complex Music Generation from Dance Videos
We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates complex musical samples conditioned on dance videos.
Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings
The proposed speaker embedding, named t-vector, is extracted synchronously with the t-SOT ASR model, enabling joint execution of speaker identification (SID) or speaker diarization (SD) with the multi-talker transcription with low latency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Streaming Multi-Talker ASR with Token-Level Serialized Output Training
This paper proposes a token-level serialized output training (t-SOT), a novel framework for streaming multi-talker automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Multi-Query Video Retrieval
Retrieving target videos based on text descriptions is a task of great practical value and has received increasing attention over the past few years.
Large-Scale Video Panoptic Segmentation in the Wild: A Benchmark
In contrast, our large-scale VIdeo Panoptic Segmentation in the Wild (VIPSeg) dataset provides 3, 536 videos and 84, 750 frames with pixel-level panoptic annotations, covering a wide range of real-world scenarios and categories.
Learning To Learn by Jointly Optimizing Neural Architecture and Weights
Existing NAS-based meta-learning methods apply a two-stage strategy, i. e., first searching architectures and then re-training meta-weights on the searched architecture.
Self-Supervised Learning for speech recognition with Intermediate layer supervision
Recently, pioneer work finds that speech pre-trained models can solve full-stack speech processing tasks, because the model utilizes bottom layers to learn speaker-related information and top layers to encode content-related information.
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Self-supervised learning (SSL) achieves great success in speech recognition, while limited exploration has been attempted for other speech processing tasks.
Ranked #1 on
Speech Recognition
on CALLHOME En
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+8
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
We integrate the proposed methods into the HuBERT framework.
Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition
In this paper we propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech via contrastive learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition
ILMA enables a fast text-only adaptation of the E2E model without increasing the run-time computational cost.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Contrastive Video-Language Segmentation
We focus on the problem of segmenting a certain object referred by a natural language sentence in video content, at the core of formulating a pinpoint vision-language relation.
Cell2State: Learning Cell State Representations From Barcoded Single-Cell Gene-Expression Transitions
Genetic barcoding coupled with single-cell sequencing technology enables direct measurement of cell-to-cell transitions and gene-expression evolution over a long timespan.
Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation
To deal with this problem, instead of introducing knowledge base as the input, we force the model to learn a better semantic representation by predicting the information in the knowledge base, only based on the input context.
Detecting Speaker Personas from Conversational Texts
This is a many-to-many semantic matching task because both contexts and personas in SPD are composed of multiple sentences.
Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally.
UniSpeech at scale: An Empirical Study of Pre-training Method on Large-Scale Speech Recognition Dataset
Recently, there has been a vast interest in self-supervised learning (SSL) where the model is pre-trained on large scale unlabeled data and then fine-tuned on a small labeled dataset.
Investigation of Practical Aspects of Single Channel Speech Separation for ASR
Speech separation has been successfully applied as a frontend processing module of conversation transcription systems thanks to its ability to handle overlapped speech and its flexibility to combine with downstream tasks such as automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Saying the Unseen: Video Descriptions via Dialog Agents
Current vision and language tasks usually take complete visual data (e. g., raw images or videos) as input, however, practical scenarios may often consist the situations where part of the visual information becomes inaccessible due to various reasons e. g., restricted view with fixed camera or intentional vision block for security concerns.
VSPW: A Large-scale Dataset for Video Scene Parsing in the Wild
To the best of our knowledge, our VSPW is the first attempt to tackle the challenging video scene parsing task in the wild by considering diverse scenarios.
Exploring Heterogeneous Clues for Weakly-Supervised Audio-Visual Video Parsing
Previous works take the overall event labels to supervise both audio and visual model predictions.
Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition
In this work, we perform LM fusion in the minimum WER (MWER) training of an E2E model to obviate the need for LM weights tuning during inference.
Template-Based Named Entity Recognition Using BART
To address the issue, we propose a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively.
Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference.
Cooperative Path Planning of UAVs & UGVs for a Persistent Surveillance Task in Urban Environments
The hybrid EDA-GA algorithm can greatly improve the performance of EDA and GA in terms of the quality and the stability of solutions.
Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone
Transcribing meetings containing overlapped speech with only a single distant microphone (SDM) has been one of the most challenging problems for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation
Text-based video segmentation is a challenging task that segments out the natural language referred objects in videos.
Ranked #4 on
Referring Expression Segmentation
on J-HMDB
(Precision@0.9 metric)
Learning Audio-Visual Correlations from Variational Cross-Modal Generation
People can easily imagine the potential sound while seeing an event.
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
In this paper, we propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data, in which supervised phonetic CTC learning and phonetically-aware contrastive self-supervised learning are conducted in a multi-task learning manner.
Learning to Anticipate Egocentric Actions by Imagination
We further improve ImagineRNN by residual anticipation, i. e., changing its target to predicting the feature difference of adjacent frames instead of the frame content.
Connection-Adaptive Meta-Learning
In this paper, we aim to obtain better meta-learners by co-optimizing the architecture and meta-weights simultaneously.
Formality Style Transfer with Shared Latent Space
Conventional approaches for formality style transfer borrow models from neural machine translation, which typically requires massive parallel data for training.
Don't shoot butterfly with rifles: Multi-channel Continuous Speech Separation with Early Exit Transformer
With its strong modeling capacity that comes from a multi-head and multi-layer structure, Transformer is a very powerful model for learning a sequential representation and has been successfully applied to speech separation recently.
Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset
Recently, Transformer based end-to-end models have achieved great success in many areas including speech recognition.
Describing Unseen Videos via Multi-Modal Cooperative Dialog Agents
With the arising concerns for the AI systems provided with direct access to abundant sensitive information, researchers seek to develop more reliable AI with implicit information sources.
Continuous Speech Separation with Conformer
Continuous speech separation plays a vital role in complicated speech related tasks such as conversation transcription.
Ranked #1 on
Speech Separation
on LibriCSS
(using extra training data)
On Commonsense Cues in BERT for Solving Commonsense Tasks
We quantitatively investigate the presence of structural commonsense cues in BERT when solving commonsense tasks, and the importance of such cues for the model prediction.
A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation
The commonly used framework for unsupervised machine translation builds initial translation models of both translation directions, and then performs iterative back-translation to jointly boost their translation performance.
Abnormal activity capture from passenger flow of elevator based on unsupervised learning and fine-grained multi-label recognition
We present a work-flow which aims at capturing residents' abnormal activities through the passenger flow of elevator in multi-storey residence buildings.
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Among all three E2E models, transformer-AED achieved the best accuracy in both streaming and non-streaming mode.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Curriculum Pre-training for End-to-End Speech Translation
End-to-end speech translation poses a heavy burden on the encoder, because it has to transcribe, understand, and learn cross-lingual semantics simultaneously.
MuTual: A Dataset for Multi-Turn Dialogue Reasoning
Non-task oriented dialogue systems have achieved great success in recent years due to largely accessible conversation data and the development of deep learning techniques.
Unsupervised Person Re-identification via Softened Similarity Learning
Person re-identification (re-ID) is an important topic in computer vision.
Low Latency End-to-End Streaming Speech Recognition with a Scout Network
The attention-based Transformer model has achieved promising results for speech recognition (SR) in the offline mode.
Audio and Speech Processing
Symbiotic Attention with Privileged Information for Egocentric Action Recognition
Due to the large action vocabulary in egocentric video datasets, recent studies usually utilize a two-branch structure for action recognition, ie, one branch for verb classification and the other branch for noun classification.
Ranked #4 on
Egocentric Activity Recognition
on EGTEA
Semantic Mask for Transformer based End-to-End Speech Recognition
Attention-based encoder-decoder model has achieved impressive results for both automatic speech recognition (ASR) and text-to-speech (TTS) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer
Formality text style transfer plays an important role in various NLP applications, such as non-native speaker assistants and child education.
Unsupervised Context Rewriting for Open Domain Conversation
Context modeling has a pivotal role in open domain conversation.
Dual Attention Matching for Audio-Visual Event Localization
The duration of these segments is usually short, making the visual and acoustic feature of each segment possibly not well aligned.
Gated Channel Transformation for Visual Recognition
This lightweight layer incorporates a simple l2 normalization, enabling our transformation unit applicable to operator-level without much increase of additional parameters.
Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation
End-to-end speech translation, a hot topic in recent years, aims to translate a segment of audio into a specific language with an end-to-end model.
Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
Pre-training has proven to be effective in unsupervised machine translation due to its ability to model deep context information in cross-lingual scenarios.
Cascaded Revision Network for Novel Object Captioning
By this novel cascaded captioning-revising mechanism, CRN can accurately describe images with unseen objects.
Baidu-UTS Submission to the EPIC-Kitchens Action Recognition Challenge 2019
In this report, we present the Baidu-UTS submission to the EPIC-Kitchens Action Recognition Challenge in CVPR 2019.
Revisiting EmbodiedQA: A Simple Baseline and Beyond
In this paper, we empirically study this problem and introduce 1) a simple yet effective baseline that achieves promising performance; 2) an easier and practical setting for EmbodiedQA where an agent has a chance to adapt the trained model to a new environment before it actually answers users questions.
Auto-ReID: Searching for a Part-aware ConvNet for Person Re-Identification
We propose to automatically search for a CNN architecture that is specifically suitable for the reID task.
Ranked #9 on
Person Re-Identification
on CUHK03 detected
Text Morphing
In this paper, we introduce a novel natural language generation task, termed as text morphing, which targets at generating the intermediate sentences that are fluency and smooth with the two input sentences.
Neural Melody Composition from Lyrics
In this paper, we study a novel task that learns to compose music from natural language.
Keyphrase Generation with Correlation Constraints
In this paper, we study automatic keyphrase generation.
Towards Explainable and Controllable Open Domain Dialogue Generation with Dialogue Acts
We study open domain dialogue generation with dialogue acts designed to explain how people engage in social chat.
Dictionary-Guided Editing Networks for Paraphrase Generation
An intuitive way for a human to write paraphrase sentences is to replace words or phrases in the original sentence with their corresponding synonyms and make necessary changes to ensure the new sentences are fluent and grammatically correct.
Exploit the Unknown Gradually: One-Shot Video-Based Person Re-Identification by Stepwise Learning
We focus on the one-shot learning for video-based person re-Identification (re-ID).
Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots
We propose a method that can leverage unlabeled data to learn a matching model for response selection in retrieval-based chatbots.
Decoupled Novel Object Captioner
Thus, the sequence model can be decoupled from the novel object descriptions.
Towards Interpretable Chit-chat: Open Domain Dialogue Generation with Dialogue Acts
Conventional methods model open domain dialogue generation as a black box through end-to-end learning from large scale conversation data.
Neural Response Generation with Dynamic Vocabularies
We study response generation for open domain conversation in chatbots.
A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
The task requires matching a response candidate with a conversation context, whose challenges include how to recognize important parts of the context, and how to model the relationships among utterances in the context.
Beihang-MSRA at SemEval-2017 Task 3: A Ranking System with Neural Matching Features for Community Question Answering
This paper presents the system in SemEval-2017 Task 3, Community Question Answering (CQA).
Non-Convex Weighted Lp Nuclear Norm based ADMM Framework for Image Restoration
Since the matrix formed by nonlocal similar patches in a natural image is of low rank, the nuclear norm minimization (NNM) has been widely used in various image processing studies.
Non-Convex Weighted Lp Minimization based Group Sparse Representation Framework for Image Denoising
Nonlocal image representation or group sparsity has attracted considerable interest in various low-level vision tasks and has led to several state-of-the-art image denoising techniques, such as BM3D, LSSC.
Improving Person Re-identification by Attribute and Identity Learning
Person re-identification (re-ID) and attribute recognition share a common target at learning pedestrian descriptions.
Ranked #78 on
Person Re-Identification
on DukeMTMC-reID
Hierarchical Recurrent Attention Network for Response Generation
With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context.
Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots
Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or important contextual information.
Ranked #7 on
Conversational Response Selection
on RRS
Knowledge Enhanced Hybrid Neural Network for Text Matching
Long text brings a big challenge to semantic matching due to their complicated semantic and syntactic structures.
Detecting Context Dependent Messages in a Conversational Environment
While automatic response generation for building chatbot systems has drawn a lot of attention recently, there is limited understanding on when we need to consider the linguistic context of an input text in the generation process.
Topic Aware Neural Response Generation
We consider incorporating topic information into the sequence-to-sequence framework to generate informative and interesting responses for chatbots.