Search Results for author:
Found 51 papers, 20 papers with code
Variational Connectionist Temporal Classification
However, CTC tends to output spiky distributions since it prefers to output blank symbol most of the time.
SNP-S3: Shared Network Pre-training and Significant Semantic Strengthening for Various Video-Text Tasks
By employing one shared BERT-type network to refine textual and cross-modal features simultaneously, SNP is lightweight and could support various downstream applications.
Knowledge-enhanced Multi-perspective Video Representation Learning for Scene Recognition
With the explosive growth of video data in real-world applications, a comprehensive representation of videos becomes increasingly important.
InfMLLM: A Unified Framework for Visual-Language Tasks
Our experiments demonstrate that preserving the positional information of visual embeddings through the pool-adapter is particularly beneficial for tasks like visual grounding.
Ranked #60 on
Visual Question Answering
on MM-Vet
LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints
Integrating first-order logic constraints (FOLCs) with neural networks is a crucial but challenging problem since it involves modeling intricate correlations to satisfy the constraints.
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning
We thereby present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to construct partial order triplet samples by automatically generating fine-grained hard negatives for matched text-video pairs.
Ranked #4 on
Video Retrieval
on MSR-VTT-1kA
Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrieval
SSAN is based on two newly proposed modules in video retrieval: (1) An efficient Self-supervised Keyframe Extraction (SKE) module to reduce redundant frame features, (2) A robust Similarity Pattern Detection (SPD) module for temporal alignment.
Switch-BERT: Learning to Model Multimodal Interactions by Switching Attention and Input
They can achieve exceptional performances on specific tasks, but face a particularly challenging problem of modality mismatch because of diversity of input modalities and their fixed structures.
Boundary-aware Backward-Compatible Representation via Adversarial Learning in Image Retrieval
For traditional model upgrades, the old model will not be replaced by the new one until the embeddings of all the images in the database are re-computed by the new model, which takes days or weeks for a large amount of data.
A CTC Alignment-based Non-autoregressive Transformer for End-to-end Automatic Speech Recognition
During inference, an error-based alignment sampling method is investigated in depth to reduce the alignment mismatch in the training and testing processes.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
DC-Former: Diverse and Compact Transformer for Person Re-Identification
In this paper, we propose a Diverse and Compact Transformer (DC-Former) that can achieve a similar effect by splitting embedding space into multiple diverse and compact subspaces.
DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional Networks
Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs and outperforms the state-of-the-art (SOTA) methods on various benchmark datasets.
Causal Effect Estimation: Recent Advances, Challenges, and Opportunities
Causal inference has numerous real-world applications in many domains, such as health care, marketing, political science, and online advertising.
Simultaneously Short- and Long-Term Temporal Modeling for Semi-Supervised Video Semantic Segmentation
In this work, we propose a novel feature enhancement network to simultaneously model short- and long-term temporal correlation.
Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation
Contrastive learning has achieved impressive success in generation tasks to militate the "exposure bias" problem and discriminatively exploit the different quality of references.
PIE: a Parameter and Inference Efficient Solution for Large Scale Knowledge Graph Embedding Reasoning
Meanwhile, the inference time grows log-linearly with the number of entities for all entities are traversed and compared.
ESCM$^2$: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation
In this paper, we theoretically demonstrate that ESMM suffers from the following two problems: (1) Inherent Estimation Bias (IEB), where the estimated CVR of ESMM is inherently higher than the ground truth; (2) Potential Independence Priority (PIP) for CTCVR estimation, where there is a risk that the ESMM overlooks the causality from click to conversion.
Training Protocol Matters: Towards Accurate Scene Text Recognition via Training Protocol Searching
The development of scene text recognition (STR) in the era of deep learning has been mainly focused on novel architectures of STR models.
Training Object Detectors From Scratch: An Empirical Study in the Era of Vision Transformer
Instead of proposing a specific vision transformer based detector, in this work, our goal is to reveal the insights of training vision transformer based detectors from scratch.
CBNet: A Composite Backbone Network Architecture for Object Detection
With multi-scale testing, we push the current best single model result to a new record of 60. 1% box AP and 52. 3% mask AP without using extra training data.
Ranked #6 on
Object Detection
on COCO-O
(using extra training data)
Discrimination-Aware Mechanism for Fine-Grained Representation Learning
Therefore, there is a need for a training mechanism that enforces the discriminativeness of all the elements in the feature to capture more the subtle visual cues.
LPSNet: A Lightweight Solution for Fast Panoptic Segmentation
Panoptic segmentation is a challenging task aiming to simultaneously segment objects (things) at instance level and background contents (stuff) at semantic level.
An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition
For the analyses, we plot attention weight distributions in the decoders to visualize the relationships between token-level acoustic embeddings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Low Resource German ASR with Untranscribed Data Spoken by Non-native Children -- INTERSPEECH 2021 Shared Task SPAPL System
~ 5 hours of transcribed data and ~ 60 hours of untranscribed data are provided to develop a German ASR system for children.
CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes
Then, we design a two-level perturbation fusion strategy to alleviate the conflict between the adversarial watermarks generated by different facial images and models.
Towards Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning
The ambiguous annotation criteria lead to divergence of Chinese Word Segmentation (CWS) datasets in various granularities.
PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
Distance based knowledge graph embedding methods show promising results on link prediction task, on which two topics have been widely studied: one is the ability to handle complex relations, such as N-to-1, 1-to-N and N-to-N, the other is to encode various relation patterns, such as symmetry/antisymmetry.
Ranked #12 on
Link Property Prediction
on ogbl-biokg
CASS-NAT: CTC Alignment-based Single Step Non-autoregressive Transformer for Speech Recognition
The information are used to extract acoustic representation for each token in parallel, referred to as token-level acoustic embedding which substitutes the word embedding in autoregressive transformer (AT) to achieve parallel generation in decoder.
Question Directed Graph Attention Network for Numerical Reasoning over Text
Numerical reasoning over texts, such as addition, subtraction, sorting and counting, is a challenging machine reading comprehension task, since it requires both natural language understanding and arithmetic computation.
Ranked #1 on
Question Answering
on DROP Test
Generating Informative Conversational Response using Recurrent Knowledge-Interaction and Knowledge-Copy
Experiments demonstrate that our model with fewer parameters yields significant improvements over competitive baselines on two datasets Wizard-of-Wikipedia(average Bleu +87{\%}; abs.
Riemannian Proximal Policy Optimization
In this paper, We propose a general Riemannian proximal optimization algorithm with guaranteed convergence to solve Markov decision process (MDP) problems.
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
This paper proposes to incorporate phonological and visual similarity knowledge into language models for CSC via a specialized graph convolutional network (SpellGCN).
Variational Policy Propagation for Multi-agent Reinforcement Learning
We propose a \emph{collaborative} multi-agent reinforcement learning algorithm named variational policy propagation (VPP) to learn a \emph{joint} policy through the interactions over agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Symmetric Regularization based BERT for Pair-wise Semantic Reasoning
To remedy this, we propose to augment the NSP task to a 3-class categorization task, which includes a category for previous sentence prediction (PSP).
BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction
Combining these three contributions, we enhance the information extracting ability of the multi-head selection model and achieve F1-score 0. 876 on testset-1 with a single model.
Singing voice conversion with non-parallel data
F0 and aperiodic are obtained through the original singing voice, and used with acoustic features to reconstruct the target singing voice through a vocoder.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning
The ambiguous annotation criteria lead to divergence of Chinese Word Segmentation (CWS) datasets in various granularities.
BandNet: A Neural Network-based, Multi-Instrument Beatles-Style MIDI Music Composition Machine
In the learning stage, a sequence of stylistically uniform, multiple-channel music samples was modeled by a RNN.
Reinforcement Learning for Uplift Modeling
Uplift modeling aims to directly model the incremental impact of a treatment on an individual response.
A Novel Integrated Framework for Learning both Text Detection and Recognition
In this paper, we propose a novel integrated framework for learning both text detection and recognition.
Variational Semi-supervised Aspect-term Sentiment Analysis via Transformer
By disentangling the latent representation into the aspect-specific sentiment and the lexical context, our method induces the underlying sentiment prediction for the unlabeled data, which then benefits the ATSA classifier.
Aspect-Based Sentiment Analysis (ABSA)
Natural Language Understanding
+1
Latent Dirichlet Allocation for Internet Price War
Then we develop a variant of Latent Dirichlet Allocation (LDA) to infer latent variables under the current market environment, which represents the preferences of customers and strategies of competitors.
Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units
In this paper, we present an end-to-end automatic speech recognition system, which successfully employs subword units in a hybrid CTC-Attention based system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
IncepText: A New Inception-Text Module with Deformable PSROI Pooling for Multi-Oriented Scene Text Detection
To solve this problem, we propose a novel end-to-end scene text detector IncepText from an instance-aware segmentation perspective.
AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience
We present AliMe Assist, an intelligent assistant designed for creating an innovative online shopping experience in E-commerce.
Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce
In this paper, we study transfer learning for the PI and NLI problems, aiming to propose a general framework, which can effectively and efficiently adapt the shared knowledge learned from a resource-rich source domain to a resource- poor target domain.
AliMe Chat: A Sequence to Sequence and Rerank based Chatbot Engine
We propose AliMe Chat, an open-domain chatbot engine that integrates the joint results of Information Retrieval (IR) and Sequence to Sequence (Seq2Seq) based generation models.
Speaker Cluster-Based Speaker Adaptive Training for Deep Neural Network Acoustic Modeling
The previously trained DNN of the matched speaker cluster is used for decoding utterances of the test speaker.
Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms
\emph{Offline} evaluation of the effectiveness of new algorithms in these applications is critical for protecting online user experiences but very challenging due to their "partial-label" nature.
A Contextual-Bandit Approach to Personalized News Article Recommendation
In this work, we model personalized recommendation of news articles as a contextual bandit problem, a principled approach in which a learning algorithm sequentially selects articles to serve users based on contextual information about the users and articles, while simultaneously adapting its article-selection strategy based on user-click feedback to maximize total user clicks.
Gaussian Process Models for Link Analysis and Transfer Learning
In this paper we develop a Gaussian process (GP) framework to model a collection of reciprocal random variables defined on the \emph{edges} of a network.
