Search Results for author:
Found 71 papers, 26 papers with code
PALM: Pre-training an Autoencoding\&Autoregressive Language Model for Context-conditioned Generation
An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.
Abstractive Text Summarization
Conversational Response Generation
+8
CLIP-VG: Self-paced Curriculum Adapting of CLIP via Exploiting Pseudo-Language Labels for Visual Grounding
Visual Grounding (VG) refers to locating a region described by expressions in a specific image, which is a critical topic in vision-language fields.
Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering
Existing knowledge-enhanced methods have achieved remarkable results in certain QA tasks via obtaining diverse knowledge from different knowledge bases.
AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference
To support cost-effective language inference in multilingual settings, we propose AMTSS, an adaptive multi-teacher single-student distillation framework, which allows distilling knowledge from multiple teachers to a single student.
Transforming Visual Scene Graphs to Image Captions
In TSG, we apply multi-head attention (MHA) to design the Graph Neural Network (GNN) for embedding scene graphs.
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
Our code, pre-trained model, instruction-tuned models, and evaluation set are available at https://github. com/X-PLUG/mPLUG-Owl.
From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping
Although previous works have created generation capacity for CLIP through additional language models, a modality gap between the CLIP representations of different modalities and the inability of CLIP to model the offset of this gap, which fails the concept to transfer across modalities.
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.
Improved Visual Fine-tuning with Natural Language Supervision
Fine-tuning a pre-trained model can leverage the semantic information from large-scale pre-training data and mitigate the over-fitting problem on downstream tasks with limited training examples.
CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions
The core of this dataset is a blending optimization process, which corrects for the pose as it drifts and is affected by the magnetic conditions.
Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image Retrieval
Thanks to the ISS and CCA, our method could encode the discrimination into the domain-invariant embedding space for unsupervised cross-domain image retrieval.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.
Ranked #1 on
Visual Grounding
on RefCOCO+ testA
Adaptively Clustering Neighbor Elements for Image Captioning
We design a novel global-local Transformer named \textbf{Ada-ClustFormer} (\textbf{ACF}) to generate captions.
Learning Trajectory-Word Alignments for Video-Language Tasks
To amend this, we propose a novel TW-BERT to learn Trajectory-Word alignment by a newly designed trajectory-to-word (T2W) attention for solving video-language tasks.
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e. g., SSv2-Template and SSv2-Label) with 8. 6% and 11. 1% improvement respectively.
Ranked #1 on
Visual Question Answering (VQA)
on TGIF-QA
FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction
Most cross-device federated learning (FL) studies focus on the model-homogeneous setting where the global server model and local client models are identical.
Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment
We further propose Anchor Augment to guide the generative model's attention to the fine-grained information in the representation of CLIP.
Eye-tracking based classification of Mandarin Chinese readers with and without dyslexia using neural sequence models
Eye movements are known to reflect cognitive processes in reading, and psychological reading research has shown that eye gaze patterns differ between readers with and without dyslexia.
Communication-Efficient Topologies for Decentralized Learning with $O(1)$ Consensus Rate
In the proposed family, EquiStatic has a degree of $\Theta(\ln(n))$, where $n$ is the network size, and a series of time-dependent one-peer topologies, EquiDyn, has a constant degree of 1.
Construction and Applications of Billion-Scale Pre-Trained Multimodal Business Knowledge Graph
We release all the open resources (OpenBG benchmarks) derived from it for the community and report experimental results of KG-centric tasks.
DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning
We evaluate our approach on a variety of knowledge driven and language understanding tasks, including NER, relation extraction, CommonsenseQA, OpenBookQA and GLUE.
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval
However, cross-grained contrast, which is the contrast between coarse-grained representations and fine-grained representations, has rarely been explored in prior research.
Ranked #10 on
Video Retrieval
on MSVD
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
Ranked #1 on
Image Captioning
on COCO Captions
WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types
In this paper, we present WikiDiverse, a high-quality human-annotated MEL dataset with diversified contextual topics and entity types from Wikinews, which uses Wikipedia as the corresponding knowledge base.
Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding
Moreover, since the backbones are query-agnostic, it is difficult to completely avoid the inconsistency issue by training the visual backbone end-to-end in the visual grounding framework.
Learning Deep Semantic Model for Code Search using CodeSearchNet Corpus
Different from typical information retrieval tasks, code search requires to bridge the semantic gap between the programming language and natural language, for better describing intrinsic concepts and semantics.
Achieving Human Parity on Visual Question Answering
The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image.
Ranked #12 on
Visual Question Answering (VQA)
on VQA v2 test-dev
Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data
Recent advancements in end-to-end speech synthesis have made it possible to generate highly natural speech.
Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training
Existing approaches to vision-language pre-training (VLP) heavily rely on an object detector based on bounding boxes (regions), where salient objects are first detected from images and then a Transformer-based model is used for cross-modal fusion.
Decentralized Composite Optimization with Compression
Decentralized optimization and communication compression have exhibited their great potential in accelerating distributed machine learning by mitigating the communication bottleneck in practice.
MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion
We describe our systems of subtask1 and subtask3 for SemEval-2021 Task 6 on Detection of Persuasion Techniques in Texts and Images.
Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction
This work focuses on generative QA which aims to generate an abstractive answer to a given question instead of extracting an answer span from a provided passage.
Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs
In this work, we consider the decentralized optimization problem in which a network of $n$ agents, each possessing a smooth and convex objective function, wish to collaboratively minimize the average of all the objective functions through peer-to-peer communication in a directed graph.
Elastic Graph Neural Networks
While many existing graph neural networks (GNNs) have been proven to perform $\ell_2$-based graph smoothing that enforces smoothness globally, in this work we aim to further enhance the local smoothness adaptivity of GNNs via $\ell_1$-based graph smoothing.
Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks
The second algorithm is a broadcast-like version of CPP (B-CPP), and it also achieves linear convergence rate under the same conditions on the objective functions.
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning
Vision-language pre-training (VLP) on large-scale image-text pairs has achieved huge success for the cross-modal downstream tasks.
StructuralLM: Structural Pre-training for Form Understanding
Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks.
RCT: Resource Constrained Training for Edge AI
For example, quantisation-aware training (QAT) method involves two copies of model parameters, which is usually beyond the capacity of on-chip memory in edge devices.
SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels
Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations.
CoRe: An Efficient Coarse-refined Training Framework for BERT
In the second phase, we transform the trained relaxed BERT model into the original BERT and further retrain the model.
Deep Neural Networks with Short Circuits for Improved Gradient Learning
Deep neural networks have achieved great success both in computer vision and natural language processing tasks.
Fast algorithms for robust principal component analysis with an upper bound on the rank
The first type of algorithm applies regularization terms on the singular values of a matrix to obtain a low-rank matrix.
Linear Convergent Decentralized Optimization with Compression
Communication compression has become a key strategy to speed up distributed optimization.
Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering
Multiple-choice question answering (MCQA) is one of the most challenging tasks in machine reading comprehension since it requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations.
Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm
However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth.
A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features
This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents.
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.
Ranked #1 on
Text Generation
on CNN/Daily Mail
Abstractive Text Summarization
Conversational Response Generation
+8
Manifold Denoising by Nonlinear Robust Principal Component Analysis
This paper extends robust principal component analysis (RPCA) to nonlinear manifolds.
A Double Residual Compression Algorithm for Efficient Distributed Learning
Large-scale machine learning models are often trained by parallel stochastic gradient descent algorithms.
Symmetric Regularization based BERT for Pair-wise Semantic Reasoning
To remedy this, we propose to augment the NSP task to a 3-class categorization task, which includes a category for previous sentence prediction (PSP).
Incorporating External Knowledge into Machine Reading for Generative Question Answering
Different from existing work on knowledge-aware QA, we focus on a more challenging task of leveraging external knowledge to generate answers in natural language for a given question with context.
Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning
This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning.
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
Recently, the pre-trained language model, BERT (and its robustly optimized version RoBERTa), has attracted a lot of attention in natural language understanding (NLU), and achieved state-of-the-art accuracy in various NLU tasks, such as sentiment classification, natural language inference, semantic textual similarity and question answering.
Ranked #1 on
Natural Language Inference
on QNLI
On linear convergence of two decentralized algorithms
In addition, we relax the requirement for the objective functions and the mixing matrices.
Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering
Extensive experiments on the large-scale SQuAD and TriviaQA datasets validate the effectiveness of the proposed method.
A Deep Cascade Model for Multi-Document Reading Comprehension
To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the document-level and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension.
Ranked #2 on
Question Answering
on MS MARCO
$D^2$: Decentralized Training over Decentralized Data
While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are unique and different.
Ranked #3 on
Multi-view Subspace Clustering
on ORL
D$^2$: Decentralized Training over Decentralized Data
While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}.
Exploring Outliers in Crowdsourced Ranking for QoE
Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years.
Nonconvex penalties with analytical solutions for one-bit compressive sensing
For several nonconvex penalties, including minimax concave penalty (MCP), $\ell_0$ norm, and sorted $\ell_1$ penalty, we provide fast algorithms for finding the analytical solutions by solving the dual problem.
A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates
This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms.
Mixed one-bit compressive sensing with applications to overexposure correction for CT reconstruction
Aiming at overexposure correction for computed tomography (CT) reconstruction, we in this paper propose a mixed one-bit compressive sensing (M1bit-CS) to acquire information from both regular and saturated measurements.
On the Convergence of Asynchronous Parallel Iteration with Unbounded Delays
Recent years have witnessed the surge of asynchronous parallel (async-parallel) iterative algorithms due to problems involving very large-scale data and a large number of decision variables.
A new primal-dual algorithm for minimizing the sum of three functions with a linear operator
For the general convex case, we prove the convergence of this new algorithm in terms of the distance to a fixed point by showing that the iteration is a nonexpansive operator.
Asynchronous Multi-Task Learning
The models for each hospital may be different because of the inherent differences in the distributions of the patient populations.
Coordinate Friendly Structures, Algorithms and Applications
To derive simple subproblems for several new classes of applications, this paper systematically studies coordinate-friendly operators that perform low-cost coordinate updates.
ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates
The agents share $x$ through either global memory or communication.
Pinball Loss Minimization for One-bit Compressive Sensing: Convex Models and Algorithms
The one-sided $\ell_1$ loss and the linear loss are two popular loss functions for 1bit-CS.
A Multiphase Image Segmentation Based on Fuzzy Membership Functions and L1-norm Fidelity
In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity.
The Continuity of Images by Transmission Imaging Revisited
In transmission imaging, it was shown very recently in [49] that almost all images are continuous functions.
Restoration of Images Corrupted by Impulse Noise and Mixed Gaussian Impulse Noise using Blind Inpainting
In addition, we provide convergence analysis for these methods, these algorithms will converge to coordinatewise minimum points.
