Search Results for author: Ming Yan

Found 71 papers, 26 papers with code

PALM: Pre-training an Autoencoding\&Autoregressive Language Model for Context-conditioned Generation

no code implementations EMNLP 2020 Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.

Abstractive Text Summarization Conversational Response Generation +8

CLIP-VG: Self-paced Curriculum Adapting of CLIP via Exploiting Pseudo-Language Labels for Visual Grounding

no code implementations15 May 2023 Linhui Xiao, Xiaoshan Yang, Fang Peng, Ming Yan, YaoWei Wang, Changsheng Xu

Visual Grounding (VG) refers to locating a region described by expressions in a specific image, which is a critical topic in vision-language fields.

Denoising Transfer Learning +1

Distinguish Before Answer: Generating Contrastive Explanation as Knowledge for Commonsense Question Answering

no code implementations14 May 2023 Qianglong Chen, Guohai Xu, Ming Yan, Ji Zhang, Fei Huang, Luo Si, Yin Zhang

Existing knowledge-enhanced methods have achieved remarkable results in certain QA tasks via obtaining diverse knowledge from different knowledge bases.

Explanation Generation Question Answering

AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference

no code implementations13 May 2023 Qianglong Chen, Feng Ji, Feng-Lin Li, Guohai Xu, Ming Yan, Ji Zhang, Yin Zhang

To support cost-effective language inference in multilingual settings, we propose AMTSS, an adaptive multi-teacher single-student distillation framework, which allows distilling knowledge from multiple teachers to a single student.

Knowledge Distillation

Transforming Visual Scene Graphs to Image Captions

no code implementations3 May 2023 Xu Yang, Jiawei Peng, Zihua Wang, Haiyang Xu, Qinghao Ye, Chenliang Li, Ming Yan, Fei Huang, Zhangzikang Li, Yu Zhang

In TSG, we apply multi-head attention (MHA) to design the Graph Neural Network (GNN) for embedding scene graphs.

Image Captioning

mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality

1 code implementation27 Apr 2023 Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Ji Zhang, Fei Huang

Our code, pre-trained model, instruction-tuned models, and evaluation set are available at https://github. com/X-PLUG/mPLUG-Owl.

From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping

1 code implementation26 Apr 2023 Junyang Wang, Ming Yan, Yi Zhang, Jitao Sang

Although previous works have created generation capacity for CLIP through additional language models, a modality gap between the CLIP representations of different modalities and the inability of CLIP to model the offset of this gap, which fails the concept to transfer across modalities.

Image Captioning Image Classification +3

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

1 code implementation16 Apr 2023 Junfeng Tian, Hehong Chen, Guohai Xu, Ming Yan, Xing Gao, Jianhai Zhang, Chenliang Li, Jiayi Liu, Wenshen Xu, Haiyang Xu, Qi Qian, Wei Wang, Qinghao Ye, Jiejing Zhang, Ji Zhang, Fei Huang, Jingren Zhou

In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format.

Improved Visual Fine-tuning with Natural Language Supervision

no code implementations4 Apr 2023 Junyang Wang, Yuanhong Xu, Juhua Hu, Ming Yan, Jitao Sang, Qi Qian

Fine-tuning a pre-trained model can leverage the semantic information from large-scale pre-training data and mitigate the over-fitting problem on downstream tasks with limited training examples.

CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions

no code implementations CVPR 2023 Ming Yan, Xin Wang, Yudi Dai, Siqi Shen, Chenglu Wen, Lan Xu, Yuexin Ma, Cheng Wang

The core of this dataset is a blending optimization process, which corrects for the pose as it drifts and is affected by the magnetic conditions.

Pose Prediction

Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image Retrieval

1 code implementation13 Feb 2023 Xu Wang, Dezhong Peng, Ming Yan, Peng Hu

Thanks to the ISS and CCA, our method could encode the discrimination into the domain-invariant embedding space for unsupervised cross-domain image retrieval.

Image Retrieval Retrieval

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video

4 code implementations1 Feb 2023 Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, Guohai Xu, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou

In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based instance discrimination, mPLUG-2 introduces a multi-module composition network by sharing common universal modules for modality collaboration and disentangling different modality modules to deal with modality entanglement.

Action Classification Image Classification +7

Learning Trajectory-Word Alignments for Video-Language Tasks

no code implementations5 Jan 2023 Xu Yang, Zhangzikang Li, Haiyang Xu, Hanwang Zhang, Qinghao Ye, Chenliang Li, Ming Yan, Yu Zhang, Fei Huang, Songfang Huang

To amend this, we propose a novel TW-BERT to learn Trajectory-Word alignment by a newly designed trajectory-to-word (T2W) attention for solving video-language tasks.

Question Answering Retrieval +4

HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training

no code implementations30 Dec 2022 Qinghao Ye, Guohai Xu, Ming Yan, Haiyang Xu, Qi Qian, Ji Zhang, Fei Huang

We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks, especially on temporal-oriented datasets (e. g., SSv2-Template and SSv2-Label) with 8. 6% and 11. 1% improvement respectively.

TGIF-Action TGIF-Frame +7

FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction

2 code implementations3 Dec 2022 Samiul Alam, Luyang Liu, Ming Yan, Mi Zhang

Most cross-device federated learning (FL) studies focus on the model-homogeneous setting where the global server model and local client models are identical.

Federated Learning Model extraction

Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment

no code implementations14 Nov 2022 Junyang Wang, Yi Zhang, Ming Yan, Ji Zhang, Jitao Sang

We further propose Anchor Augment to guide the generative model's attention to the fine-grained information in the representation of CLIP.

Image Captioning Image Retrieval +1

Eye-tracking based classification of Mandarin Chinese readers with and without dyslexia using neural sequence models

1 code implementation18 Oct 2022 Patrick Haller, Andreas Säuberli, Sarah Elisabeth Kiener, Jinger Pan, Ming Yan, Lena Jäger

Eye movements are known to reflect cognitive processes in reading, and psychological reading research has shown that eye gaze patterns differ between readers with and without dyslexia.

Word Embeddings

Communication-Efficient Topologies for Decentralized Learning with $O(1)$ Consensus Rate

1 code implementation14 Oct 2022 Zhuoqing Song, Weijian Li, Kexin Jin, Lei Shi, Ming Yan, Wotao Yin, Kun Yuan

In the proposed family, EquiStatic has a degree of $\Theta(\ln(n))$, where $n$ is the network size, and a series of time-dependent one-peer topologies, EquiDyn, has a constant degree of 1.

DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning

no code implementations1 Aug 2022 Qianglong Chen, Feng-Lin Li, Guohai Xu, Ming Yan, Ji Zhang, Yin Zhang

We evaluate our approach on a variety of knowledge driven and language understanding tasks, including NER, relation extraction, CommonsenseQA, OpenBookQA and GLUE.

Contrastive Learning Language Modelling +2

X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval

1 code implementation15 Jul 2022 Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, Rongrong Ji

However, cross-grained contrast, which is the contrast between coarse-grained representations and fine-grained representations, has rarely been explored in prior research.

Contrastive Learning Retrieval +2

mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections

2 code implementations24 May 2022 Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, Hehong Chen, Guohai Xu, Zheng Cao, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou, Luo Si

Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.

Image Captioning Question Answering +5

WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types

1 code implementation ACL 2022 Xuwu Wang, Junfeng Tian, Min Gui, Zhixu Li, Rui Wang, Ming Yan, Lihan Chen, Yanghua Xiao

In this paper, we present WikiDiverse, a high-quality human-annotated MEL dataset with diversified contextual topics and entity types from Wikinews, which uses Wikipedia as the corresponding knowledge base.

Entity Linking

Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding

1 code implementation CVPR 2022 Jiabo Ye, Junfeng Tian, Ming Yan, Xiaoshan Yang, Xuwu Wang, Ji Zhang, Liang He, Xin Lin

Moreover, since the backbones are query-agnostic, it is difficult to completely avoid the inconsistency issue by training the visual backbone end-to-end in the visual grounding framework.

Visual Grounding

Learning Deep Semantic Model for Code Search using CodeSearchNet Corpus

1 code implementation27 Jan 2022 Chen Wu, Ming Yan

Different from typical information retrieval tasks, code search requires to bridge the semantic gap between the programming language and natural language, for better describing intrinsic concepts and semantics.

Code Search Information Retrieval +4

Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training

no code implementations21 Aug 2021 Ming Yan, Haiyang Xu, Chenliang Li, Bin Bi, Junfeng Tian, Min Gui, Wei Wang

Existing approaches to vision-language pre-training (VLP) heavily rely on an object detector based on bounding boxes (regions), where salient objects are first detected from images and then a Transformer-based model is used for cross-modal fusion.

object-detection Object Detection

Decentralized Composite Optimization with Compression

no code implementations10 Aug 2021 Yao Li, Xiaorui Liu, Jiliang Tang, Ming Yan, Kun Yuan

Decentralized optimization and communication compression have exhibited their great potential in accelerating distributed machine learning by mitigating the communication bottleneck in practice.

Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction

no code implementations ACL 2021 Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang

This work focuses on generative QA which aims to generate an abstractive answer to a given question instead of extracting an answer span from a provided passage.

Generative Question Answering Machine Reading Comprehension

Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs

no code implementations26 Jul 2021 Zhuoqing Song, Lei Shi, Shi Pu, Ming Yan

In this work, we consider the decentralized optimization problem in which a network of $n$ agents, each possessing a smooth and convex objective function, wish to collaboratively minimize the average of all the objective functions through peer-to-peer communication in a directed graph.

Elastic Graph Neural Networks

1 code implementation5 Jul 2021 Xiaorui Liu, Wei Jin, Yao Ma, Yaxin Li, Hua Liu, Yiqi Wang, Ming Yan, Jiliang Tang

While many existing graph neural networks (GNNs) have been proven to perform $\ell_2$-based graph smoothing that enforces smoothness globally, in this work we aim to further enhance the local smoothness adaptivity of GNNs via $\ell_1$-based graph smoothing.

Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks

no code implementations14 Jun 2021 Zhuoqing Song, Lei Shi, Shi Pu, Ming Yan

The second algorithm is a broadcast-like version of CPP (B-CPP), and it also achieves linear convergence rate under the same conditions on the objective functions.

RCT: Resource Constrained Training for Edge AI

no code implementations26 Mar 2021 Tian Huang, Tao Luo, Ming Yan, Joey Tianyi Zhou, Rick Goh

For example, quantisation-aware training (QAT) method involves two copies of model parameters, which is usually beyond the capacity of on-chip memory in edge devices.

SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels

no code implementations14 Mar 2021 Chenliang Li, Ming Yan, Haiyang Xu, Fuli Luo, Wei Wang, Bin Bi, Songfang Huang

Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations.

CoRe: An Efficient Coarse-refined Training Framework for BERT

no code implementations27 Nov 2020 Cheng Yang, Shengnan Wang, Yuechuan Li, Chao Yang, Ming Yan, Jingqiao Zhang, Fangquan Lin

In the second phase, we transform the trained relaxed BERT model into the original BERT and further retrain the model.

Deep Neural Networks with Short Circuits for Improved Gradient Learning

no code implementations23 Sep 2020 Ming Yan, Xueli Xiao, Joey Tianyi Zhou, Yi Pan

Deep neural networks have achieved great success both in computer vision and natural language processing tasks.

Fast algorithms for robust principal component analysis with an upper bound on the rank

1 code implementation18 Aug 2020 Ningyu Sha, Lei Shi, Ming Yan

The first type of algorithm applies regularization terms on the singular values of a matrix to obtain a low-rank matrix.

Vocal Bursts Type Prediction

Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering

no code implementations ACL 2020 Ming Yan, Hao Zhang, Di Jin, Joey Tianyi Zhou

Multiple-choice question answering (MCQA) is one of the most challenging tasks in machine reading comprehension since it requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations.

Logical Reasoning Machine Reading Comprehension +4

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

1 code implementation23 Jun 2020 Xueli Xiao, Ming Yan, Sunitha Basodi, Chunyan Ji, Yi Pan

However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth.

Hyperparameter Optimization

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

no code implementations15 Jun 2020 Sulaiman A. Alghunaim, Ming Yan, Ali H. Sayed

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents.


PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

2 code implementations14 Apr 2020 Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.

Abstractive Text Summarization Conversational Response Generation +8

A Double Residual Compression Algorithm for Efficient Distributed Learning

no code implementations16 Oct 2019 Xiaorui Liu, Yao Li, Jiliang Tang, Ming Yan

Large-scale machine learning models are often trained by parallel stochastic gradient descent algorithms.

Symmetric Regularization based BERT for Pair-wise Semantic Reasoning

1 code implementation8 Sep 2019 Weidi Xu, Xingyi Cheng, Kunlong Chen, Wei Wang, Bin Bi, Ming Yan, Chen Wu, Luo Si, Wei Chu, Taifeng Wang

To remedy this, we propose to augment the NSP task to a 3-class categorization task, which includes a category for previous sentence prediction (PSP).

Machine Reading Comprehension Natural Language Inference +1

Incorporating External Knowledge into Machine Reading for Generative Question Answering

no code implementations IJCNLP 2019 Bin Bi, Chen Wu, Ming Yan, Wei Wang, Jiangnan Xia, Chenliang Li

Different from existing work on knowledge-aware QA, we focus on a more challenging task of leveraging external knowledge to generate answers in natural language for a given question with context.

Answer Generation Generative Question Answering +1

Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

no code implementations13 Aug 2019 Jiangnan Xia, Chen Wu, Ming Yan

This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning.

Language Modelling Machine Reading Comprehension +1

StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

no code implementations ICLR 2020 Wei Wang, Bin Bi, Ming Yan, Chen Wu, Zuyi Bao, Jiangnan Xia, Liwei Peng, Luo Si

Recently, the pre-trained language model, BERT (and its robustly optimized version RoBERTa), has attracted a lot of attention in natural language understanding (NLU), and achieved state-of-the-art accuracy in various NLU tasks, such as sentiment classification, natural language inference, semantic textual similarity and question answering.

Language Modelling Linguistic Acceptability +6

On linear convergence of two decentralized algorithms

no code implementations17 Jun 2019 Yao Li, Ming Yan

In addition, we relax the requirement for the objective functions and the mixing matrices.

Vocal Bursts Valence Prediction

A Deep Cascade Model for Multi-Document Reading Comprehension

no code implementations28 Nov 2018 Ming Yan, Jiangnan Xia, Chen Wu, Bin Bi, Zhongzhou Zhao, Ji Zhang, Luo Si, Rui Wang, Wei Wang, Haiqing Chen

To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the document-level and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension.

Machine Reading Comprehension Question Answering +2

$D^2$: Decentralized Training over Decentralized Data

no code implementations ICML 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are unique and different.

Image Classification Multi-view Subspace Clustering

D$^2$: Decentralized Training over Decentralized Data

no code implementations19 Mar 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}.

Image Classification

Exploring Outliers in Crowdsourced Ranking for QoE

no code implementations18 Jul 2017 Qianqian Xu, Ming Yan, Chendi Huang, Jiechao Xiong, Qingming Huang, Yuan YAO

Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years.

Outlier Detection

Nonconvex penalties with analytical solutions for one-bit compressive sensing

no code implementations4 Jun 2017 Xiaolin Huang, Ming Yan

For several nonconvex penalties, including minimax concave penalty (MCP), $\ell_0$ norm, and sorted $\ell_1$ penalty, we provide fast algorithms for finding the analytical solutions by solving the dual problem.

Compressive Sensing Learning Theory

A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates

no code implementations25 Apr 2017 Zhi Li, Wei Shi, Ming Yan

This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms.

Mixed one-bit compressive sensing with applications to overexposure correction for CT reconstruction

no code implementations3 Jan 2017 Xiaolin Huang, Yan Xia, Lei Shi, Yixing Huang, Ming Yan, Joachim Hornegger, Andreas Maier

Aiming at overexposure correction for computed tomography (CT) reconstruction, we in this paper propose a mixed one-bit compressive sensing (M1bit-CS) to acquire information from both regular and saturated measurements.

Compressive Sensing Computed Tomography (CT) +1

On the Convergence of Asynchronous Parallel Iteration with Unbounded Delays

no code implementations13 Dec 2016 Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin

Recent years have witnessed the surge of asynchronous parallel (async-parallel) iterative algorithms due to problems involving very large-scale data and a large number of decision variables.

A new primal-dual algorithm for minimizing the sum of three functions with a linear operator

1 code implementation29 Nov 2016 Ming Yan

For the general convex case, we prove the convergence of this new algorithm in terms of the distance to a fixed point by showing that the iteration is a nonexpansive operator.

Asynchronous Multi-Task Learning

1 code implementation30 Sep 2016 Inci M. Baytas, Ming Yan, Anil K. Jain, Jiayu Zhou

The models for each hospital may be different because of the inherent differences in the distributions of the patient populations.

Multi-Task Learning

Coordinate Friendly Structures, Algorithms and Applications

no code implementations5 Jan 2016 Zhimin Peng, Tianyu Wu, Yangyang Xu, Ming Yan, Wotao Yin

To derive simple subproblems for several new classes of applications, this paper systematically studies coordinate-friendly operators that perform low-cost coordinate updates.

A Multiphase Image Segmentation Based on Fuzzy Membership Functions and L1-norm Fidelity

no code implementations9 Apr 2015 Fang Li, Stanley Osher, Jing Qin, Ming Yan

In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity.

Image Segmentation Semantic Segmentation

The Continuity of Images by Transmission Imaging Revisited

no code implementations8 Jan 2014 Zhitao Fan, Feng Guan, Chunlin Wu, Ming Yan

In transmission imaging, it was shown very recently in [49] that almost all images are continuous functions.

Astronomy Medical Diagnosis +1

Restoration of Images Corrupted by Impulse Noise and Mixed Gaussian Impulse Noise using Blind Inpainting

no code implementations4 Apr 2013 Ming Yan

In addition, we provide convergence analysis for these methods, these algorithms will converge to coordinatewise minimum points.

Image Restoration

Cannot find the paper you are looking for? You can Submit a new open access paper.