Search Results for author: Ming Yan

Found 44 papers, 11 papers with code

PALM: Pre-training an Autoencoding\&Autoregressive Language Model for Context-conditioned Generation

no code implementations EMNLP 2020 Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.

Abstractive Text Summarization Conversational Response Generation +6

Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training

no code implementations21 Aug 2021 Ming Yan, Haiyang Xu, Chenliang Li, Bin Bi, Junfeng Tian, Min Gui, Wei Wang

Existing approaches to vision-language pre-training (VLP) heavily rely on an object detector based on bounding boxes (regions), where salient objects are first detected from images and then a Transformer-based model is used for cross-modal fusion.

Object Detection

Decentralized Composite Optimization with Compression

no code implementations10 Aug 2021 Yao Li, Xiaorui Liu, Jiliang Tang, Ming Yan, Kun Yuan

Decentralized optimization and communication compression have exhibited their great potential in accelerating distributed machine learning by mitigating the communication bottleneck in practice.

Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction

no code implementations ACL 2021 Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang

This work focuses on generative QA which aims to generate an abstractive answer to a given question instead of extracting an answer span from a provided passage.

Generative Question Answering Machine Reading Comprehension

Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs

no code implementations26 Jul 2021 Zhuoqing Song, Lei Shi, Shi Pu, Ming Yan

In this work, we consider the decentralized optimization problem in which a network of $n$ agents, each possessing a smooth and convex objective function, wish to collaboratively minimize the average of all the objective functions through peer-to-peer communication in a directed graph.

Elastic Graph Neural Networks

1 code implementation5 Jul 2021 Xiaorui Liu, Wei Jin, Yao Ma, Yaxin Li, Hua Liu, Yiqi Wang, Ming Yan, Jiliang Tang

While many existing graph neural networks (GNNs) have been proven to perform $\ell_2$-based graph smoothing that enforces smoothness globally, in this work we aim to further enhance the local smoothness adaptivity of GNNs via $\ell_1$-based graph smoothing.

Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks

no code implementations14 Jun 2021 Zhuoqing Song, Lei Shi, Shi Pu, Ming Yan

In the second part, we propose a broadcast-like version of CPP (B-CPP), which also achieves linear convergence rate under the same conditions for the objective functions.

RCT: Resource Constrained Training for Edge AI

no code implementations26 Mar 2021 Tian Huang, Tao Luo, Ming Yan, Joey Tianyi Zhou, Rick Goh

For example, quantisation-aware training (QAT) method involves two copies of model parameters, which is usually beyond the capacity of on-chip memory in edge devices.

SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels

no code implementations14 Mar 2021 Chenliang Li, Ming Yan, Haiyang Xu, Fuli Luo, Wei Wang, Bin Bi, Songfang Huang

Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations.

CoRe: An Efficient Coarse-refined Training Framework for BERT

no code implementations27 Nov 2020 Cheng Yang, Shengnan Wang, Yuechuan Li, Chao Yang, Ming Yan, Jingqiao Zhang, Fangquan Lin

In the second phase, we transform the trained relaxed BERT model into the original BERT and further retrain the model.

Deep Neural Networks with Short Circuits for Improved Gradient Learning

no code implementations23 Sep 2020 Ming Yan, Xueli Xiao, Joey Tianyi Zhou, Yi Pan

Deep neural networks have achieved great success both in computer vision and natural language processing tasks.

Fast algorithms for robust principal component analysis with an upper bound on the rank

1 code implementation18 Aug 2020 Ningyu Sha, Lei Shi, Ming Yan

The first type of algorithm applies regularization terms on the singular values of a matrix to obtain a low-rank matrix.

Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering

no code implementations ACL 2020 Ming Yan, Hao Zhang, Di Jin, Joey Tianyi Zhou

Multiple-choice question answering (MCQA) is one of the most challenging tasks in machine reading comprehension since it requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations.

Machine Reading Comprehension Meta-Learning +2

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

1 code implementation23 Jun 2020 Xueli Xiao, Ming Yan, Sunitha Basodi, Chunyan Ji, Yi Pan

However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth.

Hyperparameter Optimization

A Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features

no code implementations15 Jun 2020 Sulaiman A. Alghunaim, Ming Yan, Ali H. Sayed

This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents.

PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation

2 code implementations14 Apr 2020 Bin Bi, Chenliang Li, Chen Wu, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, Luo Si

An extensive set of experiments show that PALM achieves new state-of-the-art results on a variety of language generation benchmarks covering generative question answering (Rank 1 on the official MARCO leaderboard), abstractive summarization on CNN/DailyMail as well as Gigaword, question generation on SQuAD, and conversational response generation on Cornell Movie Dialogues.

Abstractive Text Summarization Conversational Response Generation +6

A Double Residual Compression Algorithm for Efficient Distributed Learning

no code implementations16 Oct 2019 Xiaorui Liu, Yao Li, Jiliang Tang, Ming Yan

Large-scale machine learning models are often trained by parallel stochastic gradient descent algorithms.

Symmetric Regularization based BERT for Pair-wise Semantic Reasoning

1 code implementation8 Sep 2019 Weidi Xu, Xingyi Cheng, Kunlong Chen, Wei Wang, Bin Bi, Ming Yan, Chen Wu, Luo Si, Wei Chu, Taifeng Wang

To remedy this, we propose to augment the NSP task to a 3-class categorization task, which includes a category for previous sentence prediction (PSP).

Document-level Machine Reading Comprehension +2

Incorporating External Knowledge into Machine Reading for Generative Question Answering

no code implementations IJCNLP 2019 Bin Bi, Chen Wu, Ming Yan, Wei Wang, Jiangnan Xia, Chenliang Li

Different from existing work on knowledge-aware QA, we focus on a more challenging task of leveraging external knowledge to generate answers in natural language for a given question with context.

Generative Question Answering Reading Comprehension

Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

no code implementations13 Aug 2019 Jiangnan Xia, Chen Wu, Ming Yan

This paper focuses on how to take advantage of external relational knowledge to improve machine reading comprehension (MRC) with multi-task learning.

Language Modelling Machine Reading Comprehension +1

StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

no code implementations ICLR 2020 Wei Wang, Bin Bi, Ming Yan, Chen Wu, Zuyi Bao, Jiangnan Xia, Liwei Peng, Luo Si

Recently, the pre-trained language model, BERT (and its robustly optimized version RoBERTa), has attracted a lot of attention in natural language understanding (NLU), and achieved state-of-the-art accuracy in various NLU tasks, such as sentiment classification, natural language inference, semantic textual similarity and question answering.

Language Modelling Linguistic Acceptability +5

On linear convergence of two decentralized algorithms

no code implementations17 Jun 2019 Yao Li, Ming Yan

In addition, we relax the requirement for the objective functions and the mixing matrices.

A Deep Cascade Model for Multi-Document Reading Comprehension

no code implementations28 Nov 2018 Ming Yan, Jiangnan Xia, Chen Wu, Bin Bi, Zhongzhou Zhao, Ji Zhang, Luo Si, Rui Wang, Wei Wang, Haiqing Chen

To address this problem, we develop a novel deep cascade learning model, which progressively evolves from the document-level and paragraph-level ranking of candidate texts to more precise answer extraction with machine reading comprehension.

Document-level Machine Reading Comprehension +1

$D^2$: Decentralized Training over Decentralized Data

no code implementations ICML 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are unique and different.

Image Classification Multi-view Subspace Clustering

D$^2$: Decentralized Training over Decentralized Data

no code implementations19 Mar 2018 Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, Ji Liu

While training a machine learning model using multiple workers, each of which collects data from their own data sources, it would be most useful when the data collected from different workers can be {\em unique} and {\em different}.

Image Classification

Exploring Outliers in Crowdsourced Ranking for QoE

no code implementations18 Jul 2017 Qianqian Xu, Ming Yan, Chendi Huang, Jiechao Xiong, Qingming Huang, Yuan YAO

Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years.

Outlier Detection

Nonconvex penalties with analytical solutions for one-bit compressive sensing

no code implementations4 Jun 2017 Xiaolin Huang, Ming Yan

For several nonconvex penalties, including minimax concave penalty (MCP), $\ell_0$ norm, and sorted $\ell_1$ penalty, we provide fast algorithms for finding the analytical solutions by solving the dual problem.

Compressive Sensing Learning Theory

A decentralized proximal-gradient method with network independent step-sizes and separated convergence rates

no code implementations25 Apr 2017 Zhi Li, Wei Shi, Ming Yan

This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms.

Mixed one-bit compressive sensing with applications to overexposure correction for CT reconstruction

no code implementations3 Jan 2017 Xiaolin Huang, Yan Xia, Lei Shi, Yixing Huang, Ming Yan, Joachim Hornegger, Andreas Maier

Aiming at overexposure correction for computed tomography (CT) reconstruction, we in this paper propose a mixed one-bit compressive sensing (M1bit-CS) to acquire information from both regular and saturated measurements.

Compressive Sensing Computed Tomography (CT) +1

On the Convergence of Asynchronous Parallel Iteration with Unbounded Delays

no code implementations13 Dec 2016 Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin

Recent years have witnessed the surge of asynchronous parallel (async-parallel) iterative algorithms due to problems involving very large-scale data and a large number of decision variables.

A new primal-dual algorithm for minimizing the sum of three functions with a linear operator

1 code implementation29 Nov 2016 Ming Yan

For the general convex case, we prove the convergence of this new algorithm in terms of the distance to a fixed point by showing that the iteration is a nonexpansive operator.

Asynchronous Multi-Task Learning

1 code implementation30 Sep 2016 Inci M. Baytas, Ming Yan, Anil K. Jain, Jiayu Zhou

The models for each hospital may be different because of the inherent differences in the distributions of the patient populations.

Multi-Task Learning

Coordinate Friendly Structures, Algorithms and Applications

no code implementations5 Jan 2016 Zhimin Peng, Tianyu Wu, Yangyang Xu, Ming Yan, Wotao Yin

To derive simple subproblems for several new classes of applications, this paper systematically studies coordinate-friendly operators that perform low-cost coordinate updates.

A Multiphase Image Segmentation Based on Fuzzy Membership Functions and L1-norm Fidelity

no code implementations9 Apr 2015 Fang Li, Stanley Osher, Jing Qin, Ming Yan

In this paper, we propose a variational multiphase image segmentation model based on fuzzy membership functions and L1-norm fidelity.

Semantic Segmentation

The Continuity of Images by Transmission Imaging Revisited

no code implementations8 Jan 2014 Zhitao Fan, Feng Guan, Chunlin Wu, Ming Yan

In transmission imaging, it was shown very recently in [49] that almost all images are continuous functions.

Medical Diagnosis Object Reconstruction

Restoration of Images Corrupted by Impulse Noise and Mixed Gaussian Impulse Noise using Blind Inpainting

no code implementations4 Apr 2013 Ming Yan

In addition, we provide convergence analysis for these methods, these algorithms will converge to coordinatewise minimum points.

Image Restoration

Cannot find the paper you are looking for? You can Submit a new open access paper.