Search Results for author: Eric P. Xing

Found 183 papers, 61 papers with code

Understanding Masked Autoencoders via Hierarchical Latent Variable Models

1 code implementation8 Jun 2023 Lingjing Kong, Martin Q. Ma, Guangyi Chen, Eric P. Xing, Yuejie Chi, Louis-Philippe Morency, Kun Zhang

In this work, we formally characterize and justify existing empirical insights and provide theoretical guarantees of MAE.

Cuttlefish: Low-Rank Model Training without All the Tuning

1 code implementation4 May 2023 Hongyi Wang, Saurabh Agarwal, Pongsakorn U-chupala, Yoshiki Tanaka, Eric P. Xing, Dimitris Papailiopoulos

Cuttlefish leverages the observation that after a few epochs of full-rank training, the stable rank (i. e., an approximation of the true rank) of each layer stabilizes at a constant value.

Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach

1 code implementation8 Feb 2023 Han Guo, Philip Greengard, Hongyi Wang, Andrew Gelman, Yoon Kim, Eric P. Xing

A recent alternative formulation instead treats federated learning as a distributed inference problem, where the goal is to infer a global posterior from partitioned client data (Al-Shedivat et al., 2021).

Distributed Optimization Federated Learning +1

Does compressing activations help model parallel training?

no code implementations6 Jan 2023 Song Bian, Dacheng Li, Hongyi Wang, Eric P. Xing, Shivaram Venkataraman

Finally, we provide insights for future development of model parallelism compression algorithms.


On Optimizing the Communication of Model Parallelism

no code implementations10 Nov 2022 Yonghao Zhuang, Hexu Zhao, Lianmin Zheng, Zhuohan Li, Eric P. Xing, Qirong Ho, Joseph E. Gonzalez, Ion Stoica, Hao Zhang

This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters.

MPCFormer: fast, performant and private Transformer inference with MPC

1 code implementation2 Nov 2022 Dacheng Li, Rulin Shao, Hongyi Wang, Han Guo, Eric P. Xing, Hao Zhang

Through extensive evaluations, we show that MPCFORMER significantly speeds up Transformer inference in MPC settings while achieving similar ML performance to the input model.

Knowledge Distillation

ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Models

1 code implementation9 Oct 2022 Jiannan Xiang, Zhengzhong Liu, Yucheng Zhou, Eric P. Xing, Zhiting Hu

In the data disambiguation stage, we employ the prompted GPT-3 model to understand possibly ambiguous triples from the input data and convert each into a short sentence with reduced ambiguity.

Data-to-Text Generation Sentence Fusion

Meta-DETR: Image-Level Few-Shot Detection with Inter-Class Correlation Exploitation

1 code implementation30 Jul 2022 Gongjie Zhang, Zhipeng Luo, Kaiwen Cui, Shijian Lu, Eric P. Xing

Despite its success, the said paradigm is still constrained by several factors, such as (i) low-quality region proposals for novel classes and (ii) negligence of the inter-class correlation among different classes.

Few-Shot Object Detection Meta-Learning +1

Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion

1 code implementation28 Jul 2022 Gongjie Zhang, Zhipeng Luo, Jiaxing Huang, Shijian Lu, Eric P. Xing

The recently proposed DEtection TRansformer (DETR) has established a fully end-to-end paradigm for object detection.

object-detection Object Detection

MRCLens: an MRC Dataset Bias Detection Toolkit

no code implementations18 Jul 2022 Yifan Zhong, Haohan Wang, Eric P. Xing

Many recent neural models have shown remarkable empirical results in Machine Reading Comprehension, but evidence suggests sometimes the models take advantage of dataset biases to predict and fail to generalize on out-of-sample data.

Bias Detection Machine Reading Comprehension

Robustar: Interactive Toolbox Supporting Precise Data Annotation for Robust Vision Learning

1 code implementation18 Jul 2022 Chonghan Chen, Haohan Wang, Leyang Hu, Yuhao Zhang, Shuguang Lyu, Jingcheng Wu, Xinnuo Li, Linjing Sun, Eric P. Xing

We introduce the initial release of our software Robustar, which aims to improve the robustness of vision classification machine learning models through a data-driven perspective.

BIG-bench Machine Learning Image Classification

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

1 code implementation28 Jun 2022 Shibo Hao, Bowen Tan, Kaiwen Tang, Bin Ni, Xiyan Shao, Hengzhe Zhang, Eric P. Xing, Zhiting Hu

The resulting KGs as a symbolic interpretation of the source LMs also reveal new insights into the LMs' knowledge capacities.

Knowledge Graphs

Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentation

1 code implementation4 Jun 2022 Haohan Wang, Zeyi Huang, Xindi Wu, Eric P. Xing

Finally, we test this simple technique we identify (worst-case data augmentation with squared l2 norm alignment regularization) and show that the benefits of this method outrun those of the specially designed methods.

Data Augmentation

The Two Dimensions of Worst-case Training and the Integrated Effect for Out-of-domain Generalization

1 code implementation9 Apr 2022 Zeyi Huang, Haohan Wang, Dong Huang, Yong Jae Lee, Eric P. Xing

Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e. g., generalization across distributions) is valued.

BIG-bench Machine Learning Domain Generalization

Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation

1 code implementation2 Feb 2022 Yi-Fan Zhang, HANLIN ZHANG, Zachary C. Lipton, Li Erran Li, Eric P. Xing

Previous works on Treatment Effect Estimation (TEE) are not in widespread use because they are predominantly theoretical, where strong parametric assumptions are made but untractable for practical application.

POS Selection bias

Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning

1 code implementation28 Jan 2022 Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica

Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.

The Two Dimensions of Worst-Case Training and Their Integrated Effect for Out-of-Domain Generalization

no code implementations CVPR 2022 Zeyi Huang, Haohan Wang, Dong Huang, Yong Jae Lee, Eric P. Xing

Training with an emphasis on "hard-to-learn" components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e. g., generalization across distributions) is valued.

BIG-bench Machine Learning Domain Generalization

Towards Principled Disentanglement for Domain Generalization

1 code implementation CVPR 2022 HANLIN ZHANG, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, Eric P. Xing

To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG).

Disentanglement Domain Generalization

NOTMAD: Estimating Bayesian Networks with Sample-Specific Structures and Parameters

1 code implementation1 Nov 2021 Ben Lengerich, Caleb Ellington, Bryon Aragam, Eric P. Xing, Manolis Kellis

We encode the acyclicity constraint as a smooth regularization loss which is back-propagated to the mixing function; in this way, NOTMAD shares information between context-specific acyclic graphs, enabling the estimation of Bayesian network structures and parameters at even single-sample resolution.

Cooperative Multi-Agent Actor-Critic for Privacy-Preserving Load Scheduling in a Residential Microgrid

no code implementations6 Oct 2021 Zhaoming Qin, Nanqing Dong, Eric P. Xing, Junwei Cao

As a scalable data-driven approach, multi-agent reinforcement learning (MARL) has made remarkable advances in solving the cooperative residential load scheduling problems.

Multi-agent Reinforcement Learning Privacy Preserving +2

Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation

1 code implementation EMNLP 2021 Mingkai Deng, Bowen Tan, Zhengzhong Liu, Eric P. Xing, Zhiting Hu

Based on the nature of information change from input to output, we classify NLG tasks into compression (e. g., summarization), transduction (e. g., text rewriting), and creation (e. g., dialog).

Style Transfer Text Generation +1

Knowledge-Aware Meta-learning for Low-Resource Text Classification

1 code implementation EMNLP 2021 Huaxiu Yao, Yingxin Wu, Maruan Al-Shedivat, Eric P. Xing

Meta-learning has achieved great success in leveraging the historical learned knowledge to facilitate the learning process of the new task.

Meta-Learning text-classification +1

Toward a `Standard Model' of Machine Learning

no code implementations17 Aug 2021 Zhiting Hu, Eric P. Xing

Machine learning (ML) is about computational methods that enable machines to learn concepts from experience.

BIG-bench Machine Learning

Amortized Auto-Tuning: Cost-Efficient Bayesian Transfer Optimization for Hyperparameter Recommendation

1 code implementation17 Jun 2021 Yuxin Xiao, Eric P. Xing, Willie Neiswanger

With the surge in the number of hyperparameters and training times of modern machine learning models, hyperparameter tuning is becoming increasingly expensive.

Bayesian Optimization Transfer Learning

Efficient (Soft) Q-Learning for Text Generation with Limited Good Data

1 code implementation14 Jun 2021 Han Guo, Bowen Tan, Zhengzhong Liu, Eric P. Xing, Zhiting Hu

We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.

Q-Learning Reinforcement Learning (RL) +1

GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning

1 code implementation Findings (ACL) 2021 Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, Liang Lin

Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 4, 998 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems.

Mathematical Reasoning Question Answering

A Data-Centric Framework for Composable NLP Workflows

1 code implementation EMNLP 2020 Zhengzhong Liu, Guanxiong Ding, Avinash Bukkittu, Mansi Gupta, Pengzhi Gao, Atif Ahmed, Shikun Zhang, Xin Gao, Swapnil Singhavi, Linwei Li, Wei Wei, Zecong Hu, Haoran Shi, Haoying Zhang, Xiaodan Liang, Teruko Mitamura, Eric P. Xing, Zhiting Hu

Empirical natural language processing (NLP) systems in application domains (e. g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis, generation, and visualization.

Retrieval Text Retrieval

Towards Robust Partially Supervised Multi-Structure Medical Image Segmentation on Small-Scale Data

no code implementations28 Nov 2020 Nanqing Dong, Michael Kampffmeyer, Xiaodan Liang, Min Xu, Irina Voiculescu, Eric P. Xing

To bridge the methodological gaps in partially supervised learning (PSL) under data scarcity, we propose Vicinal Labels Under Uncertainty (VLUU), a simple yet efficient framework utilizing the human structure similarity for partially supervised medical image segmentation.

Data Augmentation Image Segmentation +4

Validate and Enable Machine Learning in Industrial AI

no code implementations30 Oct 2020 Hongbo Zou, Guangjing Chen, Pengtao Xie, Sean Chen, Yongtian He, Hochih Huang, Zheng Nie, Hongbao Zhang, Tristan Bala, Kazi Tulip, Yuqi Wang, Shenlin Qin, Eric P. Xing

However, manufacturers and solution partners need to understand how to implement and integrate an AI model into the existing industrial control system.

BIG-bench Machine Learning

Iterative Graph Self-Distillation

no code implementations23 Oct 2020 HANLIN ZHANG, Shuai Lin, Weiyang Liu, Pan Zhou, Jian Tang, Xiaodan Liang, Eric P. Xing

Recently, there has been increasing interest in the challenge of how to discriminatively vectorize graphs.

Contrastive Learning Graph Learning +1

Word Shape Matters: Robust Machine Translation with Visual Embedding

no code implementations20 Oct 2020 Haohan Wang, Peiyan Zhang, Eric P. Xing

Neural machine translation has achieved remarkable empirical performance over standard benchmark datasets, yet recent evidence suggests that the models can still fail easily dealing with substandard inputs such as misspelled words, To overcome this issue, we introduce a new encoding heuristic of the input symbols for character-level NLP models: it encodes the shape of each character through the images depicting the letters when printed.

Machine Translation Translation

Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach

1 code implementation EMNLP 2020 Bowen Tan, Lianhui Qin, Eric P. Xing, Zhiting Hu

Given a document and a target aspect (e. g., a topic of interest), aspect-based abstractive summarization attempts to generate a summary with respect to the aspect.

Abstractive Text Summarization

Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning

2 code implementations27 Aug 2020 Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, Eric P. Xing

Some recent schedulers choose job resources for users, but do so without awareness of how DL training can be re-optimized to better utilize the provided resources.

Fairness Scheduling

Self-Challenging Improves Cross-Domain Generalization

8 code implementations ECCV 2020 Zeyi Huang, Haohan Wang, Eric P. Xing, Dong Huang

We introduce a simple training heuristic, Representation Self-Challenging (RSC), that significantly improves the generalization of CNN to the out-of-domain data.

Domain Generalization Image Classification

Dropout as a Regularizer of Interaction Effects

no code implementations2 Jul 2020 Benjamin Lengerich, Eric P. Xing, Rich Caruana

We examine Dropout through the perspective of interactions.

Progressive Generation of Long Text with Pretrained Language Models

1 code implementation NAACL 2021 Bowen Tan, Zichao Yang, Maruan AI-Shedivat, Eric P. Xing, Zhiting Hu

However, as our systematic examination reveals, it is still challenging for such models to generate coherent long passages of text (e. g., 1000 tokens), especially when the models are fine-tuned to the target domain on a small corpus.

Improving GAN Training with Probability Ratio Clipping and Sample Reweighting

1 code implementation NeurIPS 2020 Yue Wu, Pan Zhou, Andrew Gordon Wilson, Eric P. Xing, Zhiting Hu

Despite success on a wide range of problems related to vision, generative adversarial networks (GANs) often suffer from inferior performance due to unstable training, especially for text generation.

Image Generation Style Transfer +1

Distributed, partially collapsed MCMC for Bayesian Nonparametrics

no code implementations15 Jan 2020 Avinava Dubey, Michael Minyi Zhang, Eric P. Xing, Sinead A. Williamson

Bayesian nonparametric (BNP) models provide elegant methods for discovering underlying latent features within a data set, but inference in such models can be slow.

Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks

no code implementations CL 2019 Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, Eric P. Xing

At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information.

Specific and Shared Causal Relation Modeling and Mechanism-Based Clustering

1 code implementation NeurIPS 2019 Biwei Huang, Kun Zhang, Pengtao Xie, Mingming Gong, Eric P. Xing, Clark Glymour

The learned SSCM gives the specific causal knowledge for each individual as well as the general trend over the population.

Causal Discovery

Learning Data Manipulation for Augmentation and Weighting

2 code implementations NeurIPS 2019 Zhiting Hu, Bowen Tan, Ruslan Salakhutdinov, Tom Mitchell, Eric P. Xing

In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm.

Data Augmentation Reinforcement Learning (RL) +2

Learning Sample-Specific Models with Low-Rank Personalized Regression

1 code implementation NeurIPS 2019 Benjamin Lengerich, Bryon Aragam, Eric P. Xing

Modern applications of machine learning (ML) deal with increasingly heterogeneous datasets comprised of data collected from overlapping latent subpopulations.


Learning Sparse Nonparametric DAGs

2 code implementations29 Sep 2019 Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing

We develop a framework for learning sparse nonparametric directed acyclic graphs (DAGs) from data.

Causal Discovery

ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations

2 code implementations5 Aug 2019 Ksenia Korovina, Sailun Xu, Kirthevasan Kandasamy, Willie Neiswanger, Barnabas Poczos, Jeff Schneider, Eric P. Xing

In applications such as molecule design or drug discovery, it is desirable to have an algorithm which recommends new candidate molecules based on the results of past tests.

Bayesian Optimization Drug Discovery

Adversarial Domain Adaptation Being Aware of Class Relationships

no code implementations28 May 2019 Zeya Wang, Baoyu Jing, Yang Ni, Nanqing Dong, Pengtao Xie, Eric P. Xing

In this paper, we propose a novel relationship-aware adversarial domain adaptation (RADA) algorithm, which first utilizes a single multi-class domain discriminator to enforce the learning of inter-class dependency structure during domain-adversarial training and then aligns this structure with the inter-class dependencies that are characterized from training the label predictor on source domain.

Domain Adaptation Transfer Learning

High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks

1 code implementation28 May 2019 Haohan Wang, Xindi Wu, Zeyi Huang, Eric P. Xing

We investigate the relationship between the frequency spectrum of image data and the generalization behavior of convolutional neural networks (CNN).

Adversarial Attack Vocal Bursts Intensity Prediction

Graph Transformer

no code implementations ICLR 2019 Yuan Li, Xiaodan Liang, Zhiting Hu, Yinbo Chen, Eric P. Xing

Graph neural networks (GNN) have gained increasing research interests as a mean to the challenging goal of robust and universal graph learning.

Few-Shot Learning General Classification +3

Knowledge-driven Encode, Retrieve, Paraphrase for Medical Image Report Generation

no code implementations25 Mar 2019 Christy Y. Li, Xiaodan Liang, Zhiting Hu, Eric P. Xing

Generating long and semantic-coherent reports to describe medical images poses great challenges towards bridging visual and linguistic modalities, incorporating medical domain knowledge, and generating realistic and accurate descriptions.

Graph Learning Knowledge Graphs +3

Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly

1 code implementation15 Mar 2019 Kirthevasan Kandasamy, Karun Raju Vysyaraju, Willie Neiswanger, Biswajit Paria, Christopher R. Collins, Jeff Schneider, Barnabas Poczos, Eric P. Xing

We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO.

Bayesian Optimisation

Learning Robust Representations by Projecting Superficial Statistics Out

no code implementations ICLR 2019 Haohan Wang, Zexue He, Zachary C. Lipton, Eric P. Xing

We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training.

Domain Generalization

Theoretically Principled Trade-off between Robustness and Accuracy

7 code implementations24 Jan 2019 Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, Michael. I. Jordan

We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples.

Adversarial Attack Adversarial Defense +2

The Sample Complexity of Semi-Supervised Learning with Nonparametric Mixture Models

no code implementations NeurIPS 2018 Chen Dan, Liu Leqi, Bryon Aragam, Pradeep K. Ravikumar, Eric P. Xing

We study the sample complexity of semi-supervised learning (SSL) and introduce new assumptions based on the mismatch between a mixture model learned from unlabeled data and the true mixture model induced by the (unknown) class conditional distributions.

Binary Classification Classification +2

Learning Pipelines with Limited Data and Domain Knowledge: A Study in Parsing Physics Problems

no code implementations NeurIPS 2018 Mrinmaya Sachan, Kumar Avinava Dubey, Tom M. Mitchell, Dan Roth, Eric P. Xing

Finally, we also show how Nuts&Bolts can be used to achieve improvements on a relation extraction task and on the end task of answering Newtonian physics problems.

BIG-bench Machine Learning Relation Extraction

Symbolic Graph Reasoning Meets Convolutions

1 code implementation NeurIPS 2018 Xiaodan Liang, Zhiting Hu, Hao Zhang, Liang Lin, Eric P. Xing

To cooperate with local convolutions, each SGR is constituted by three modules: a) a primal local-to-semantic voting module where the features of all symbolic nodes are generated by voting from local representations; b) a graph reasoning module propagates information over knowledge graph to achieve global semantic coherency; c) a dual semantic-to-local mapping module learns new associations of the evolved symbolic nodes with local representations, and accordingly enhances local features.

Image Classification Semantic Segmentation

Discourse in Multimedia: A Case Study in Information Extraction

no code implementations13 Nov 2018 Mrinmaya Sachan, Kumar Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, Eric P. Xing

At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information.

Fault Tolerance in Iterative-Convergent Machine Learning

no code implementations17 Oct 2018 Aurick Qiao, Bryon Aragam, Bingjing Zhang, Eric P. Xing

In this paper, we develop a general framework to quantify the effects of calculation errors on iterative-convergent algorithms and use this framework to design new strategies for checkpoint-based fault tolerance.

BIG-bench Machine Learning

Toward Understanding the Impact of Staleness in Distributed Machine Learning

no code implementations ICLR 2019 Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric P. Xing

Many distributed machine learning (ML) systems adopt the non-synchronous execution in order to alleviate the network communication bottleneck, resulting in stale parameters that do not reflect the latest updates.

BIG-bench Machine Learning

Connecting the Dots Between MLE and RL for Sequence Generation

no code implementations ICLR Workshop drlStructPred 2019 Bowen Tan*, Zhiting Hu*, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing

We present a generalized entropy regularized policy optimization formulation, and show that the apparently divergent algorithms can all be reformulated as special instances of the framework, with the only difference being the configurations of reward function and a couple of hyperparameters.

Machine Translation Text Summarization +1

Differentiable Expected BLEU for Text Generation

no code implementations27 Sep 2018 Wentao Wang, Zhiting Hu, Zichao Yang, Haoran Shi, Eric P. Xing

Neural text generation models such as recurrent networks are typically trained by maximizing data log-likelihood based on cross entropy.

Image Captioning Machine Translation +2

Sample Complexity of Nonparametric Semi-Supervised Learning

no code implementations NeurIPS 2018 Chen Dan, Liu Leqi, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing

We study the sample complexity of semi-supervised learning (SSL) and introduce new assumptions based on the mismatch between a mixture model learned from unlabeled data and the true mixture model induced by the (unknown) class conditional distributions.

Binary Classification Classification +2

Hybrid Subspace Learning for High-Dimensional Data

no code implementations5 Aug 2018 Micol Marchetti-Bowick, Benjamin J. Lengerich, Ankur P. Parikh, Eric P. Xing

One way to achieve this goal is to perform subspace learning to estimate a small set of latent features that capture the majority of the variance in the original data.

Dimensionality Reduction Video Background Subtraction +1

Reinforced Auto-Zoom Net: Towards Accurate and Fast Breast Cancer Segmentation in Whole-slide Images

no code implementations29 Jul 2018 Nanqing Dong, Michael Kampffmeyer, Xiaodan Liang, Zeya Wang, Wei Dai, Eric P. Xing

Motivated by the zoom-in operation of a pathologist using a digital microscope, RAZN learns a policy network to decide whether zooming is required in a given region of interest.

whole slide images

Unsupervised Text Style Transfer using Language Models as Discriminators

1 code implementation NeurIPS 2018 Zichao Yang, Zhiting Hu, Chris Dyer, Eric P. Xing, Taylor Berg-Kirkpatrick

Binary classifiers are often employed as discriminators in GAN-based unsupervised style transfer systems to ensure that transferred sentences are similar to sentences in the target domain.

Decipherment Language Modelling +4

Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation

no code implementations NeurIPS 2018 Christy Y. Li, Xiaodan Liang, Zhiting Hu, Eric P. Xing

Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents.

Decision Making Retrieval

Image-derived generative modeling of pseudo-macromolecular structures - towards the statistical assessment of Electron CryoTomography template matching

no code implementations12 May 2018 Kai Wen Wang, Xiangrui Zeng, Xiaodan Liang, Zhiguang Huo, Eric P. Xing, Min Xu

Cellular Electron CryoTomography (CECT) is a 3D imaging technique that captures information about the structure and spatial organization of macromolecular complexes within single cells, in near-native state and at sub-molecular resolution.

Template Matching Two-sample testing

Dilated Temporal Relational Adversarial Network for Generic Video Summarization

no code implementations30 Apr 2018 Yu-jia Zhang, Michael Kampffmeyer, Xiaodan Liang, Dingwen Zhang, Min Tan, Eric P. Xing

Specifically, DTR-GAN learns a dilated temporal relational generator and a discriminator with three-player loss in an adversarial manner.

Video Summarization Video Understanding

ConnNet: A Long-Range Relation-Aware Pixel-Connectivity Network for Salient Segmentation

no code implementations20 Apr 2018 Michael Kampffmeyer, Nanqing Dong, Xiaodan Liang, Yu-jia Zhang, Eric P. Xing

We argue that semantic salient segmentation can instead be effectively resolved by reformulating it as a simple yet intuitive pixel-pair based connectivity prediction task.

Semantic Segmentation

Removing Confounding Factors Associated Weights in Deep Neural Networks Improves the Prediction Accuracy for Healthcare Applications

1 code implementation20 Mar 2018 Haohan Wang, Zhenglin Wu, Eric P. Xing

The proliferation of healthcare data has brought the opportunities of applying data-driven approaches, such as machine learning methods, to assist diagnosis.

Electroencephalogram (EEG)

Orthogonality-Promoting Distance Metric Learning: Convex Relaxation and Theoretical Analysis

no code implementations ICML 2018 Pengtao Xie, Wei Wu, Yichen Zhu, Eric P. Xing

In this paper, we address these three issues by (1) seeking convex relaxations of the original nonconvex problems so that the global optimal is guaranteed to be achievable; (2) providing a formal analysis on OPR's capability of promoting balancedness; (3) providing a theoretical analysis that directly reveals the relationship between OPR and generalization performance.

Metric Learning

DiCE: The Infinitely Differentiable Monte-Carlo Estimator

5 code implementations14 Feb 2018 Jakob Foerster, Gregory Farquhar, Maruan Al-Shedivat, Tim Rocktäschel, Eric P. Xing, Shimon Whiteson

Lastly, to match the first-order gradient under differentiation, SL treats part of the cost as a fixed sample, which we show leads to missing and wrong terms for estimators of higher-order derivatives.


Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering

no code implementations12 Feb 2018 Bryon Aragam, Chen Dan, Eric P. Xing, Pradeep Ravikumar

Motivated by problems in data clustering, we establish general conditions under which families of nonparametric mixture models are identifiable, by introducing a novel framework involving clustering overfitted \emph{parametric} (i. e. misspecified) mixture models.

Nonparametric Clustering

The Intriguing Properties of Model Explanations

1 code implementation30 Jan 2018 Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing

Linear approximations to the decision boundary of a complex model have become one of the most popular tools for interpreting predictions.

Transformation Autoregressive Networks

no code implementations ICML 2018 Junier B. Oliva, Avinava Dubey, Manzil Zaheer, Barnabás Póczos, Ruslan Salakhutdinov, Eric P. Xing, Jeff Schneider

Further, through a comprehensive study over both real world and synthetic data, we show for that jointly leveraging transformations of variables and autoregressive conditional models, results in a considerable improvement in performance.

Density Estimation Outlier Detection

Personalized Survival Prediction with Contextual Explanation Networks

1 code implementation30 Jan 2018 Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing

Accurate and transparent prediction of cancer survival times on the level of individual patients can inform and improve patient care and treatment practices.

Survival Prediction

Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption

1 code implementation5 Jan 2018 Peilun Li, Xiaodan Liang, Daoyuan Jia, Eric P. Xing

It presents two main contributions to traditional GANs: 1) a soft gradient-sensitive objective for keeping semantic boundaries; 2) a semantic-aware discriminator for validating the fidelity of personalized adaptions with respect to each semantic region.

Domain Adaptation Semantic Segmentation

Unsupervised Object-Level Video Summarization with Online Motion Auto-Encoder

no code implementations2 Jan 2018 Yu-jia Zhang, Xiaodan Liang, Dingwen Zhang, Min Tan, Eric P. Xing

Unsupervised video summarization plays an important role on digesting, browsing, and searching the ever-growing videos every day, and the underlying fine-grained semantic and motion information (i. e., objects of interest and their key motions) in online videos has been barely touched.

Unsupervised Video Summarization

Stability Selection for Structured Variable Selection

no code implementations13 Dec 2017 George Philipp, Seunghak Lee, Eric P. Xing

Recently, a meta-algorithm called Stability Selection was proposed that can provide reliable finite-sample control of the number of false positives.

Variable Selection

Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks

no code implementations11 Dec 2017 Hao Zhang, Shizhen Xu, Graham Neubig, Wei Dai, Qirong Ho, Guangwen Yang, Eric P. Xing

Recent deep learning (DL) models have moved beyond static network architectures to dynamic ones, handling data where the network structure changes every example, such as sequences of variable lengths, trees, and graphs.

graph construction Management +1

Learning Less-Overlapping Representations

no code implementations ICLR 2018 Pengtao Xie, Hongbao Zhang, Eric P. Xing

In representation learning (RL), how to make the learned representations easy to interpret and less overfitted to training data are two important but challenging issues.

Representation Learning

Diversity-Promoting Bayesian Learning of Latent Variable Models

no code implementations23 Nov 2017 Pengtao Xie, Jun Zhu, Eric P. Xing

We also extend our approach to "diversify" Bayesian nonparametric models where the number of components is infinite.

Variational Inference

Effective Use of Bidirectional Language Modeling for Transfer Learning in Biomedical Named Entity Recognition

2 code implementations21 Nov 2017 Devendra Singh Sachan, Pengtao Xie, Mrinmaya Sachan, Eric P. Xing

We also show that BiLM weight transfer leads to a faster model training and the pretrained model requires fewer training examples to achieve a particular F1 score.

Language Modelling named-entity-recognition +3

Techniques for proving Asynchronous Convergence results for Markov Chain Monte Carlo methods

no code implementations17 Nov 2017 Alexander Terenin, Eric P. Xing

Markov Chain Monte Carlo (MCMC) methods such as Gibbs sampling are finding widespread use in applied statistics and machine learning.

Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning

no code implementations12 Nov 2017 Shiyue Zhang, Pengtao Xie, Dong Wang, Eric P. Xing

In hospital, physicians rely on massive clinical data to make diagnosis decisions, among which laboratory tests are one of the most important resources.

Decision Making Imputation +1

Towards Automated ICD Coding Using Deep Learning

no code implementations11 Nov 2017 Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, Eric P. Xing

Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis.

General Classification Management

Generative Semantic Manipulation with Contrasting GAN

no code implementations1 Aug 2017 Xiaodan Liang, Hao Zhang, Eric P. Xing

Generative Adversarial Networks (GANs) have recently achieved significant improvement on paired/unpaired image-to-image translation, such as photo$\rightarrow$ sketch and artist painting style transfer.

Image-to-Image Translation Style Transfer

Dual Motion GAN for Future-Flow Embedded Video Prediction

no code implementations ICCV 2017 Xiaodan Liang, Lisa Lee, Wei Dai, Eric P. Xing

To make both synthesized future frames and flows indistinguishable from reality, a dual adversarial training method is proposed to ensure that the future-flow prediction is able to help infer realistic future-frames, while the future-frame prediction in turn leads to realistic optical flows.

Representation Learning Video Prediction

Uncorrelation and Evenness: a New Diversity-Promoting Regularizer

no code implementations ICML 2017 Pengtao Xie, Aarti Singh, Eric P. Xing

Latent space models (LSMs) provide a principled and effective way to extract hidden patterns from observed data.

Learning Latent Space Models with Angular Constraints

no code implementations ICML 2017 Pengtao Xie, Yuntian Deng, Yi Zhou, Abhimanu Kumar, Yao-Liang Yu, James Zou, Eric P. Xing

The large model capacity of latent space models (LSMs) enables them to achieve great performance on various applications, but meanwhile renders LSMs to be prone to overfitting.

Efficient Correlated Topic Modeling with Topic Embedding

no code implementations1 Jul 2017 Junxian He, Zhiting Hu, Taylor Berg-Kirkpatrick, Ying Huang, Eric P. Xing

Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling.

Document Classification General Classification +2

Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters

no code implementations11 Jun 2017 Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P. Xing

We show that Poseidon enables Caffe and TensorFlow to achieve 15. 5x speed-up on 16 single-GPU machines, even with limited bandwidth (10GbE) and the challenging VGG19-22K network for image classification.

Image Classification

On Unifying Deep Generative Models

no code implementations ICLR 2018 Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing

Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), as emerging families for generative model learning, have largely been considered as two distinct paradigms and received extensive independent studies respectively.

Contextual Explanation Networks

1 code implementation ICLR 2018 Maruan Al-Shedivat, Avinava Dubey, Eric P. Xing

Our results on image and text classification and survival analysis tasks demonstrate that CENs are not only competitive with the state-of-the-art methods but also offer additional insights behind each prediction, that can be valuable for decision support.

Image Classification Interpretability Techniques for Deep Learning +4

Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification

no code implementations ACL 2017 Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, Eric P. Xing

Implicit discourse relation classification is of great challenge due to the lack of connectives as strong linguistic cues, which motivates the use of annotated implicit connectives to improve the recognition.

Classification General Classification +1

SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-rays

no code implementations26 Mar 2017 Wei Dai, Joseph Doyle, Xiaodan Liang, Hao Zhang, Nanqing Dong, Yuan Li, Eric P. Xing

Through this adversarial process the critic network learns the higher order structures and guides the segmentation model to achieve realistic segmentation outcomes.

Organ Segmentation

Recurrent Topic-Transition GAN for Visual Paragraph Generation

no code implementations ICCV 2017 Xiaodan Liang, Zhiting Hu, Hao Zhang, Chuang Gan, Eric P. Xing

The proposed Recurrent Topic-Transition Generative Adversarial Network (RTT-GAN) builds an adversarial framework between a structured paragraph generator and multi-level paragraph discriminators.

Image Paragraph Captioning

ZM-Net: Real-time Zero-shot Image Manipulation Network

no code implementations21 Mar 2017 Hao Wang, Xiaodan Liang, Hao Zhang, Dit-yan Yeung, Eric P. Xing

We cast this problem as manipulating an input image according to a parametric model whose key parameters can be conditionally generated from any guiding signal (even unseen ones).

Colorization Descriptive +2

Interpretable Structure-Evolving LSTM

no code implementations CVPR 2017 Xiaodan Liang, Liang Lin, Xiaohui Shen, Jiashi Feng, Shuicheng Yan, Eric P. Xing

Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.

Small Data Image Classification

Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection

1 code implementation CVPR 2017 Xiaodan Liang, Lisa Lee, Eric P. Xing

To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image.

Image Classification reinforcement-learning +3

Toward Controlled Generation of Text

3 code implementations ICML 2017 Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, Eric P. Xing

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain.

Variance Reduction in Stochastic Gradient Langevin Dynamics

no code implementations NeurIPS 2016 Kumar Avinava Dubey, Sashank J. Reddi, Sinead A. Williamson, Barnabas Poczos, Alexander J. Smola, Eric P. Xing

In this paper, we present techniques for reducing variance in stochastic gradient Langevin dynamics, yielding novel stochastic Monte Carlo methods that improve performance by reducing the variance in the stochastic gradient.

BIG-bench Machine Learning

SeDMiD for Confusion Detection: Uncovering Mind State from Time Series Brain Wave Data

no code implementations29 Nov 2016 Jingkang Yang, Haohan Wang, Jun Zhu, Eric P. Xing

In this paper, we propose an extension of State Space Model to work with different sources of information together with its learning and inference algorithms.

Time Series Analysis

Stochastic Variational Deep Kernel Learning

no code implementations NeurIPS 2016 Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing

We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task learning, additive covariance structures, and stochastic gradient training.

Gaussian Processes General Classification +2

Select-Additive Learning: Improving Generalization in Multimodal Sentiment Analysis

1 code implementation16 Sep 2016 Haohan Wang, Aaksha Meghawat, Louis-Philippe Morency, Eric P. Xing

In this paper, we propose a Select-Additive Learning (SAL) procedure that improves the generalizability of trained neural networks for multimodal sentiment analysis.

Multimodal Sentiment Analysis Sentiment Classification

Closed-Form Training of Mahalanobis Distance for Supervised Clustering

no code implementations CVPR 2016 Marc T. Law, Yao-Liang Yu, Matthieu Cord, Eric P. Xing

Clustering is the task of grouping a set of objects so that objects in the same cluster are more similar to each other than to those in other clusters.

Metric Learning Structured Prediction

Strategies and Principles of Distributed Machine Learning on Big Data

no code implementations31 Dec 2015 Eric P. Xing, Qirong Ho, Pengtao Xie, Wei Dai

Taking the view that Big ML systems can benefit greatly from ML-rooted statistical and algorithmic insights --- and that ML researchers should therefore not shy away from such systems design --- we discuss a series of principles and strategies distilled from our recent efforts on industrial-scale ML solutions.

BIG-bench Machine Learning

Scalable Modeling of Conversational-role based Self-presentation Characteristics in Large Online Forums

no code implementations10 Dec 2015 Abhimanu Kumar, Shriphani Palakodety, Chong Wang, Carolyn P. Rose, Eric P. Xing, Miaomiao Wen

Online discussion forums are complex webs of overlapping subcommunities (macrolevel structure, across threads) in which users enact different roles depending on which subcommunity they are participating in within a particular time point (microlevel structure, within threads).

Topic Models Variational Inference

Distributed Training of Deep Neural Networks with Theoretical Analysis: Under SSP Setting

no code implementations9 Dec 2015 Abhimanu Kumar, Pengtao Xie, Junming Yin, Eric P. Xing

We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines.

General Classification Image Classification

Deep Kernel Learning

4 code implementations6 Nov 2015 Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, Eric P. Xing

We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods.

Gaussian Processes

The Human Kernel

no code implementations NeurIPS 2015 Andrew Gordon Wilson, Christoph Dann, Christopher G. Lucas, Eric P. Xing

Bayesian nonparametric models, such as Gaussian processes, provide a compelling framework for automatic statistical modelling: these models have a high degree of flexibility, and automatically calibrated complexity.

Gaussian Processes

Bayesian Nonparametric Kernel-Learning

no code implementations29 Jun 2015 Junier Oliva, Avinava Dubey, Andrew G. Wilson, Barnabas Poczos, Jeff Schneider, Eric P. Xing

In this paper we introduce Bayesian nonparmetric kernel-learning (BaNK), a generic, data-driven framework for scalable learning of kernels.

LightLDA: Big Topic Models on Modest Compute Clusters

1 code implementation4 Dec 2014 Jinhui Yuan, Fei Gao, Qirong Ho, Wei Dai, Jinliang Wei, Xun Zheng, Eric P. Xing, Tie-Yan Liu, Wei-Ying Ma

When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized clusters with thousands of nodes, which are out of reach for most practitioners or academic researchers.

Topic Models

Dependent nonparametric trees for dynamic hierarchical clustering

no code implementations NeurIPS 2014 Kumar Avinava Dubey, Qirong Ho, Sinead A. Williamson, Eric P. Xing

Hierarchical clustering methods offer an intuitive and powerful way to model a wide variety of data sets.

On Model Parallelization and Scheduling Strategies for Distributed Machine Learning

no code implementations NeurIPS 2014 Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A. Gibson, Eric P. Xing

Distributed machine learning has typically been approached from a data parallel perspective, where big data are partitioned to multiple workers and an algorithm is executed concurrently over different data subsets under various synchronization schemes to ensure speed-up and/or correctness.

BIG-bench Machine Learning Scheduling

Model-Parallel Inference for Big Topic Models

no code implementations10 Nov 2014 Xun Zheng, Jin Kyu Kim, Qirong Ho, Eric P. Xing

In real world industrial applications of topic modeling, the ability to capture gigantic conceptual space by learning an ultra-high dimensional topical representation, i. e., the so-called "big model", is becoming the next desideratum after enthusiasms on "big data", especially for fine-grained downstream tasks such as online advertising, where good performances are usually achieved by regression-based predictors built on millions if not billions of input features.

Topic Models

High-Performance Distributed ML at Scale through Parameter Server Consistency Models

no code implementations29 Oct 2014 Wei Dai, Abhimanu Kumar, Jinliang Wei, Qirong Ho, Garth Gibson, Eric P. Xing

As Machine Learning (ML) applications increase in data size and model complexity, practitioners turn to distributed clusters to satisfy the increased computational and memory demands.

Vocal Bursts Intensity Prediction

Screening Rules for Overlapping Group Lasso

no code implementations25 Oct 2014 Seunghak Lee, Eric P. Xing

However, screening for overlapping group lasso remains an open challenge because the overlaps between groups make it infeasible to test each group independently.

Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms

no code implementations22 Sep 2014 Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wei Dai, Willie Neiswanger, Suvrit Sra, Eric P. Xing

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework.

Primitives for Dynamic Big Model Parallelism

no code implementations18 Jun 2014 Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A. Gibson, Eric P. Xing

When training large machine learning models with many variables or parameters, a single machine is often inadequate since the model may be too large to fit in memory, while training can take a long time even with stochastic updates.


Joint Summarization of Large-scale Collections of Web Images and Videos for Storyline Reconstruction

no code implementations CVPR 2014 Gunhee Kim, Leonid Sigal, Eric P. Xing

The reconstruction of storyline graphs is formulated as the inference of sparse time-varying directed graphs from a set of photo streams with assistance of videos.

Video Summarization

Hierarchical Feature Hashing for Fast Dimensionality Reduction

no code implementations CVPR 2014 Bin Zhao, Eric P. Xing

Curse of dimensionality is a practical and challenging problem in image categorization, especially in cases with a large number of classes.

Classification Dimensionality Reduction +5

Quasi Real-Time Summarization for Consumer Videos

no code implementations CVPR 2014 Bin Zhao, Eric P. Xing

With the widespread availability of video cameras, we are facing an ever-growing enormous collection of unedited and unstructured video data.

Reconstructing Storyline Graphs for Image Recommendation from Web Community Photos

no code implementations CVPR 2014 Gunhee Kim, Eric P. Xing

In this paper, we investigate an approach for reconstructing storyline graphs from large-scale collections of Internet images, and optionally other side information such as friendship graphs.

Dynamic Language Models for Streaming Text

no code implementations TACL 2014 Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, Eric P. Xing

We present a probabilistic language model that captures temporal dynamics and conditions on arbitrary non-linguistic context features.

Language Modelling Machine Translation +1

Consistent Bounded-Asynchronous Parameter Servers for Distributed ML

no code implementations30 Dec 2013 Jinliang Wei, Wei Dai, Abhimanu Kumar, Xun Zheng, Qirong Ho, Eric P. Xing

Many ML algorithms fall into the category of \emph{iterative convergent algorithms} which start from a randomly chosen initial point and converge to optima by repeating iteratively a set of procedures.

Petuum: A New Platform for Distributed Machine Learning on Big Data

no code implementations30 Dec 2013 Eric P. Xing, Qirong Ho, Wei Dai, Jin Kyu Kim, Jinliang Wei, Seunghak Lee, Xun Zheng, Pengtao Xie, Abhimanu Kumar, Yao-Liang Yu

What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)?

BIG-bench Machine Learning Scheduling

Language Modeling with Power Low Rank Ensembles

no code implementations EMNLP 2014 Ankur P. Parikh, Avneesh Saluja, Chris Dyer, Eric P. Xing

We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context.

Language Modelling Machine Translation +1

Structure-Aware Dynamic Scheduler for Parallel Machine Learning

no code implementations19 Dec 2013 Seunghak Lee, Jin Kyu Kim, Qirong Ho, Garth A. Gibson, Eric P. Xing

Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates.

BIG-bench Machine Learning Distributed Computing

Variance Reduction for Stochastic Gradient Optimization

no code implementations NeurIPS 2013 Chong Wang, Xi Chen, Alexander J. Smola, Eric P. Xing

We demonstrate how to construct the control variate for two practical problems using stochastic gradient optimization.

Variational Inference

More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server

no code implementations NeurIPS 2013 Qirong Ho, James Cipar, Henggang Cui, Seunghak Lee, Jin Kyu Kim, Phillip B. Gibbons, Garth A. Gibson, Greg Ganger, Eric P. Xing

We propose a parameter server system for distributed ML, which follows a Stale Synchronous Parallel (SSP) model of computation that maximizes the time computational workers spend doing useful work on ML algorithms, while still providing correctness guarantees.

Restricting exchangeable nonparametric distributions

no code implementations NeurIPS 2013 Sinead A. Williamson, Steve N. Maceachern, Eric P. Xing

Distributions over exchangeable matrices with infinitely many columns are useful in constructing nonparametric latent variable models.

Integrating Document Clustering and Topic Modeling

no code implementations26 Sep 2013 Pengtao Xie, Eric P. Xing

Document clustering and topic modeling are two closely related tasks which can mutually benefit each other.

Topic Models Variational Inference

Sharp Threshold for Multivariate Multi-Response Linear Regression via Block Regularized Lasso

no code implementations30 Jul 2013 Weiguang Wang, Yingbin Liang, Eric P. Xing

The goal is to recover the support union of all regression vectors using $l_1/l_2$-regularized Lasso.


Jointly Aligning and Segmenting Multiple Web Photo Streams for the Inference of Collective Photo Storylines

no code implementations CVPR 2013 Gunhee Kim, Eric P. Xing

To this end, we design a scalable message-passing based optimization framework to jointly achieve both tasks for the whole input image set at once.

Image Segmentation Semantic Segmentation

Sparse Output Coding for Large-Scale Visual Recognition

no code implementations CVPR 2013 Bin Zhao, Eric P. Xing

Many vision tasks require a multi-class classifier to discriminate multiple categories, on the order of hundreds or thousands.

Classification General Classification +3

On Triangular versus Edge Representations --- Towards Scalable Modeling of Networks

no code implementations NeurIPS 2012 Qirong Ho, Junming Yin, Eric P. Xing

A triangular motif is a vertex triple containing 2 or 3 edges, and the number of such motifs is $\Theta(\sum_{i}D_{i}^{2})$ (where $D_i$ is the degree of vertex $i$), which is much smaller than $N^2$ for low-maximum-degree networks.

Community Detection

Monte Carlo Methods for Maximum Margin Supervised Topic Models

no code implementations NeurIPS 2012 Qixia Jiang, Jun Zhu, Maosong Sun, Eric P. Xing

An effective strategy to exploit the supervising side information for discovering predictive topic representations is to impose discriminative constraints induced by such information on the posterior distributions under a topic model.

Topic Models

Graph Estimation From Multi-attribute Data

no code implementations29 Oct 2012 Mladen Kolar, Han Liu, Eric P. Xing

Many real world network problems often concern multivariate nodal attributes such as image, textual, and multi-view feature vectors on nodes, rather than simple univariate nodal attributes.

Diffusion of Lexical Change in Social Media

no code implementations18 Oct 2012 Jacob Eisenstein, Brendan O'Connor, Noah A. Smith, Eric P. Xing

Computer-mediated communication is driving fundamental changes in the nature of written language.

Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs

no code implementations5 Oct 2012 Jun Zhu, Ning Chen, Eric P. Xing

When the regularization is induced from a linear operator on the posterior distributions, such as the expectation operator, we present a general convex-analysis theorem to characterize the solution of RegBayes.

Bayesian Inference Multi-Task Learning

High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso

no code implementations2 Feb 2012 Makoto Yamada, Wittawat Jitkrittum, Leonid Sigal, Eric P. Xing, Masashi Sugiyama

We first show that, with particular choices of kernel functions, non-redundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures.

feature selection Vocal Bursts Intensity Prediction

Kernel Embeddings of Latent Tree Graphical Models

no code implementations NeurIPS 2011 Le Song, Eric P. Xing, Ankur P. Parikh

Latent tree graphical models are natural tools for expressing long range and hierarchical dependencies among many variables which are common in computer vision, bioinformatics and natural language processing problems.

Infinite Latent SVM for Classification and Multi-task Learning

no code implementations NeurIPS 2011 Jun Zhu, Ning Chen, Eric P. Xing

Unlike existing nonparametric Bayesian models, which rely solely on specially conceived priors to incorporate domain knowledge for discovering improved latent representations, we study nonparametric Bayesian inference with regularization on the desired posterior distributions.

Bayesian Inference Classification +2

Large-Scale Category Structure Aware Image Categorization

no code implementations NeurIPS 2011 Bin Zhao, Fei Li, Eric P. Xing

With the emergence of structured large-scale dataset such as the ImageNet, rich information about the conceptual relationships between images, such as a tree hierarchy among various image categories, become available.

Image Categorization

Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification

no code implementations NeurIPS 2010 Li-Jia Li, Hao Su, Li Fei-Fei, Eric P. Xing

Robust low-level image features have been proven to be effective representations for a variety of visual recognition tasks such as object recognition and scene classification; but pixels, or even local image patches, carry little semantic meanings.

General Classification Object Recognition +1