Search Results for author: YIngyu Liang

Found 64 papers, 27 papers with code

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning

1 code implementation22 Feb 2024 Zhuoyan Xu, Zhenmei Shi, Junyi Wei, Fangzhou Mu, Yin Li, YIngyu Liang

An emerging solution with recent success in vision and NLP involves finetuning a foundation model on a selection of relevant tasks, before its adaptation to a target task with limited labeled samples.

Fourier Circuits in Neural Networks: Unlocking the Potential of Large Language Models in Mathematical Reasoning and Modular Arithmetic

no code implementations12 Feb 2024 Jiuxiang Gu, Chenyang Li, YIngyu Liang, Zhenmei Shi, Zhao Song, Tianyi Zhou

Our research presents a thorough analytical characterization of the features learned by stylized one-hidden layer neural networks and one-layer Transformers in addressing this task.

Mathematical Reasoning

When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysis

1 code implementation9 Aug 2023 Yiyou Sun, Zhenmei Shi, YIngyu Liang, Yixuan Li

This paper bridges the gap by providing an analytical framework to formalize and investigate when and how known classes can help discover novel classes.

Novel Class Discovery

Two Heads are Better than One: Towards Better Adversarial Robustness by Combining Transduction and Rejection

no code implementations27 May 2023 Nils Palumbo, Yang Guo, Xi Wu, Jiefeng Chen, YIngyu Liang, Somesh Jha

Nevertheless, under recent strong adversarial attacks (GMSA, which has been shown to be much more effective than AutoAttack against transduction), Goldwasser et al.'s work was shown to have low performance in a practical deep-learning setting.

Adversarial Robustness

Stratified Adversarial Robustness with Rejection

1 code implementation2 May 2023 Jiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, YIngyu Liang, Somesh Jha

We theoretically analyze the stratified rejection setting and propose a novel defense method -- Adversarial Training with Consistent Prediction-based Rejection (CPR) -- for building a robust selective classifier.

Adversarial Robustness Robust classification

Domain Generalization via Nuclear Norm Regularization

1 code implementation13 Mar 2023 Zhenmei Shi, Yifei Ming, Ying Fan, Frederic Sala, YIngyu Liang

In this paper, we propose a simple and effective regularization method based on the nuclear norm of the learned features for domain generalization.

Domain Generalization

The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning

1 code implementation28 Feb 2023 Zhenmei Shi, Jiefeng Chen, Kunyang Li, Jayaram Raghuram, Xi Wu, YIngyu Liang, Somesh Jha

foundation models) has recently become a prevalent learning paradigm, where one first pre-trains a representation using large-scale unlabeled data, and then learns simple predictors on top of the representation using small labeled data from the downstream tasks.

Contrastive Learning

A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage over Fixed Features

no code implementations ICLR 2022 Zhenmei Shi, Junyi Wei, YIngyu Liang

These results provide theoretical evidence showing that feature learning in neural networks depends strongly on the input structure and leads to the superior performance.

On the identifiability of mixtures of ranking models

no code implementations31 Jan 2022 Xiaomin Zhang, Xucheng Zhang, Po-Ling Loh, YIngyu Liang

Mixtures of ranking models are standard tools for ranking problems.

Revisiting Adversarial Robustness of Classifiers With a Reject Option

no code implementations AAAI Workshop AdvML 2022 Jiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, YIngyu Liang, Somesh Jha

Motivated by this metric, we propose novel loss functions and a robust training method -- \textit{stratified adversarial training with rejection} (SATR) -- for a classifier with reject option, where the goal is to accept and correctly-classify small input perturbations, while allowing the rejection of larger input perturbations that cannot be correctly classified.

Adversarial Robustness Image Classification

Towards Evaluating the Robustness of Neural Networks Learned by Transduction

1 code implementation ICLR 2022 Jiefeng Chen, Xi Wu, Yang Guo, YIngyu Liang, Somesh Jha

There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021).

Adversarial Robustness Bilevel Optimization +1

Attentive Walk-Aggregating Graph Neural Networks

1 code implementation6 Oct 2021 Mehmet F. Demirel, Shengchao Liu, Siddhant Garg, Zhenmei Shi, YIngyu Liang

Our experiments demonstrate the strong performance of AWARE in graph-level prediction tasks in the standard setting in the domains of molecular property prediction and social networks.

Molecular Property Prediction Property Prediction

Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles

1 code implementation NeurIPS 2021 Jiefeng Chen, Frederick Liu, Besim Avci, Xi Wu, YIngyu Liang, Somesh Jha

This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs.

Towards Adversarial Robustness via Transductive Learning

no code implementations15 Jun 2021 Jiefeng Chen, Yang Guo, Xi Wu, Tianqi Li, Qicheng Lao, YIngyu Liang, Somesh Jha

Compared to traditional "test-time" defenses, these defense mechanisms "dynamically retrain" the model based on test time input via transductive learning; and theoretically, attacking these defenses boils down to bilevel optimization, which seems to raise the difficulty for adaptive attacks.

Adversarial Robustness Bilevel Optimization +1

Deep Online Fused Video Stabilization

1 code implementation2 Feb 2021 Zhenmei Shi, Fuhao Shi, Wei-Sheng Lai, Chia-Kai Liang, YIngyu Liang

We present a deep neural network (DNN) that uses both sensor data (gyroscope) and image content (optical flow) to stabilize videos through unsupervised learning.

Video Stabilization

Test-Time Adaptation and Adversarial Robustness

no code implementations1 Jan 2021 Xi Wu, Yang Guo, Tianqi Li, Jiefeng Chen, Qicheng Lao, YIngyu Liang, Somesh Jha

On the positive side, we show that, if one is allowed to access the training data, then Domain Adversarial Neural Networks (${\sf DANN}$), an algorithm designed for unsupervised domain adaptation, can provide nontrivial robustness in the test-time maximin threat model against strong transfer attacks and adaptive fixed point attacks.

Adversarial Robustness Test-time Adaptation +1

PBoS: Probabilistic Bag-of-Subwords for Generalizing Word Embedding

1 code implementation Findings of the Association for Computational Linguistics 2020 Zhao Jinman, Shawn Zhong, Xiaomin Zhang, YIngyu Liang

We look into the task of \emph{generalizing} word embeddings: given a set of pre-trained word vectors over a finite vocabulary, the goal is to predict embedding vectors for out-of-vocabulary words, \emph{without} extra contextual information.

POS POS Tagging +2

Graph Neural Networks for an Accurate and Interpretable Prediction of the Properties of Polycrystalline Materials

1 code implementation12 Oct 2020 Minyi Dai, Mehmet F. Demirel, YIngyu Liang, Jia-Mian Hu

Various machine learning models have been used to predict the properties of polycrystalline materials, but none of them directly consider the physical interactions among neighboring grains despite such microscopic interactions critically determining macroscopic material properties.

Materials Science

Informative Outlier Matters: Robustifying Out-of-distribution Detection Using Outlier Mining

no code implementations28 Sep 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Functional Regularization for Representation Learning: A Unified Theoretical Perspective

1 code implementation NeurIPS 2020 Siddhant Garg, YIngyu Liang

Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks.

Representation Learning Self-Supervised Learning +1

Can Adversarial Weight Perturbations Inject Neural Backdoors?

1 code implementation4 Aug 2020 Siddhant Garg, Adarsh Kumar, Vibhor Goel, YIngyu Liang

We introduce adversarial perturbations in the model weights using a composite loss on the predictions of the original model and the desired trigger through projected gradient descent.

Learning Entangled Single-Sample Gaussians in the Subset-of-Signals Model

no code implementations10 Jul 2020 Yingyu Liang, Hui Yuan

In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of $n$ distributions, given one single sample from each distribution.

ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining

1 code implementation26 Jun 2020 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

We show that, by mining informative auxiliary OOD data, one can significantly improve OOD detection performance, and somewhat surprisingly, generalize to unseen adversarial attacks.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Representation Bayesian Risk Decompositions and Multi-Source Domain Adaptation

no code implementations22 Apr 2020 Xi Wu, Yang Guo, Jiefeng Chen, YIngyu Liang, Somesh Jha, Prasad Chalasani

Recent studies provide hints and failure examples for domain invariant representation learning, a common approach for this problem, but the explanations provided are somewhat different and do not provide a unified picture.

Domain Adaptation Representation Learning

Learning Entangled Single-Sample Distributions via Iterative Trimming

no code implementations20 Apr 2020 Hui Yuan, YIngyu Liang

We study mean estimation and linear regression under general conditions, and analyze a simple and computationally efficient method based on iteratively trimming samples and re-estimating the parameter on the trimmed sample set.

Gradients as Features for Deep Representation Learning

no code implementations ICLR 2020 Fangzhou Mu, YIngyu Liang, Yin Li

We address the challenging problem of deep representation learning--the efficient adaption of a pre-trained deep network to different tasks.

Representation Learning

Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer

no code implementations Asian Chapter of the Association for Computational Linguistics 2020 Siddhant Garg, Rohit Kumar Sharma, YIngyu Liang

In this paper we show that concatenating the embeddings from the pre-trained model with those from a simple sentence embedding model trained only on the target data, can improve over the performance of FT for few-sample tasks.

Dimensionality Reduction General Classification +5

Robust Out-of-distribution Detection for Neural Networks

1 code implementation AAAI Workshop AdvML 2022 Jiefeng Chen, Yixuan Li, Xi Wu, YIngyu Liang, Somesh Jha

Formally, we extensively study the problem of Robust Out-of-Distribution Detection on common OOD detection approaches, and show that state-of-the-art OOD detectors can be easily fooled by adding small perturbations to the in-distribution and OOD inputs.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Sketching Transformed Matrices with Applications to Natural Language Processing

no code implementations23 Feb 2020 Yingyu Liang, Zhao Song, Mengdi Wang, Lin F. Yang, Xin Yang

We show that our approach obtains small error and is efficient in both space and time.

Shallow Domain Adaptive Embeddings for Sentiment Analysis

no code implementations IJCNLP 2019 Prathusha K Sarma, YIngyu Liang, William A. Sethares

This paper proposes a way to improve the performance of existing algorithms for text classification in domains with strong language semantics.

Domain Adaptation General Classification +5

Robust Attribution Regularization

1 code implementation NeurIPS 2019 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

An emerging problem in trustworthy machine learning is to train models that produce robust interpretations for their predictions.

Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

no code implementations NeurIPS 2019 Zeyuan Allen-Zhu, Yuanzhi Li, YIngyu Liang

In this work, we prove that overparameterized neural networks can learn some notable concept classes, including two and three-layer networks with fewer parameters and smooth activations.

Learning Theory Vocal Bursts Valence Prediction

Generalizing Word Embeddings using Bag of Subwords

1 code implementation EMNLP 2018 Jinman Zhao, Sidharth Mudgal, YIngyu Liang

We approach the problem of generalizing pre-trained word embeddings beyond fixed-size vocabularies without using additional contextual information.

TAG Word Embeddings +1

Towards Understanding Limitations of Pixel Discretization Against Adversarial Attacks

1 code implementation20 May 2018 Jiefeng Chen, Xi Wu, Vaibhav Rastogi, YIngyu Liang, Somesh Jha

We analyze our results in a theoretical framework and offer strong evidence that pixel discretization is unlikely to work on all but the simplest of the datasets.

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

1 code implementation ACL 2018 Mikhail Khodak, Nikunj Saunshi, YIngyu Liang, Tengyu Ma, Brandon Stewart, Sanjeev Arora

Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features.

Document Classification Domain Adaptation +2

Domain Adapted Word Embeddings for Improved Sentiment Classification

1 code implementation ACL 2018 Prathusha K Sarma, YIngyu Liang, William A. Sethares

Generic word embeddings are trained on large-scale generic corpora; Domain Specific (DS) word embeddings are trained only on data from a domain of interest.

Classification General Classification +5

Learning Mixtures of Linear Regressions with Nearly Optimal Complexity

no code implementations22 Feb 2018 Yuanzhi Li, YIngyu Liang

Mixtures of Linear Regressions (MLR) is an important mixture model with many applications.

Differentially Private Clustering in High-Dimensional Euclidean Spaces

no code implementations ICML 2017 Maria-Florina Balcan, Travis Dick, YIngyu Liang, Wenlong Mou, Hongyang Zhang

We study the problem of clustering sensitive data while preserving the privacy of individuals represented in the dataset, which has broad applications in practical machine learning and data analysis tasks.

Clustering Vocal Bursts Intensity Prediction

Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlations

1 code implementation ICML 2017 Yuanzhi Li, YIngyu Liang

Non-negative matrix factorization is a basic tool for decomposing data into the feature and weight matrices under non-negativity constraints, and in practice is often solved in the alternating minimization framework.

Matrix Completion and Related Problems via Strong Duality

no code implementations27 Apr 2017 Maria-Florina Balcan, YIngyu Liang, David P. Woodruff, Hongyang Zhang

This work studies the strong duality of non-convex matrix factorization problems: we show that under certain dual conditions, these problems and its dual have the same optimum.

Matrix Completion

Generalization and Equilibrium in Generative Adversarial Nets (GANs)

1 code implementation ICML 2017 Sanjeev Arora, Rong Ge, YIngyu Liang, Tengyu Ma, Yi Zhang

We show that training of generative adversarial network (GAN) may not have good generalization properties; e. g., training may appear successful but the trained distribution may be far from target distribution in standard metrics.

Generative Adversarial Network

Scalable Influence Maximization for Multiple Products in Continuous-Time Diffusion Networks

no code implementations8 Dec 2016 Nan Du, YIngyu Liang, Maria-Florina Balcan, Manuel Gomez-Rodriguez, Hongyuan Zha, Le Song

A typical viral marketing model identifies influential users in a social network to maximize a single product adoption assuming unlimited user attention, campaign budgets, and time.

Marketing

Recovery Guarantee of Non-negative Matrix Factorization via Alternating Updates

no code implementations NeurIPS 2016 Yuanzhi Li, YIngyu Liang, Andrej Risteski

Non-negative matrix factorization is a popular tool for decomposing data into feature and weight matrices under non-negativity constraints.

Diverse Neural Network Learns True Target Functions

no code implementations9 Nov 2016 Bo Xie, YIngyu Liang, Le Song

In this paper, we answer these questions by analyzing one-hidden-layer neural networks with ReLU activation, and show that despite the non-convexity, neural networks with diverse units have no spurious local minima.

Relation Linking

Recovery guarantee of weighted low-rank approximation via alternating minimization

no code implementations6 Feb 2016 Yuanzhi Li, YIngyu Liang, Andrej Risteski

We show that the properties only need to hold in an average sense and can be achieved by the clipping step.

Matrix Completion

Linear Algebraic Structure of Word Senses, with Applications to Polysemy

1 code implementation TACL 2018 Sanjeev Arora, Yuanzhi Li, YIngyu Liang, Tengyu Ma, Andrej Risteski

A novel aspect of our technique is that each extracted word sense is accompanied by one of about 2000 "discourse atoms" that gives a succinct description of which other words co-occur with that word sense.

Information Retrieval Retrieval +1

Why are deep nets reversible: A simple theory, with implications for training

no code implementations18 Nov 2015 Sanjeev Arora, YIngyu Liang, Tengyu Ma

Under this assumption ---which is experimentally tested on real-life nets like AlexNet--- it is formally proved that feed forward net is a correct inference method for recovering the hidden layer.

Denoising

Scale Up Nonlinear Component Analysis with Doubly Stochastic Gradients

no code implementations NeurIPS 2015 Bo Xie, YIngyu Liang, Le Song

We propose a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD.

Communication Efficient Distributed Kernel Principal Component Analysis

no code implementations23 Mar 2015 Maria-Florina Balcan, YIngyu Liang, Le Song, David Woodruff, Bo Xie

Can we perform kernel PCA on the entire dataset in a distributed and communication efficient fashion while maintaining provable and strong guarantees in solution quality?

A Latent Variable Model Approach to PMI-based Word Embeddings

4 code implementations TACL 2016 Sanjeev Arora, Yuanzhi Li, YIngyu Liang, Tengyu Ma, Andrej Risteski

Semantic word embeddings represent the meaning of a word via a vector, and are created by diverse methods.

Word Embeddings

Learning Time-Varying Coverage Functions

no code implementations NeurIPS 2014 Nan Du, YIngyu Liang, Maria-Florina F. Balcan, Le Song

Coverage functions are an important class of discrete functions that capture laws of diminishing returns.

Improved Distributed Principal Component Analysis

no code implementations NeurIPS 2014 Maria-Florina Balcan, Vandana Kanchanapally, YIngyu Liang, David Woodruff

We give new algorithms and analyses for distributed PCA which lead to improved communication and computational costs for $k$-means clustering and related problems.

Clustering Computational Efficiency +1

Scalable Kernel Methods via Doubly Stochastic Gradients

1 code implementation NeurIPS 2014 Bo Dai, Bo Xie, Niao He, YIngyu Liang, Anant Raj, Maria-Florina Balcan, Le Song

The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems.

A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning

no code implementations9 Apr 2014 Aurélien Bellet, YIngyu Liang, Alireza Bagheri Garakani, Maria-Florina Balcan, Fei Sha

We further show that the communication cost of dFW is optimal by deriving a lower-bound on the communication cost required to construct an $\epsilon$-approximate solution.

Sparse Learning

Robust Hierarchical Clustering

no code implementations1 Jan 2014 Maria-Florina Balcan, YIngyu Liang, Pramod Gupta

One of the most widely used techniques for data clustering is agglomerative clustering.

Clustering

Budgeted Influence Maximization for Multiple Products

no code implementations8 Dec 2013 Nan Du, YIngyu Liang, Maria Florina Balcan, Le Song

The typical algorithmic problem in viral marketing aims to identify a set of influential users in a social network, who, when convinced to adopt a product, shall influence other users in the network and trigger a large cascade of adoptions.

Combinatorial Optimization Marketing

Distributed k-Means and k-Median Clustering on General Topologies

no code implementations NeurIPS 2013 Maria Florina Balcan, Steven Ehrlich, YIngyu Liang

We provide a distributed method for constructing a global coreset which improves over the previous methods by reducing the communication complexity, and which works over general communication topologies.

Clustering

Clustering under Perturbation Resilience

no code implementations5 Dec 2011 Maria Florina Balcan, YIngyu Liang

For $k$-median, a center-based objective of special interest, we additionally give algorithms for a more relaxed assumption in which we allow the optimal solution to change in a small $\epsilon$ fraction of the points after perturbation.

Clustering

Cannot find the paper you are looking for? You can Submit a new open access paper.