Search Results for author:
Found 43 papers, 16 papers with code
L3Ms -- Lagrange Large Language Models
Supervised fine-tuning (SFT) and alignment of large language models (LLMs) are key steps in providing a good user experience.
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
Diffusion models that are based on iterative denoising have been recently proposed and leveraged in various generation tasks like image generation.
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition
This work proposes POMP, a prompt pre-training method for vision-language models.
RLSbench: Domain Adaptation Under Relaxed Label Shift
Despite the emergence of principled methods for domain adaptation under label shift, their sensitivity to shifts in class conditional distributions is precariously under explored.
Multimodal Chain-of-Thought Reasoning in Language Models
Experimental results on ScienceQA and A-OKVQA benchmark datasets show the effectiveness of our proposed approach.
Ranked #4 on
Science Question Answering
on ScienceQA
Parameter-Efficient Fine-Tuning Design Spaces
We discover the following design patterns: (i) group layers in a spindle pattern; (ii) allocate the number of trainable parameters to layers uniformly; (iii) tune all the groups; (iv) assign proper tuning strategies to different groups.
Automatic Chain of Thought Prompting in Large Language Models
Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting.
Ranked #4 on
Question Answering
on TruthfulQA
(EM metric)
Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition
However, in real-world applications, it is common for the training sets to have long-tailed distributions.
Mixture Proportion Estimation and PU Learning: A Modern Approach
Formally, this task is broken down into two subtasks: (i) Mixture Proportion Estimation (MPE) -- determining the fraction of positive examples in the unlabeled data; and (ii) PU-learning -- given such an estimate, learning the desired positive-versus-negative classifier.
Mixture Proportion Estimation and PU Learning:A Modern Approach
Formally, this task is broken down into two subtasks: (i) Mixture Proportion Estimation (MPE)---determining the fraction of positive examples in the unlabeled data; and (ii) PU-learning---given such an estimate, learning the desired positive-versus-negative classifier.
Multimodal AutoML on Structured Tables with Text Fields
We design automated supervised learning systems for data tables that not only contain numeric/categorical columns, but text fields as well.
Explore with Dynamic Map: Graph Structured Reinforcement Learning
In reinforcement learning, a map with states and transitions built based on historical trajectories is often helpful in exploration and exploitation.
Regioned Episodic Reinforcement Learning
Goal-oriented reinforcement learning algorithms are often good at exploration, not exploitation, while episodic algorithms excel at exploitation, not exploration.
TraDE: A Simple Self-Attention-Based Density Estimator
We present TraDE, a self-attention-based architecture for auto-regressive density estimation with continuous and discrete valued data.
Tiering as a Stochastic Submodular Optimization Problem
Tiering is an essential technique for building large-scale information retrieval systems.
Recognizing Variables from their Data via Deep Embeddings of Distributions
A key obstacle in automated analytics and meta-learning is the inability to recognize when different datasets contain measurements of the same variable.
Deep Factors for Forecasting
We provide both theoretical and empirical evidence for the soundness of our approach through a necessary and sufficient decomposition of exchangeable time series into a global and a local part.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Deep Factors with Gaussian Processes for Forecasting
A large collection of time series poses significant challenges for classical and neural forecasting approaches.
Learning Steady-States of Iterative Algorithms over Graphs
Many graph analytics problems can be solved via iterative algorithms where the solutions are often characterized by a set of steady-state conditions.
Detecting and Correcting for Label Shift with Black Box Predictors
Faced with distribution shift between training and test set, we wish to detect and quantify the shift, and to correct our classifiers without test set labels.
Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information.
Canopy --- Fast Sampling with Cover Trees
In this work, we propose Canopy, a sampler based on Cover Trees that is exact, has guaranteed runtime logarithmic in the number of atoms, and is provably polynomial in the inherent dimensionality of the underlying parameter space.
Efficient Multitask Feature and Relationship Learning
In this paper, we consider a formulation of multitask learning that learns the relationships both between tasks and between features, represented through a task covariance and a feature covariance matrix, respectively.
Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy
In this context, the MMD may be used in two roles: first, as a discriminator, either directly on the samples, or on features of the samples.
AIDE: Fast and Communication Efficient Distributed Optimization
It is well known that DANE algorithm does not match the communication complexity lower bounds.
Stochastic Frank-Wolfe Methods for Nonconvex Optimization
Finally, we show that the faster convergence rates of our variance reduced methods also translate into improved convergence rates for the stochastic setting.
Neural Machine Translation with Recurrent Attention Modeling
Knowing which words have been attended to in previous time steps while generating a translation is a rich source of information for predicting what words will be attended to in the future.
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization
This paper builds upon our recent series of papers on fast stochastic methods for smooth nonconvex optimization [22, 23], with a novel analysis for nonconvex and nonsmooth functions.
Fast Incremental Method for Nonconvex Optimization
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form $\min_x \sum_i f_i(x)$.
Stochastic Variance Reduction for Nonconvex Optimization
We study nonconvex finite-sum problems and analyze stochastic variance reduced gradient (SVRG) methods for them.
Data Driven Resource Allocation for Distributed Learning
In distributed machine learning, data is dispatched to multiple machines for processing.
Stacked Attention Networks for Image Question Answering
Thus, we develop a multiple-layer SAN in which we query an image multiple times to infer the answer progressively.
Ranked #5 on
Visual Question Answering (VQA)
on VQA v1 test-std
On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants
We demonstrate the empirical performance of our method through a concrete realization of asynchronous SVRG.
Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo
We consider the problem of Bayesian learning on sensitive datasets and present two simple but somewhat surprising results that connect Bayesian learning to "differential privacy:, a cryptographic approach to protect individual-level privacy while permiting database-level utility.
Deep Fried Convnets
The fully connected layers of a deep convolutional neural network typically contain over 90% of the network parameters, and consume the majority of the memory required to store the network parameters.
Ranked #44 on
Image Classification
on MNIST
Trend Filtering on Graphs
We introduce a family of adaptive estimators on graphs, based on penalizing the $\ell_1$ norm of discrete graph differences.
The Falling Factorial Basis and Its Statistical Applications
We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis.
Randomized Nonlinear Component Analysis
Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale.
Super-Samples from Kernel Herding
We extend the herding algorithm to continuous spaces by using the kernel trick.
Feature Hashing for Large Scale Multitask Learning
Empirical evidence suggests that hashing is an effective strategy for dimensionality reduction and practical nonparametric estimation.
