Search Results for author:
Found 68 papers, 28 papers with code
Understanding Fixed Predictions via Confined Regions
Consequently, models can assign fixed predictions that deny individuals recourse to change their outcome.
Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
We provide the first rigorous convergence analysis of HDM using the online learning framework of [Gao24] and apply this analysis to develop new state-of-the-art adaptive gradient methods with empirical and theoretical support.
Diverse Inference and Verification for Advanced Reasoning
Reasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding, yet find challenging advanced tasks such as International Mathematical Olympiad (IMO) combinatorics problems, Abstraction and Reasoning Corpus (ARC) puzzles, and Humanity's Last Exam (HLE) questions.
Enhancing Physics-Informed Neural Networks Through Feature Engineering
It consistently uses fewer parameters -- on average, 65% fewer than the competing feature engineering methods -- while achieving comparable accuracy in less than 30% of the training epochs.
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning
Regularized empirical risk minimization (rERM) has become important in data-intensive fields such as genomics and advertising, with stochastic gradient methods typically used to solve the largest problems.
LLMs for Cold-Start Cutting Plane Separator Configuration
Mixed integer linear programming (MILP) solvers ship with a staggering number of parameters that are challenging to select a priori for all but expert optimization users, but can have an outsized impact on the performance of the MILP solver.
DNAMite: Interpretable Calibrated Survival Analysis with Discretized Additive Models
Most machine learning models for survival analysis are black-box models, limiting their use in healthcare settings where interpretability is paramount.
Gradient Methods with Online Scaling
We introduce a framework to accelerate the convergence of gradient-based methods with online learning.
AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviews
We evaluate the alignment of automatic paper reviews with human reviews using an arena of human preferences by pairwise comparisons.
OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare.
Have ASkotch: A Neat Solution for Large-scale Kernel Ridge Regression
ASkotch outperforms state-of-the-art KRR solvers on a testbed of 23 large-scale KRR regression and classification tasks derived from a wide range of application domains, demonstrating the superiority of full KRR over inducing points KRR.
Interpretable Prediction and Feature Selection for Survival Analysis
While DyS works well for all survival analysis problems, it is particularly useful for large (in $n$ and $p$) survival datasets such as those commonly found in observational healthcare studies.
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare.
Challenges in Training PINNs: A Loss Landscape Perspective
This paper explores challenges in training Physics-Informed Neural Networks (PINNs), emphasizing the role of the loss landscape in the training process.
Interpretable Survival Analysis for Heart Failure Risk Prediction
Specifically, we use an improved version of survival stacking to transform a survival analysis problem to a classification problem, ControlBurn to perform feature selection, and Explainable Boosting Machines to generate interpretable predictions.
OptiMUS: Optimization Modeling Using MIP Solvers and large language models
Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare.
PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
This paper introduces PROMISE ($\textbf{Pr}$econditioned Stochastic $\textbf{O}$ptimization $\textbf{M}$ethods by $\textbf{I}$ncorporating $\textbf{S}$calable Curvature $\textbf{E}$stimates), a suite of sketching-based preconditioned stochastic gradient algorithms for solving large-scale convex optimization problems arising in machine learning.
Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models
We curate a comprehensive dataset of 4, 550 questions and solutions from problem sets, midterm exams, and final exams across all MIT Mathematics and Electrical Engineering and Computer Science (EECS) courses required for obtaining a degree.
SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
Numerical experiments on both ridge and logistic regression problems with dense and sparse data, show that SketchySGD equipped with its default hyperparameters can achieve comparable or better results than popular stochastic gradient methods, even when they have been tuned to yield their best performance.
The Missing Indicator Method: From Low to High Dimensions
In this paper, we show empirically and theoretically that MIM improves performance for informative missing values, and we prove that MIM does not hurt linear models asymptotically for uninformative missing values.
ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles
ControlBurn is a Python package to construct feature-sparse tree ensembles that support nonlinear feature selection and interpretable machine learning.
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
We curate a dataset and benchmark of questions from machine learning final exams available online and code for answering these questions and generating new questions.
TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc.
Data-Efficient and Interpretable Tabular Anomaly Detection
In addition, the proposed framework, DIAD, can incorporate a small amount of labeled data to further boost anomaly detection performances in semi-supervised settings.
Towards Group Robustness in the presence of Partial Group Labels
Such a requirement is impractical in situations where the data labeling efforts for minority or rare groups are significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
Invariant Learning with Partial Group Labels
Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
CDF Normalization for Controlling the Distribution of Hidden Nodes
Batch Normalizaiton (BN) is a normalization method for deep neural networks that has been shown to accelerate training.
Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression
In the present paper, we show that, in the case of ridge regression, the CV loss may fail to be quasiconvex and thus may have multiple local optima.
ControlBurn: Feature Selection by Sparse Forests
Like the linear LASSO, ControlBurn assigns all the feature importance of a correlated group of features to a single feature.
Privileged Zero-Shot AutoML
This work improves the quality of automated machine learning (AutoML) systems by using dataset and function descriptions while significantly decreasing computation time from minutes to milliseconds by using a zero-shot approach.
How Low Can We Go: Trading Memory for Error in Low-Precision Training
Low-precision arithmetic trains deep learning models using less energy, less memory and less time.
Tensor Random Projection for Low Memory Dimension Reduction
The TRP map is formed as the Khatri-Rao product of several smaller random projections, and is compatible with any base random projection including sparse maps, which enable dimension reduction with very low query cost and no floating point operations.
Real-Time AutoML
We train a graph neural network in which each node represents a dataset to predict the best machine learning pipeline for a new test dataset.
TenIPS: Inverse Propensity Sampling for Tensor Completion
Tensors are widely used to represent multiway arrays of data.
Euclidean-Norm-Induced Schatten-p Quasi-Norm Regularization for Low-Rank Tensor Completion and Tensor Robust Principal Component Analysis
The theorems show that a relatively sharper regularizer leads to a tighter error bound, which is consistent with our numerical results.
Impact of Accuracy on Model Interpretations
Model interpretations are often used in practice to extract real world insights from machine learning models.
GalaxyTSP: A New Billion-Node Benchmark for TSP
We approximate a Traveling Salesman Problem (TSP) three orders of magnitude larger than the largest known benchmark, increasing the number of nodes from millions to billions.
Online Missing Value Imputation and Change Point Detection with the Gaussian Copula
Missing value imputation is crucial for real-world data science workflows.
An Information-Theoretic Approach to Persistent Environment Monitoring Through Low Rank Model Based Planning and Prediction
We combine a low rank model of a target attribute with an information-maximizing path planner to predict the state of the attribute throughout a region.
Approximate Cross-Validation with Low-Rank Data in High Dimensions
Our second key insight is that, in the presence of ALR data, error in existing ACV methods roughly grows with the (approximate, low) rank rather than with the (full, high) dimension.
$k$FW: A Frank-Wolfe style algorithm with stronger subproblem oracles
This paper proposes a new variant of Frank-Wolfe (FW), called $k$FW.
Matrix Completion with Quantified Uncertainty through Low Rank Gaussian Copula
The time required to fit the model scales linearly with the number of rows and the number of columns in the dataset.
Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
Data scientists seeking a good supervised learning model on a new dataset have many choices to make: they must preprocess the data, select features, possibly reduce the dimension, select an estimation algorithm, and choose hyperparameters for each of these pipeline components.
Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time
Combinatorial optimization algorithms for graph problems are usually designed afresh for each new problem with careful attention by an expert to the problem structure.
Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering
RNLMF constructs a dictionary for the data space by factoring a kernelized feature space; a noisy matrix can then be decomposed as the sum of a sparse noise matrix and a clean data matrix that lies in a low dimensional nonlinear manifold.
On the simplicity and conditioning of low rank semidefinite programs
It is more challenging to show that an approximate solution to the SDP formulated with noisy problem data acceptably solves the original problem; arguments are usually ad hoc for each problem setting, and can be complex.
Online high rank matrix completion
Recent advances in matrix completion enable data imputation in full-rank matrices by exploiting low dimensional (nonlinear) latent structure.
Polynomial Matrix Completion for Missing Data Imputation and Transductive Learning
This paper develops new methods to recover the missing entries of a high-rank or even full-rank matrix when the intrinsic dimension of the data is low compared to the ambient dimension.
Factor Group-Sparse Regularization for Efficient Low-Rank Matrix Recovery
Compared to the max norm and the factored formulation of the nuclear norm, factor group-sparse regularizers are more efficient, accurate, and robust to the initial guess of rank.
AutoML using Metadata Language Embeddings
We use these embeddings in a neural architecture to learn the distance between best-performing pipelines.
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Methods for interpreting machine learning black-box models increase the outcomes' transparency and in turn generates insight into the reliability and fairness of the algorithms.
Low-Rank Tucker Approximation of a Tensor From Streaming Data
This paper describes a new algorithm for computing a low-Tucker-rank approximation of a tensor.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity
This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.
Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved
We also propose an alternative weighted estimator that uses soft classification, and show that its bias arises simply from the conditional covariance of the outcome with the true class membership.
Frank-Wolfe Style Algorithms for Large Scale Optimization
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization.
OBOE: Collaborative Filtering for AutoML Model Selection
Algorithm selection and hyperparameter tuning remain two of the most challenging tasks in machine learning.
Causal Inference with Noisy and Missing Covariates via Matrix Factorization
Valid causal inference in observational studies often requires controlling for confounders.
Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data
Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.
Why are Big Data Matrices Approximately Low Rank?
Here, we explain the effectiveness of low rank models in data science by considering a simple generative model for these matrices: we suppose that each row or column is associated to a (possibly high dimensional) bounded latent variable, and entries of the matrix are generated by applying a piecewise analytic function to these latent variables.
Sketchy Decisions: Convex Low-Rank Matrix Optimization with Optimal Storage
This paper concerns a fundamental class of convex matrix optimization problems.
The Sound of APALM Clapping: Faster Nonsmooth Nonconvex Optimization with Stochastic Asynchronous PALM
We introduce the Stochastic Asynchronous Proximal Alternating Linearized Minimization (SAPALM) method, a block coordinate stochastic proximal-gradient method for solving nonconvex, nonsmooth optimization problems.
Dynamic Assortment Personalization in High Dimensions
In the dynamic setting, we show that structure-aware dynamic assortment personalization can have regret that is an order of magnitude smaller than structure-ignorant approaches.
Disciplined Multi-Convex Programming
A multi-convex optimization problem is one in which the variables can be partitioned into sets over which the problem is convex when the other variables are fixed.
Optimization and Control
Practical sketching algorithms for low-rank matrix approximation
This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.
Revealed Preference at Scale: Learning Personalized Preferences from Assortment Choices
In our model, the preferences of each customer or segment follow a separate parametric choice model, but the underlying structure of these parameters over all the models has low dimension.
Convex Optimization in Julia
This paper describes Convex, a convex optimization modeling framework in Julia.
Generalized Low Rank Models
Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types.
