Search Results for author:
Found 56 papers, 21 papers with code
Convex Optimization in Julia
This paper describes Convex, a convex optimization modeling framework in Julia.
Generalized Low Rank Models
Here, we extend the idea of PCA to handle arbitrary data sets consisting of numerical, Boolean, categorical, ordinal, and other data types.
AutoML using Metadata Language Embeddings
We use these embeddings in a neural architecture to learn the distance between best-performing pipelines.
OBOE: Collaborative Filtering for AutoML Model Selection
Algorithm selection and hyperparameter tuning remain two of the most challenging tasks in machine learning.
Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
Data scientists seeking a good supervised learning model on a new dataset have many choices to make: they must preprocess the data, select features, possibly reduce the dimension, select an estimation algorithm, and choose hyperparameters for each of these pipeline components.
OptiMUS: Optimization Modeling Using MIP Solvers and large language models
Optimization problems are pervasive across various sectors, from manufacturing and distribution to healthcare.
OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models
Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare.
Disciplined Multi-Convex Programming
A multi-convex optimization problem is one in which the variables can be partitioned into sets over which the problem is convex when the other variables are fixed.
Optimization and Control
ControlBurn: Feature Selection by Sparse Forests
Like the linear LASSO, ControlBurn assigns all the feature importance of a correlated group of features to a single feature.
ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles
ControlBurn is a Python package to construct feature-sparse tree ensembles that support nonlinear feature selection and interpretable machine learning.
Matrix Completion with Quantified Uncertainty through Low Rank Gaussian Copula
The time required to fit the model scales linearly with the number of rows and the number of columns in the dataset.
Low-Rank Tucker Approximation of a Tensor From Streaming Data
This paper describes a new algorithm for computing a low-Tucker-rank approximation of a tensor.
TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc.
Causal Inference with Noisy and Missing Covariates via Matrix Factorization
Valid causal inference in observational studies often requires controlling for confounders.
Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved
We also propose an alternative weighted estimator that uses soft classification, and show that its bias arises simply from the conditional covariance of the outcome with the true class membership.
SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
Numerical experiments on both ridge and logistic regression problems with dense and sparse data, show that SketchySGD equipped with its default hyperparameters can achieve comparable or better results than popular stochastic gradient methods, even when they have been tuned to yield their best performance.
PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
This paper introduces PROMISE ($\textbf{Pr}$econditioned Stochastic $\textbf{O}$ptimization $\textbf{M}$ethods by $\textbf{I}$ncorporating $\textbf{S}$calable Curvature $\textbf{E}$stimates), a suite of sketching-based preconditioned stochastic gradient algorithms for solving large-scale convex optimization problems arising in machine learning.
Why are Big Data Matrices Approximately Low Rank?
Here, we explain the effectiveness of low rank models in data science by considering a simple generative model for these matrices: we suppose that each row or column is associated to a (possibly high dimensional) bounded latent variable, and entries of the matrix are generated by applying a piecewise analytic function to these latent variables.
Practical sketching algorithms for low-rank matrix approximation
This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.
Dynamic Assortment Personalization in High Dimensions
In the dynamic setting, we show that structure-aware dynamic assortment personalization can have regret that is an order of magnitude smaller than structure-ignorant approaches.
Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data
Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.
Sketchy Decisions: Convex Low-Rank Matrix Optimization with Optimal Storage
This paper concerns a fundamental class of convex matrix optimization problems.
Revealed Preference at Scale: Learning Personalized Preferences from Assortment Choices
In our model, the preferences of each customer or segment follow a separate parametric choice model, but the underlying structure of these parameters over all the models has low dimension.
Frank-Wolfe Style Algorithms for Large Scale Optimization
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization.
The Sound of APALM Clapping: Faster Nonsmooth Nonconvex Optimization with Stochastic Asynchronous PALM
We introduce the Stochastic Asynchronous Proximal Alternating Linearized Minimization (SAPALM) method, a block coordinate stochastic proximal-gradient method for solving nonconvex, nonsmooth optimization problems.
An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity
This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Methods for interpreting machine learning black-box models increase the outcomes' transparency and in turn generates insight into the reliability and fairness of the algorithms.
Factor Group-Sparse Regularization for Efficient Low-Rank Matrix Recovery
Compared to the max norm and the factored formulation of the nuclear norm, factor group-sparse regularizers are more efficient, accurate, and robust to the initial guess of rank.
Polynomial Matrix Completion for Missing Data Imputation and Transductive Learning
This paper develops new methods to recover the missing entries of a high-rank or even full-rank matrix when the intrinsic dimension of the data is low compared to the ambient dimension.
Online high rank matrix completion
Recent advances in matrix completion enable data imputation in full-rank matrices by exploiting low dimensional (nonlinear) latent structure.
On the simplicity and conditioning of low rank semidefinite programs
It is more challenging to show that an approximate solution to the SDP formulated with noisy problem data acceptably solves the original problem; arguments are usually ad hoc for each problem setting, and can be complex.
Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering
RNLMF constructs a dictionary for the data space by factoring a kernelized feature space; a noisy matrix can then be decomposed as the sum of a sparse noise matrix and a clean data matrix that lies in a low dimensional nonlinear manifold.
Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time
Combinatorial optimization algorithms for graph problems are usually designed afresh for each new problem with careful attention by an expert to the problem structure.
$k$FW: A Frank-Wolfe style algorithm with stronger subproblem oracles
This paper proposes a new variant of Frank-Wolfe (FW), called $k$FW.
Approximate Cross-Validation with Low-Rank Data in High Dimensions
Our second key insight is that, in the presence of ALR data, error in existing ACV methods roughly grows with the (approximate, low) rank rather than with the (full, high) dimension.
An Information-Theoretic Approach to Persistent Environment Monitoring Through Low Rank Model Based Planning and Prediction
We combine a low rank model of a target attribute with an information-maximizing path planner to predict the state of the attribute throughout a region.
Online Missing Value Imputation and Change Point Detection with the Gaussian Copula
Missing value imputation is crucial for real-world data science workflows.
Real-Time AutoML
We train a graph neural network in which each node represents a dataset to predict the best machine learning pipeline for a new test dataset.
Impact of Accuracy on Model Interpretations
Model interpretations are often used in practice to extract real world insights from machine learning models.
Euclidean-Norm-Induced Schatten-p Quasi-Norm Regularization for Low-Rank Tensor Completion and Tensor Robust Principal Component Analysis
The theorems show that a relatively sharper regularizer leads to a tighter error bound, which is consistent with our numerical results.
TenIPS: Inverse Propensity Sampling for Tensor Completion
Tensors are widely used to represent multiway arrays of data.
Tensor Random Projection for Low Memory Dimension Reduction
The TRP map is formed as the Khatri-Rao product of several smaller random projections, and is compatible with any base random projection including sparse maps, which enable dimension reduction with very low query cost and no floating point operations.
How Low Can We Go: Trading Memory for Error in Low-Precision Training
Low-precision arithmetic trains deep learning models using less energy, less memory and less time.
Privileged Zero-Shot AutoML
This work improves the quality of automated machine learning (AutoML) systems by using dataset and function descriptions while significantly decreasing computation time from minutes to milliseconds by using a zero-shot approach.
Can we globally optimize cross-validation loss? Quasiconvexity in ridge regression
In the present paper, we show that, in the case of ridge regression, the CV loss may fail to be quasiconvex and thus may have multiple local optima.
Invariant Learning with Partial Group Labels
Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
GalaxyTSP: A New Billion-Node Benchmark for TSP
We approximate a Traveling Salesman Problem (TSP) three orders of magnitude larger than the largest known benchmark, increasing the number of nodes from millions to billions.
CDF Normalization for Controlling the Distribution of Hidden Nodes
Batch Normalizaiton (BN) is a normalization method for deep neural networks that has been shown to accelerate training.
Towards Group Robustness in the presence of Partial Group Labels
Such a requirement is impractical in situations where the data labeling efforts for minority or rare groups are significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.
Data-Efficient and Interpretable Tabular Anomaly Detection
In addition, the proposed framework, DIAD, can incorporate a small amount of labeled data to further boost anomaly detection performances in semi-supervised settings.
From Human Days to Machine Seconds: Automatically Answering and Generating Machine Learning Final Exams
We curate a dataset and benchmark of questions from machine learning final exams available online and code for answering these questions and generating new questions.
The Missing Indicator Method: From Low to High Dimensions
In this paper, we show empirically and theoretically that MIM improves performance for informative missing values, and we prove that MIM does not hurt linear models asymptotically for uninformative missing values.
Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models
We curate a comprehensive dataset of 4, 550 questions and solutions from problem sets, midterm exams, and final exams across all MIT Mathematics and Electrical Engineering and Computer Science (EECS) courses required for obtaining a degree.
Interpretable Survival Analysis for Heart Failure Risk Prediction
Specifically, we use an improved version of survival stacking to transform a survival analysis problem to a classification problem, ControlBurn to perform feature selection, and Explainable Boosting Machines to generate interpretable predictions.
Challenges in Training PINNs: A Loss Landscape Perspective
This paper explores challenges in training Physics-Informed Neural Networks (PINNs), emphasizing the role of the loss landscape in the training process.
