Search Results for author:
Found 14 papers, 8 papers with code
Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences
We prove that, if the data is curated according to a reward model, then the expected reward of the iterative retraining procedure is maximized.
On the Stability of Iterative Retraining of Generative Models on their own Data
In this paper, we develop a framework to rigorously study the impact of training generative models on mixed datasets -- from classical training on real data to self-consuming generative models trained on purely synthetic data.
Omega: Optimistic EMA Gradients
Stochastic min-max optimization has gained interest in the machine learning community with the advancements in GANs and adversarial training.
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited.
The Curse of Unrolling: Rate of Differentiating Through Optimization
Computing the Jacobian of the solution of an optimization problem is a central problem in machine learning, with applications in hyperparameter optimization, meta-learning, optimization as a layer, and dataset distillation, to name a few.
On the Limitations of Elo: Real-World Games, are Transitive, not Additive
In this study, we investigate the challenge of identifying the strength of the transitive component in games.
Beyond L1: Faster and Better Sparse Models with skglm
We propose a new fast algorithm to estimate any sparse generalized linear model with convex or non-convex separable penalties.
Implicit differentiation for fast hyperparameter selection in non-smooth convex learning
Finding the optimal hyperparameters of a model can be cast as a bilevel optimization problem, typically solved using zero-order techniques.
Anderson acceleration of coordinate descent
Acceleration of first order methods is mainly obtained via inertial techniques \`a la Nesterov, or via nonlinear extrapolation.
Model identification and local linear convergence of coordinate descent
For composite nonsmooth optimization problems, Forward-Backward algorithm achieves model identification (e. g. support identification for the Lasso) after a finite number of iterations, provided the objective function is regular enough.
Implicit differentiation of Lasso-type models for hyperparameter optimization
Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
Support recovery and sup-norm convergence rates for sparse pivotal estimation
In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level.
Handling correlated and repeated measurements with the smoothed multivariate square-root Lasso
Sparsity promoting norms are frequently used in high dimensional regression.
Anytime Exact Belief Propagation
In this paper we present work in progress on an Anytime Exact Belief Propagation algorithm that is very similar to Belief Propagation but is exact even for graphical models with cycles, while exhibiting soft short-circuiting, amortized constant time complexity in the model size, and which can provide probabilistic proof trees.
