Search Results for author:
Found 22 papers, 2 papers with code
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Gaussian random field approximation via Stein's method with applications to wide random neural networks
Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level.
Textbooks Are All You Need
Despite this small scale, phi-1 attains pass@1 accuracy 50. 6% on HumanEval and 55. 5% on MBPP.
Ranked #40 on
Code Generation
on HumanEval
Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space
Of key interest in statistics and machine learning is Gaussian VI, which approximates $\pi$ by minimizing the Kullback-Leibler (KL) divergence to $\pi$ over the space of Gaussians.
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of large-scale real-world generative models such as DALL$\cdot$E 2.
Federated Learning with a Sampling Algorithm under Isoperimetry
Federated learning uses a set of techniques to efficiently distribute the training of a machine learning algorithm across several devices, who own the training data.
Improved analysis for a proximal algorithm for sampling
We study the proximal sampler of Lee, Shen, and Tian (2021) and obtain new convergence guarantees under weaker assumptions than strong log-concavity: namely, our results hold for (1) weakly log-concave targets, and (2) targets satisfying isoperimetric assumptions which allow for non-log-concavity.
Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo
For the task of sampling from a density $\pi \propto \exp(-V)$ on $\mathbb{R}^d$, where $V$ is possibly non-convex but $L$-gradient Lipschitz, we prove that averaged Langevin Monte Carlo outputs a sample with $\varepsilon$-relative Fisher information after $O( L^2 d^2/\varepsilon^2)$ iterations.
A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1
We first establish the convergence of the algorithm.
An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints
Optimization problems under affine constraints appear in various areas of machine learning.
Optimization and Control
Optimal and Practical Algorithms for Smooth and Strongly Convex Decentralized Optimization
We propose two new algorithms for this decentralized optimization problem and equip them with complexity guarantees.
A Non-Asymptotic Analysis for Stein Variational Gradient Descent
We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $\pi\propto e^{-V}$ on $\mathbb{R}^d$.
Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm
In the second part of this paper, we use the duality gap arising from the first part to study the complexity of the Proximal Stochastic Gradient Langevin Algorithm (PSGLA), which can be seen as a generalization of the Projected Langevin Algorithm.
Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms
We consider minimizing the sum of three convex functions, where the first one F is smooth, the second one is nonsmooth and proximable and the third one is the composition of a nonsmooth proximable function with a linear operator L. This template problem has many applications, for instance, in image processing and machine learning.
The Wasserstein Proximal Gradient Algorithm
Using techniques from convex optimization and optimal transport, we analyze the FB scheme as a minimization algorithm on the Wasserstein space.
Distributed Fixed Point Methods with Compressed Iterates
We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed.
Learning to Optimize via Dual space Preconditioning
Preconditioning an minimization algorithm improve its convergence and can lead to a minimizer in one iteration in some extreme cases.
Maximum Mean Discrepancy Gradient Flow
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties.
Stochastic Proximal Langevin Algorithm: Potential Splitting and Nonasymptotic Rates
We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPLA)---for sampling from a log concave distribution.
A Fully Stochastic Primal-Dual Algorithm
The proposed algorithm is proven to converge to a saddle point of the Lagrangian function.
A Constant Step Stochastic Douglas-Rachford Algorithm with Application to Non Separable Regularizations
The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions.
Snake: a Stochastic Proximal Gradient Algorithm for Regularized Problems over Large Graphs
When applying the proximal gradient algorithm to solve this problem, there exist quite affordable methods to implement the proximity operator (backward step) in the special case where the graph is a simple path without loops.
