Search Results for author: Licong Lin

Found 10 papers, 6 papers with code

Scaling Laws in Linear Regression: Compute, Parameters, and Data

no code implementations12 Jun 2024 Licong Lin, Jingfeng Wu, Sham M. Kakade, Peter L. Bartlett, Jason D. Lee

Empirically, large-scale deep learning models often satisfy a neural scaling law: the test error of the trained model improves polynomially as the model size and data size grow.

regression

Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning

1 code implementation8 Apr 2024 Ruiqi Zhang, Licong Lin, Yu Bai, Song Mei

LLM unlearning aims to eliminate the influence of undesirable data from the pre-trained model while preserving the model's utilities on other tasks.

Mean-field variational inference with the TAP free energy: Geometric and statistical properties in linear models

no code implementations14 Nov 2023 Michael Celentano, Zhou Fan, Licong Lin, Song Mei

In settings where it is conjectured that no efficient algorithm can find this local neighborhood, we prove analogous geometric properties for a local minimizer of the TAP free energy reachable by AMP, and show that posterior inference based on this minimizer remains correctly calibrated.

Variational Inference

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

1 code implementation12 Oct 2023 Licong Lin, Yu Bai, Song Mei

This provides the first quantitative analysis of the ICRL capabilities of transformers pretrained from offline trajectories.

reinforcement-learning Thompson Sampling

Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference

1 code implementation NeurIPS 2023 Licong Lin, Mufang Ying, Suvrojit Ghosh, Koulik Khamaru, Cun-Hui Zhang

Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate estimation and have inflated error.

Plug-in Performative Optimization

no code implementations30 May 2023 Licong Lin, Tijana Zrnic

A complementary family of solutions makes use of explicit \emph{models} for the feedback, such as best-response models in strategic classification, enabling faster rates.

Semi-parametric inference based on adaptively collected data

no code implementations5 Mar 2023 Licong Lin, Koulik Khamaru, Martin J. Wainwright

Many standard estimators, when applied to adaptively collected data, fail to be asymptotically normal, thereby complicating the construction of confidence intervals.

Near-optimal multiple testing in Bayesian linear models with finite-sample FDR control

1 code implementation4 Nov 2022 Taejoo Ahn, Licong Lin, Song Mei

In this paper, we develop near-optimal multiple testing procedures for high dimensional Bayesian linear models with isotropic covariates.

Open-Ended Question Answering Variable Selection

What causes the test error? Going beyond bias-variance via ANOVA

1 code implementation11 Oct 2020 Licong Lin, Edgar Dobriban

This leads to discovering the unimodality of variance as a function of the level of parametrization, and to decomposing the variance into that arising from label noise, initialization, and randomness in the training data to understand the sources of the error.

Second-order Information in First-order Optimization Methods

1 code implementation20 Dec 2019 Yuzheng Hu, Licong Lin, Shange Tang

To the best of our knowledge, this is the first paper that seriously considers the necessity of square root among all adaptive methods.

2D Human Pose Estimation

Cannot find the paper you are looking for? You can Submit a new open access paper.