Search Results for author:
Found 59 papers, 19 papers with code
A Theoretical Framework for Prompt Engineering: Approximating Smooth Functions with Transformer Prompts
Prompt engineering has emerged as a powerful technique for guiding large language models (LLMs) toward desired responses, significantly enhancing their performance across diverse tasks.
An Overview of Large Language Models for Statisticians
Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI), exhibiting remarkable capabilities across diverse tasks such as text generation, reasoning, and decision-making.
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
Reinforcement Learning from Human Feedback (RLHF) has shown promise in aligning large language models (LLMs).
RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
Large language models (LLMs) have achieved impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings.
A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models
To address this issue, we propose embedding watermarks into the copyrighted training data and formulating the detection of data misappropriation as a hypothesis testing problem.
S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Utilizing this key insight, we propose a family of Structured Sparse Fine-Tuning (S$^{2}$FT) methods for LLMs, which concurrently achieve state-of-the-art fine-tuning performance, training efficiency, and inference scalability.
Differentially Private Learning Beyond the Classical Dimensionality Regime
We provide sharp theoretical estimates of the error of several well-studied differentially private algorithms for robust linear regression and logistic regression, including output perturbation, objective perturbation, and noisy stochastic gradient descent, in the proportional dimensionality regime.
FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees
The propensity of Large Language Models (LLMs) to generate hallucinations and non-factual content undermines their reliability in high-stakes domains, where rigorous control over Type I errors (the conditional probability of incorrectly classifying hallucinations as truthful content) is essential.
Finite-Sample and Distribution-Free Fair Classification: Optimal Trade-off Between Excess Risk and Fairness, and the Cost of Group-Blindness
Algorithmic fairness in machine learning has recently garnered significant attention.
MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models
Artificial Intelligence (AI) has demonstrated significant potential in healthcare, particularly in disease diagnosis and treatment planning.
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models
The nonlinear approximation directly models the cumulative updates, effectively capturing complex and non-linear structures in the weight updates.
RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models
Second, in cases where the model originally responds correctly, applying RAG can lead to an over-reliance on retrieved contexts, resulting in incorrect answers.
F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data
In this paper, we propose a novel approach that leverages strategically chosen proxy data reflective of potential sales patterns from similar entities during non-peak periods, enriched by features learned from a graph neural networks (GNNs)-based forecasting model, to predict demand during peak events.
Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance
In the context of imbalanced data, LLMs are used to oversample underrepresented groups and have shown promising improvements.
Order-Independence Without Fine Tuning
One problematic inconsistency when LLMs are used to answer multiple-choice questions or analyze multiple inputs is order dependency: the output of an LLM can (and often does) change significantly when sub-sequences are swapped, despite both orderings being semantically identical.
Calibrated Self-Rewarding Vision Language Models
In the reward modeling, we employ a step-wise strategy and incorporate visual constraints into the self-rewarding process to place greater emphasis on visual input.
Ranked #127 on
Visual Question Answering
on MM-Vet
Fair Risk Control: A Generalized Framework for Calibrating Multi-group Fairness Risks
This paper introduces a framework for post-processing machine learning models so that their predictions satisfy multi-group fairness guarantees.
Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference
Differentially private federated learning is crucial for maintaining privacy in distributed environments.
A Unified Combination Framework for Dependent Tests with Applications to Microbiome Association Studies
Compared to the existing $p$-value combination methods, including the vanilla Cauchy combination method, the proposed combination framework can handle the dependence accurately and utilizes the information efficiently to construct tests with accurate size and enhanced power.
FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality
Machine learning methods often assume that the test data have the same distribution as the training data.
Contrastive Learning on Multimodal Analysis of Electronic Health Records
To accommodate the statistical analysis of multimodal EHR data, in this paper, we propose a novel multimodal feature embedding generative model and design a multimodal contrastive loss to obtain the multimodal EHR feature representation.
Provable Multi-Party Reinforcement Learning with Diverse Human Feedback
Our work \textit{initiates} the theoretical study of multi-party RLHF that explicitly models the diverse preferences of multiple individuals.
Distribution-Free Fair Federated Learning with Small Samples
As federated learning gains increasing importance in real-world applications due to its capacity for decentralized data training, addressing fairness concerns across demographic groups becomes critically important.
Selective Learning: Towards Robust Calibration with Dynamic Regularization
The objective can be understood as seeking a model that fits the ground-truth labels by increasing the confidence while also maximizing the entropy of predicted probabilities by decreasing the confidence.
Differentially Private Sliced Inverse Regression: Minimax Optimality and Algorithm
Privacy preservation has become a critical concern in high-dimensional data analysis due to the growing prevalence of data-driven applications.
Can AI Be as Creative as Humans?
With the rise of advanced generative AI models capable of tasks once reserved for human creativity, the study of AI's creative potential becomes imperative for its responsible development and application.
Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges
To bridge this gap, we introduce a new benchmark, namely, the Bias and Interference Challenges in Visual Language Models (Bingo).
Conformal Prediction for Deep Classifier via Label Ranking
Conformal prediction is a statistical framework that generates prediction sets containing ground-truth labels with a desired coverage guarantee.
Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
Large vision-language models (LVLMs) have shown remarkable abilities in understanding visual information with human languages.
Multi-dimensional domain generalization with low-rank structures
In conventional statistical and machine learning methods, it is typically assumed that the test data are identically distributed with the training data.
What Should Data Science Education Do with Large Language Models?
LLMs can also play a significant role in the classroom as interactive teaching and learning tools, contributing to personalized education.
Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training
The surge in multimodal AI's success has sparked concerns over data privacy in vision-and-language tasks.
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Deep neural networks often rely on spurious correlations to make predictions, which hinders generalization beyond training environments.
Score Attack: A Lower Bound Technique for Optimal Differentially Private Learning
The score attack method is based on the tracing attack concept in differential privacy and can be applied to any statistical model with a well-defined score statistic.
HappyMap: A Generalized Multi-calibration Method
Fairness is captured by incorporating demographic subgroups into the class of functions~$\mathcal{C}$.
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data
We show that the algorithm can detect the ground-truth pairs and improve performance by fully exploiting unpaired datasets.
FaiREE: Fair Classification with Finite-Sample and Distribution-Free Guarantee
Several group fairness notions and algorithms have been proposed.
Reinforcement Learning with Stepwise Fairness Constraints
AI methods are used in societally important settings, ranging from credit to employment to housing, and it is crucial to provide fairness in regard to algorithmic decision making.
Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise
This opens a promising strategy to first train a feature learner rather than a classifier, and then perform linear probing (last layer retraining) in the test environment.
C-Mixup: Improving Generalization in Regression
In this paper, we propose a simple yet powerful algorithm, C-Mixup, to improve generalization on regression tasks.
FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Data
Specifically, FIFA encourages both classification and fairness generalization and can be flexibly combined with many existing fair learning methods with logits-based losses.
Improving Out-of-Distribution Robustness via Selective Augmentation
Machine learning algorithms typically assume that training and test examples are drawn from the same distribution.
The Power of Contrast for Feature Learning: A Theoretical Analysis
Contrastive learning has achieved state-of-the-art performance in various self-supervised learning tasks and even outperforms its supervised counterpart.
Understanding Dynamics of Nonlinear Representation Learning and Its Application
Representation learning allows us to automatically discover suitable representations from raw sensory data.
Adversarial Training Helps Transfer Learning via Better Representations
Recent works empirically demonstrate that adversarial training in the source data can improve the ability of models to transfer to new domains.
Meta-Learning with Fewer Tasks through Task Interpolation
Meta-learning enables algorithms to quickly learn a newly encountered task with just a few labeled examples by transferring previously learned knowledge.
High-Dimensional Differentially-Private EM Algorithm: Methods and Near-Optimal Statistical Guarantees
In this paper, we develop a general framework to design differentially private expectation-maximization (EM) algorithms in high-dimensional latent variable models, based on the noisy iterative hard-thresholding.
A Central Limit Theorem for Differentially Private Query Answering
The central question, therefore, is to understand which noise distribution optimizes the privacy-accuracy trade-off, especially when the dimension of the answer vector is high.
When and How Mixup Improves Calibration
In addition, we study how Mixup improves calibration in semi-supervised learning.
The Cost of Privacy in Generalized Linear Models: Algorithms and Minimax Lower Bounds
We propose differentially private algorithms for parameter estimation in both low-dimensional and high-dimensional sparse generalized linear models (GLMs) by constructing private versions of projected gradient descent.
Estimation, Confidence Intervals, and Large-Scale Hypotheses Testing for High-Dimensional Mixed Linear Regression
Based on the iterative estimators, we further construct debiased estimators and establish their asymptotic normality.
How Does Mixup Help With Robustness and Generalization?
For robustness, we show that minimizing the Mixup loss corresponds to approximately minimizing an upper bound of the adversarial loss.
Interpreting Robust Optimization via Adversarial Influence Functions
Robust optimization has been widely used in nowadays data science, especially in adversarial training.
A Lightweight Algorithm to Uncover Deep Relationships in Data Tables
The key to our approach is a computationally lightweight forward addition algorithm that we developed to recursively extract the functional dependencies between table columns that are scalable to tables with many columns.
Improving Generalization in Meta-learning via Task Augmentation
Moreover, both MetaMix and Channel Shuffle outperform state-of-the-art results by a large margin across many datasets and are compatible with existing meta-learning algorithms.
Improving Adversarial Robustness via Unlabeled Out-of-Domain Data
In this work, we investigate how adversarial robustness can be enhanced by leveraging out-of-domain unlabeled data.
The Cost of Privacy: Optimal Rates of Convergence for Parameter Estimation with Differential Privacy
By refining the "tracing adversary" technique for lower bounds in the theoretical computer science literature, we formulate a general lower bound argument for minimax risks with differential privacy constraints, and apply this argument to high-dimensional mean estimation and linear regression problems.
A Sparse PCA Approach to Clustering
We discuss a clustering method for Gaussian mixture model based on the sparse principal component analysis (SPCA) method and compare it with the IF-PCA method.
