Search Results for author:
Found 51 papers, 13 papers with code
Optimization and Analysis of the pAp@k Metric for Recommender Systems
Modern recommendation and notification systems must be robust to data imbalance, limitations on the number of recommendations/notifications, and heterogeneous engagement profiles across users.
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
Proxy Methods for Domain Adaptation
We study the problem of domain adaptation under distribution shift, where the shift is due to a change in the distribution of an unobserved, latent variable that confounds both the covariates and the labels.
The Case for Globalizing Fairness: A Mixed Methods Study on Colonialism, AI, and Health in Africa
Whereas experts generally expressed a shared view about the relevance of colonial history for the development and implementation of AI technologies in Africa, the majority of the general population participants surveyed did not think there was a direct link between AI and colonialism.
Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models
Recent advancements in large language models (LLMs) have underscored their importance in the evolution of artificial intelligence.
Robustness to Subpopulation Shift with Domain Label Noise via Regularized Annotation of Domains
Existing methods for last layer retraining that aim to optimize worst-group accuracy (WGA) rely heavily on well-annotated groups in the training data.
Bridging Associative Memory and Probabilistic Modeling
Based on the observation that associative memory's energy functions can be seen as probabilistic modeling's negative log likelihoods, we build a bridge between the two that enables useful flow of ideas in both directions.
Rethinking Machine Unlearning for Large Language Models
We explore machine unlearning (MU) in the domain of large language models (LLMs), referred to as LLM unlearning.
Scaling Laws for Downstream Task Performance of Large Language Models
With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data.
Transforming and Combining Rewards for Aligning Large Language Models
A common approach for aligning language models to human preferences is to first learn a reward model from preference data, and then use this reward model to update the language model.
Investigating Data Contamination for Pre-training Language Models
Language models pre-trained on web-scale corpora demonstrate impressive capabilities on diverse downstream tasks.
Disentangling Fact from Grid Cell Fiction in Trained Deep Path Integrators
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i. e., track one's spatial position by integrating self-velocity signals.
What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
Using a combination of theory and experiments, we show that incidental polysemanticity can arise due to multiple reasons including regularization and neural noise; this incidental polysemanticity occurs because random initialization can, by chance alone, initially assign multiple features to the same neuron, and the training dynamics then strengthen such overlap.
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
At both the population and single-cell levels, we find evidence suggesting that neither of the assumptions are likely true in biological neural representations.
Learning to (Learn at Test Time)
Our inner loop turns out to be equivalent to linear attention when the inner-loop learner is only a linear model, and to self-attention when it is a kernel estimator.
Representation Engineering: A Top-Down Approach to AI Transparency
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
Ranked #3 on
Question Answering
on TruthfulQA
Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting
We find that the logically \textit{invalid} reasoning prompts do indeed achieve similar performance gains on BBH tasks as logically valid reasoning prompts.
FACADE: A Framework for Adversarial Circuit Anomaly Detection and Evaluation
We present FACADE, a novel probabilistic and geometric framework designed for unsupervised mechanistic anomaly detection in deep neural networks.
Deceptive Alignment Monitoring
As the capabilities of large machine learning models continue to grow, and as the autonomy afforded to such models continues to expand, the spectre of a new adversary looms: the models themselves.
Is Pre-training Truly Better Than Meta-Learning?
Using this analysis, we demonstrate the following: 1. when the formal diversity of a data set is low, PT beats MAML on average and 2. when the formal diversity is high, MAML beats PT on average.
Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data
Current trends to pre-train capable Large Language Models (LLMs) mostly focus on scaling of model and dataset size.
Communication-Efficient Federated Learning through Importance Sampling
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL).
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly.
Pairwise Ranking Losses of Click-Through Rates Prediction for Welfare Maximization in Ad Auctions
We study the design of loss functions for click-through rates (CTR) to optimize (social) welfare in advertising auctions.
HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
To address texture flickering issues in NeRFs, we introduce a kernel smoothing technique that refines importance sampling weights coarse-to-fine, ensuring accurate and thorough sampling in high-density regions.
Are Emergent Abilities of Large Language Models a Mirage?
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models.
Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africa
With growing machine learning (ML) applications in healthcare, there have been calls for fairness in ML to understand and mitigate ethical concerns these systems may pose.
Finite-sample Guarantees for Nash Q-learning with Linear Function Approximation
Nash Q-learning may be considered one of the first and most known algorithms in multi-agent reinforcement learning (MARL) for learning policies that constitute a Nash equilibrium of an underlying general-sum Markov game.
Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting
Further, we propose a novel lightweight aggregation rule, Federated Gradient Projection ($\texttt{FedGP}$), which significantly improves the target performance with domain shift and data scarcity.
Adapting to Latent Subgroup Shifts via Concepts and Proxies
We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain, and unlabeled data from the target.
Latent Multimodal Functional Graphical Model Estimation
Joint multimodal functional data acquisition, where functional data from multiple modes are measured simultaneously from the same subject, has emerged as an exciting modern approach enabled by recent engineering breakthroughs in the neurological and biological sciences.
Invariant Aggregator for Defending against Federated Backdoor Attacks
Federated learning enables training high-utility models across several clients without directly sharing their private data.
Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attacks
We then provide two robustness certification criteria: certified prediction and certified attack inefficacy for DPFL on both user and instance levels.
The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence
This novel insight contextualizes claims that transfer learning solutions are better than meta-learned solutions in the regime of low diversity under a fair comparison.
One Policy is Enough: Parallel Exploration with a Single Policy is Near-Optimal for Reward-Free Reinforcement Learning
Although parallelism has been extensively used in reinforcement learning (RL), the quantitative effects of parallel exploration are not well understood theoretically.
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Predictions
More and more investors and machine learning models rely on social media (e. g., Twitter and Reddit) to gather real-time information and sentiment to predict stock price movements.
Fair Wrapping for Black-box Predictions
We introduce a new family of techniques to post-process ("wrap") a black-box classifier in order to reduce its bias.
The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence
We hypothesize that the diversity coefficient of the few-shot learning benchmark is predictive of whether meta-learning solutions will succeed or not.
Does MAML Only Work via Feature Re-use? A Data Centric Perspective
Recent work has suggested that a good embedding is all we need to solve many few-shot learning benchmarks.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision
EMIXER is an conditional generative adversarial model by 1) generating an image based on a label, 2) encoding the image to a hidden embedding, 3) producing the corresponding text via a hierarchical decoder from the image embedding, and 4) a joint discriminator for assessing both the image and the corresponding text.
Fairness with Overlapping Groups
In algorithmically fair prediction problems, a standard goal is to ensure the equality of fairness metrics across multiple overlapping groups simultaneously.
Advances and Open Problems in Federated Learning
FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
FMRI data augmentation via synthesis
Thus, our results suggest that data augmentation via synthesis is a promising approach to address the limited availability of fMRI data, and to improve the quality of predictive fMRI models.
Max-Sliced Wasserstein Distance and its use for GANs
Generative adversarial nets (GANs) and variational auto-encoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, image-to-image translation and feature learning.
Zeno++: Robust Fully Asynchronous SGD
We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers.
SLSGD: Secure and Efficient Distributed On-device Machine Learning
We consider distributed on-device learning with limited communication and security requirements.
Asynchronous Federated Optimization
Federated learning enables training on a massive number of edge devices.
Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning.
On the Consistency of Top-k Surrogate Losses
Our analysis continues by showing previously proposed hinge-like top-$k$ surrogate losses are not top-$k$ calibrated and suggests no convex hinge loss is top-$k$ calibrated.
A Contextual-bandit-based Approach for Informed Decision-making in Clinical Trials
However, those approaches have failed to account for the variability among participants that is becoming increasingly evident as a result of recent clinical-trial-based studies.
