Search Results for author:
Found 34 papers, 21 papers with code
Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
We demonstrate that it's possible to predict which heads are crucial for long-context processing using only local keys.
Compositional Risk Minimization
In this work, we tackle a challenging and extreme form of subpopulation shift, which is termed compositional shift.
DRoP: Distributionally Robust Pruning
In the era of exceptionally data-hungry models, careful selection of the training data is essential to mitigate the extensive costs of deep learning.
On Provable Length and Compositional Generalization
In the first part, we study structured limited capacity variants of different architectures and arrive at the generalization guarantees with limited diversity requirements on the training distribution.
Multi-Domain Causal Representation Learning via Weak Distributional Invariances
Causal representation learning has emerged as the center of action in causal machine learning research.
On the Identifiability of Quantized Factors
We introduce this novel form of identifiability, termed quantized factor identifiability, and provide a comprehensive proof of the recovery of the quantized factors.
A Closer Look at In-Context Learning under Distribution Shifts
In-context learning, a capability that enables a model to learn from input examples on the fly without necessitating weight updates, is a defining characteristic of large language models.
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization
In this paper, we thus propose model ratatouille, a new strategy to recycle the multiple fine-tunings of the same foundation model on diverse auxiliary tasks.
Ranked #19 on
Domain Generalization
on TerraIncognita
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Modern deep learning systems do not generalize well when the test data distribution is slightly different to the training data distribution.
FL Games: A Federated Learning Framework for Distribution Shifts
Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server.
Interventional Causal Representation Learning
Can interventional data facilitate causal representation learning?
Weakly Supervised Representation Learning with Sparse Perturbations
We show that if the perturbations are applied only on mutually exclusive blocks of latents, we identify the latents up to those blocks.
FL Games: A federated learning framework for distribution shifts
Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server.
Why does Throwing Away Data Improve Worst-Group Error?
When facing data with imbalanced classes or groups, practitioners follow an intriguing strategy to achieve best results.
Towards efficient representation identification in supervised learning
In this work, we depart from these assumptions and ask: a) How can we get disentanglement when the auxiliary information does not provide conditional independence over the factors of variation?
WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series
We revise the existing OOD generalization algorithms for time series tasks and evaluate them using our systematic framework.
Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning
These results suggest that by exploiting inductive biases on mechanisms, it is possible to design a range of new identifiable representation learning approaches.
Locally Invariant Explanations: Towards Causal Explanations through Local Invariant Learning
Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level.
Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledge
Our main result strengthens these prior results by showing that under a different expert-driven structural knowledge -- that one variable is a direct causal parent of treatment variable -- remarkably, testing for subsets (not involving the known parent variable) that are valid back-doors is equivalent to an invariance test.
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
To answer these questions, we revisit the fundamental assumptions in linear regression tasks, where invariance-based approaches were shown to provably generalize OOD.
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
Can models with particular structure avoid being biased towards spurious correlation in out-of-distribution (OOD) generalization?
SAND-mask: An Enhanced Gradient Masking Strategy for the Discovery of Invariances in Domain Generalization
A major bottleneck in the real-world applications of machine learning models is their failure in generalizing to unseen domains whose data distribution is not i. i. d to the training domains.
Treatment Effect Estimation using Invariant Risk Minimization
Inferring causal individual treatment effect (ITE) from observational data is a challenging problem whose difficulty is exacerbated by the presence of treatment assignment bias.
Learning to Initialize Gradient Descent Using Gradient Descent
Non-convex optimization problems are challenging to solve; the success and computational expense of a gradient descent algorithm or variant depend heavily on the initialization strategy.
Empirical or Invariant Risk Minimization? A Sample Complexity Perspective
Recently, invariant risk minimization (IRM) was proposed as a promising solution to address out-of-distribution (OOD) generalization.
Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
In Ahuja et al., it was shown that solving for the Nash equilibria of a new class of "ensemble-games" is equivalent to solving IRM.
Adversarial Feature Desensitization
Our method, called Adversarial Feature Desensitization (AFD), aims at learning features that are invariant towards adversarial perturbations of the inputs.
Invariant Risk Minimization Games
The standard risk minimization paradigm of machine learning is brittle when operating in environments whose test distributions are different from the training distribution due to spurious correlations.
Estimating Kullback-Leibler Divergence Using Kernel Machines
Recently, a method called the Mutual Information Neural Estimator (MINE) that uses neural networks has been proposed to estimate mutual information and more generally the Kullback-Leibler (KL) divergence between two distributions.
Risk-Stratify: Confident Stratification Of Patients Based On Risk
A clinician desires to use a risk-stratification method that achieves confident risk-stratification - the risk estimates of the different patients reflect the true risks with a high probability.
Joint Concordance Index
We use the new metric to develop a variable importance ranking approach.
Methodology
Optimal Piecewise Local-Linear Approximations
Piecewise local-linear models provide a natural way to extend local-linear models to explain the global behavior of the model.
DPSCREEN: Dynamic Personalized Screening
However, there has been limited work to address the personalized screening for these different diseases.
