Search Results for author:
Found 83 papers, 30 papers with code
Minimum-Norm Interpolation Under Covariate Shift
We follow our analysis with empirical studies that show these beneficial and malignant covariate shifts for linear interpolators on real image data, and for fully-connected neural networks in settings where the input data dimension is larger than the training sample size.
Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency
We consider gradient descent (GD) with a constant stepsize applied to logistic regression with linearly separable data, where the constant stepsize $\eta$ is so large that the loss initially oscillates.
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance
Using publicly available patient data, we collaborate with ED clinicians to curate MIMIC-ED-Assist, a benchmark that measures the ability of AI systems in suggesting laboratory tests that minimize ED wait times, while correctly predicting critical outcomes such as death.
LoRA+: Efficient Low Rank Adaptation of Large Models
In this paper, we show that Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021) leads to suboptimal finetuning of models with large width (embedding dimension).
Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
We propose explanation-consistency finetuning (EC-finetuning), a method that adapts LLMs to generate more consistent natural-language explanations on related examples.
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers.
Using Experience Classification for Training Non-Markovian Tasks
Unlike the standard Reinforcement Learning (RL) model, many real-world tasks are non-Markovian, whose rewards are predicated on state history rather than solely on the current state.
Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms
Domain adaptation (DA) is a statistical learning problem that arises when the distribution of the source data used to train a model differs from that of the target data used to evaluate the model.
TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Models
This integration yields the following key enhancements: 1) empowering ChatGPT with the capacity to view, analyze, process traffic data, and provide insightful decision support for urban transportation system management; 2) facilitating the intelligent deconstruction of broad and complex tasks and sequential utilization of traffic foundation models for their gradual completion; 3) aiding human decision-making in traffic control through natural language dialogues; and 4) enabling interactive feedback and solicitation of revised outcomes.
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
On the other hand, for any batch size strictly smaller than the number of samples, SGD finds a global minimum which is sparse and nearly orthogonal to its initialization, showing that the randomness of stochastic gradients induces a qualitatively different type of "feature selection" in this setting.
Improving Prototypical Part Networks with Reward Reweighing, Reselection, and Retraining
Hoping to remedy this, we take inspiration from the recent developments in Reinforcement Learning with Human Feedback (RLHF) to fine-tune these prototypes.
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
While existing token-level early exit methods show promising results for online inference, they cannot be readily applied for batch inferencing and Key-Value caching.
MDI+: A Flexible Random Forest-Based Feature Importance Framework
We show that the MDI for a feature $X_k$ in each tree in an RF is equivalent to the unnormalized $R^2$ value in a linear regression of the response on the collection of decision stumps that split on $X_k$.
Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
These findings showcase the utility of SUFO in enhancing trust and safety when using transformers in medicine, and we believe SUFO can aid practitioners in evaluating fine-tuned language models for other applications in medicine and in more critical domains.
Explaining black box text modules in natural language with language models
Here, we ask whether we can automatically obtain natural language explanations for black box text modules.
Enhancing Peak Network Traffic Prediction via Time-Series Decomposition
For network administration and maintenance, it is critical to anticipate when networks will receive peak volumes of traffic so that adequate resources can be allocated to service requests made to servers.
A Mixing Time Lower Bound for a Simplified Version of BART
In this paper, we provide the first lower bound on the mixing time for a simplified version of BART in which we reduce the sum to a single tree and use a subset of the possible moves for the MCMC proposal distribution.
Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data
A central goal in social science is to evaluate the causal effect of a policy.
Learning Using Privileged Information for Zero-Shot Action Recognition
This approach has been challenged by the semantic gap between the visual space and semantic space.
Group Probability-Weighted Tree Sums for Interpretable Modeling of Heterogeneous Data
Machine learning in high-stakes domains, such as healthcare, faces two critical challenges: (1) generalizing to diverse data distributions given limited training data while (2) maintaining interpretability.
Hierarchical Shrinkage: improving the accuracy and interpretability of tree-based methods
Tree-based models such as decision trees and random forests (RF) are a cornerstone of modern machine-learning practice.
Fast Interpretable Greedy-Tree Sums
In such settings, practitioners often use highly interpretable decision tree models, but these suffer from inductive bias against additive structure.
C2AM Loss: Chasing a Better Decision Boundary for Long-Tail Object Detection
The decision boundary between any two categories is the angular bisector of their weight vectors.
The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods
To understand how deep learning works, it is crucial to understand the training dynamics of neural networks.
A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds
We prove a sharp squared error generalization lower bound for a large class of decision tree algorithms fitted to sparse additive models with $C^1$ component functions.
Towards Robust Waveform-Based Acoustic Models
We study the problem of learning robust acoustic models in adverse environments, characterized by a significant mismatch between training and test conditions.
Interpreting and improving deep-learning models with reality checks
Recent deep-learning models have achieved impressive predictive performance by learning complex functions of many variables, often at the cost of interpretability.
Adaptive wavelet distillation from neural networks through interpretations
Moreover, interpretable models are concise and often yield computational efficiency.
Provable Boolean Interaction Recovery from Tree Ensemble obtained via Random Forests
Iterative Random Forests (iRF) use a tree ensemble from iteratively modified RF to obtain predictive and stable non-linear or Boolean interactions of features.
Statistics Theory Statistics Theory
High-Performance Discriminative Tracking With Transformers
End-to-end discriminative trackers improve the state of the art significantly, yet the improvement in robustness and efficiency is restricted by the conventional discriminative model, i. e., least-squares based regression.
Enriched Annotations for Tumor Attribute Classification from Pathology Reports with Limited Labeled Data
Precision medicine has the potential to revolutionize healthcare, but much of the data for patients is locked away in unstructured free-text, limiting research and delivery of effective personalized treatments.
Stable discovery of interpretable subgroups via calibration in causal studies
Building on Yu and Kumbier's PCS framework and for randomized experiments, we introduce a novel methodology for Stable Discovery of Interpretable Subgroups via Calibration (StaDISC), with large heterogeneous treatment effects.
Two-stage growth mode for lift-off mechanism in oblique shock-wave/jet interaction
Based on the objective coordinate system in frame of oblique shock structure, it is found that the nature of three-dimensional lift-off structure of a shockinduced streamwise vortex is inherently and precisely controlled by a two-stage growth mode of structure kinetics of a shock bubble interaction (SBI for short).
Fluid Dynamics
Revisiting minimum description length complexity in overparameterized models
We provide an extensive theoretical characterization of MDL-COMP for linear models and kernel methods and show that it is not just a function of parameter count, but rather a function of the singular values of the design or the kernel matrix and the signal-to-noise ratio.
A Survey on Dynamic Network Embedding
Real-world networks are composed of diverse interacting and evolving entities, while most of existing researches simply characterize them as particular static networks, without consideration of the evolution trend in dynamic networks.
Social and Information Networks Physics and Society
Instability, Computational Efficiency and Statistical Accuracy
Many statistical estimators are defined as the fixed point of a data-dependent operator, with estimators based on minimizing a cost function being an important special case.
Curating a COVID-19 data repository and forecasting county-level death counts in the United States
We use this data to develop predictions and corresponding prediction intervals for the short-term trajectory of COVID-19 cumulative death counts at the county-level in the United States up to two weeks ahead.
Transformation Importance with Applications to Cosmology
Machine learning lies at the heart of new possibilities for scientific discovery, knowledge generation, and artificial intelligence.
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge
For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective.
A Debiased MDI Feature Importance Measure for Random Forests
Based on the original definition of MDI by Breiman et al. for a single tree, we derive a tight non-asymptotic bound on the expected bias of MDI importance of noisy features, showing that deep trees have higher (expected) feature selection bias than shallow ones.
Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients
This bound gives a precise quantification of the faster convergence of Metropolized HMC relative to simpler MCMC algorithms such as the Metropolized random walk, or Metropolized Langevin algorithm.
Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees
Tree ensembles, such as random forests and AdaBoost, are ubiquitous machine learning models known for achieving strong predictive performance across a wide variety of domains.
CharBot: A Simple and Effective Method for Evading DGA Classifiers
In this work, we present a novel DGA called CharBot which is capable of producing large numbers of unregistered domain names that are not detected by state-of-the-art classifiers for real-time detection of DGAs, including the recently published methods FANCI (a random forest based on human-engineered features) and LSTM. MI (a deep learning approach).
Unique Sharp Local Minimum in $\ell_1$-minimization Complete Dictionary Learning
First, we obtain a necessary and sufficient norm condition for the reference dictionary $D^*$ to be a sharp local minimum of the expected $\ell_1$ objective function.
Sharp Analysis of Expectation-Maximization for Weakly Identifiable Models
We study a class of weakly identifiable location-scale mixture models for which the maximum likelihood estimates based on $n$ i. i. d.
Veridical Data Science
It augments predictability and computability with an overarching stability principle for the data science life cycle.
Interpretable machine learning: definitions, methods, and applications
Official code for using / reproducing ACD (ICLR 2019) from the paper "Hierarchical interpretations for neural network predictions" https://arxiv. org/abs/1806. 05337
An objective-adaptive refinement criterion based on modified ridge extraction method for finite-time Lyapunov exponent (FTLE) calculation
High-accuracy and high-efficiency finite-time Lyapunov exponent (FTLE) calculation method has long been a research hot point, and adaptive refinement method is a kind of method in this field.
Fluid Dynamics
Signed iterative random forests to identify enhancer-associated transcription factor binding
Standard ChIP-seq peak calling pipelines seek to differentiate biochemically reproducible signals of individual genomic elements from background noise.
Singularity, Misspecification, and the Convergence Rate of EM
A line of recent work has analyzed the behavior of the Expectation-Maximization (EM) algorithm in the well-specified setting, in which the population likelihood is locally strongly concave around its maximizing argument.
Fast Kernelized Correlation Filters without Boundary Effect
To achieve the fast training and detection, a set of cyclic bases is introduced to construct the filter.
High-speed Tracking with Multi-kernel Correlation Filters
In this paper, we will introduce the MKL into KCF in a different way than MKCF.
Hierarchical interpretations for neural network predictions
Deep neural networks (DNNs) have achieved impressive predictive performance due to their ability to learn complex, non-linear relationships between variables.
Stability and Convergence Trade-off of Iterative Optimization Algorithms
Applying existing stability upper bounds for the gradient methods in our trade-off framework, we obtain lower bounds matching the well-established convergence upper bounds up to constants for these algorithms and conjecture similar lower bounds for NAG and HB.
Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs
On the task of sentiment analysis with the Yelp and SST data sets, we show that CD is able to reliably identify words and phrases of contrasting sentiment, and how they are combined to yield the LSTM's final prediction.
Log-concave sampling: Metropolis-Hastings algorithms are fast
Relative to known guarantees for the unadjusted Langevin algorithm (ULA), our bounds show that the use of an accept-reject step in MALA leads to an exponentially improved dependence on the error-tolerance.
Character Level Based Detection of DGA Domain Names
Recently several different deep learning architectures have been proposed that take a string of characters as the raw input signal and automatically derive features for text classification.
Interpreting Convolutional Neural Networks Through Compression
In our compression, the filter importance index is defined as the classification accuracy reduction (CAR) of the network after pruning that filter.
Fast MCMC sampling algorithms on polytopes
We propose and analyze two new MCMC sampling algorithms, the Vaidya walk and the John walk, for generating samples from the uniform distribution over a polytope.
Iterative Random Forests to detect predictive and stable high-order interactions
Genomics has revolutionized biology, enabling the interrogation of whole transcriptomes, genome-wide binding sites for proteins, and many other molecular processes.
Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning
There is growing interest in estimating and analyzing heterogeneous treatment effects in experimental and observational studies.
Statistics Theory Methodology Statistics Theory
Structural Compression of Convolutional Neural Networks
Deep convolutional neural networks (CNNs) have been successful in many tasks in machine vision, however, millions of weights in the form of thousands of convolutional filters in CNNs makes them difficult for human intepretation or understanding in science.
Formulas for Counting the Sizes of Markov Equivalence Classes of Directed Acyclic Graphs
A Markov equivalence class can be represented by an essential graph and its undirected subgraphs determine the size of the class.
Optimal Subsampling Approaches for Large Sample Linear Regression
For unweighted estimation algorithm, we show that its resulting subsample estimator is not consistent to the full sample OLS estimator.
Local identifiability of $l_1$-minimization dictionary learning: a sufficient and almost necessary condition
Moreover, our local identifiability results also translate to the finite sample case with high probability provided that the number of signals $N$ scales as $O(K\log K)$.
Error Rate Bounds and Iterative Weighted Majority Voting for Crowdsourcing
We propose an iterative weighted majority voting (IWMV) method that optimizes the error rate bound and approximates the oracle MAP rule.
Statistical guarantees for the EM algorithm: From population to sample-based analysis
Leveraging this characterization, we then provide non-asymptotic guarantees on the EM and gradient EM algorithms when applied to a finite set of samples.
Concise comparative summaries (CCS) of large text corpora with a human experiment
In this paper we propose a general framework for topic-specific summarization of large text corpora and illustrate how it can be used for the analysis of news databases.
The geometry of kernelized spectral clustering
As a corollary we control the fraction of samples mislabeled by spectral clustering under finite mixtures with nonparametric components.
Impact of regularization on Spectral Clustering
Under the stochastic block model (SBM), and its extensions, previous results on spectral clustering relied on the minimum degree of the graph being sufficiently large for its good performance.
Error Rate Bounds in Crowdsourcing Models
We show that the oracle Maximum A Posterior (MAP) rule approximately optimizes our upper bound on the mean error rate for any hyperplane binary labeling rule, and propose a simple data-driven weighted majority voting (WMV) rule (called one-step WMV) that attempts to approximate the oracle MAP and has a provable theoretical guarantee on the error rate.
A Statistical Perspective on Algorithmic Leveraging
A detailed empirical evaluation of existing leverage-based methods as well as these two new methods is carried out on both synthetic and real data sets.
Early stopping and non-parametric regression: An optimal data-dependent stopping rule
The strategy of early stopping is a regularization technique based on choosing a stopping time for an iterative algorithm.
Estimation Stability with Cross Validation (ESCV)
For the two real data sets from neuroscience and cell biology, the models found by ESCV are less than half of the model sizes by CV.
Supplement to "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs"
This supplementary material includes three parts: some preliminary results, four examples, an experiment, three new algorithms, and all proofs of the results in the paper "Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs".
Reversible MCMC on Markov equivalence classes of sparse directed acyclic graphs
In this paper, we design reversible irreducible Markov chains on the space of Markov equivalent classes by proposing a perfect set of operators that determine the transitions of the Markov chain.
Supervised Feature Selection in Graphs with Path Coding Penalties and Network Flows
We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network.
Co-clustering for directed graphs: the Stochastic co-Blockmodel and spectral algorithm Di-Sim
In each example, a small subset of nodes have persistent asymmetries; these nodes send edges with one cluster, but receive edges with another cluster.
Predicting Execution Time of Computer Programs Using Sparse Polynomial Regression
Our two SPORE algorithms are able to build relationships between responses (e. g., the execution time of a computer program) and features, and select a few from hundreds of the retrieved features to construct an explicitly sparse and non-linear model to predict the response variable.
Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness
components from some distribution $\mP$, we determine tight lower bounds on the minimax rate for estimating the regression function with respect to squared $\LTP$ error.
A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
The estimation of high-dimensional parametric models requires imposing some structure on the models, for instance that they be sparse, or that matrix structured parameters have low rank.
Nonparametric sparse hierarchical models describe V1 fMRI responses to natural images
We propose a novel hierarchical, nonlinear model that predicts brain activity in area V1 evoked by natural images.
