1 code implementation • NeurIPS 2023 • Abhinav Nippani, Dongyue Li, Haotian Ju, Haris N. Koutsopoulos, Hongyang R. Zhang
This paper constructs a large-scale, unified dataset of traffic accident records from official reports of various states in the US, totaling 9 million records, accompanied by road networks and traffic volume reports.
3 code implementations • 24 Jun 2023 • Dongyue Li, Haotian Ju, Aneesh Sharma, Hongyang R. Zhang
Lastly, we provide a theoretical analysis to show that under a planted block model of tasks on graphs, our affinity scores can provably separate tasks into groups.
no code implementations • 14 Jun 2023 • Haotian Ju, Dongyue Li, Hongyang R. Zhang
It leads to a 17. 7% (and 12. 8%) reduction in the trace (and largest eigenvalue) of the Hessian matrix of the loss surface.
2 code implementations • 25 Mar 2023 • Dongyue Li, Huy L. Nguyen, Hongyang R. Zhang
This problem is computationally challenging since the number of subsets grows exponentially with the number of source tasks; efficient heuristics for subset selection do not always capture the relationship between task subsets and multitask learning performances.
2 code implementations • 9 Feb 2023 • Haotian Ju, Dongyue Li, Aneesh Sharma, Hongyang R. Zhang
Graph neural networks are widely used tools for graph prediction tasks.
3 code implementations • 6 Jun 2022 • Haotian Ju, Dongyue Li, Hongyang R. Zhang
We study the generalization properties of fine-tuning to understand the problem of overfitting, which has often been observed (e. g., when the target dataset is small or when the training labels are noisy).
1 code implementation • 20 Apr 2022 • Thien Hang Nguyen, Hongyang R. Zhang, Huy Le Nguyen
Given a limited amount of group labels at training time, Just Train Twice (Liu et al., 2021), or JTT in short, is a two-stage method that infers a pseudo group label for every unlabeled example first, then applies group DRO based on the inferred group labels.
1 code implementation • 3 Mar 2022 • Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang, Chelsea Finn, Christopher Ré
As ERM models can be good spurious attribute predictors, CNC works by (1) using a trained ERM model's outputs to identify samples with the same class but dissimilar spurious features, and (2) training a robust model with contrastive learning to learn similar representations for same-class samples.
1 code implementation • NeurIPS 2021 • Dongyue Li, Hongyang R. Zhang
We present a PAC-Bayes generalization bound that depends on the distance traveled in each layer during fine-tuning and the noise stability of the fine-tuned model.
no code implementations • 22 Oct 2020 • Fan Yang, Hongyang R. Zhang, Sen Wu, Christopher Ré, Weijie J. Su
Intuitively, the transfer effect from one task to another task depends on dataset shifts such as sample sizes and covariance matrices.
no code implementations • 9 Jul 2020 • Yuanzhi Li, Tengyu Ma, Hongyang R. Zhang
We consider the dynamic of gradient descent for learning a two-layer neural network.
no code implementations • ICLR 2020 • Sen Wu, Hongyang R. Zhang, Christopher Ré
We investigate multi-task learning approaches that use a shared feature representation for all tasks.
2 code implementations • ICML 2020 • Sen Wu, Hongyang R. Zhang, Gregory Valiant, Christopher Ré
We validate our proposed scheme on image and text datasets.
no code implementations • 31 Oct 2018 • Hongyang R. Zhang, Vatsal Sharan, Moses Charikar, YIngyu Liang
We consider the tensor completion problem of predicting the missing entries of a tensor.