Search Results for author:
Found 21 papers, 5 papers with code
Training Text-to-Text Transformers with Privacy Guarantees
We focus on T5 and show that by using recent advances in JAX and XLA we can train models with DP that do not suffer a large drop in pre-training utility, nor in training speed, and can still be fine-tuned to high accuracies on downstream tasks (e. g.
JetTrain: IDE-Native Machine Learning Experiments
Integrated development environments (IDEs) are prevalent code-writing and debugging tools.
DART: A Principled Approach to Adversarially Robust Unsupervised Domain Adaptation
To address this challenge, we first establish a generalization bound for the adversarial target loss, which consists of (i) terms related to the loss on the data, and (ii) a measure of worst-case domain divergence.
Scaling Laws for Downstream Task Performance of Large Language Models
With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data.
COMET: Learning Cardinality Constrained Mixture of Experts with Trees and Local Search
To deal with this challenge, we propose a novel, permutation-based local search method that can complement first-order methods in training any sparse gate, e. g., Hash routing, Top-k, DSelect-k, and COMET.
Explaining and Adapting Graph Conditional Shift
Graph Neural Networks (GNNs) have shown remarkable performance on graph-structured data.
Harnessing large-language models to generate private synthetic text
An alternative approach, which this paper studies, is to use a sensitive dataset to generate synthetic data that is differentially private with respect to the original data, and then non-privately training a model on the synthetic data.
Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models
We address the challenge of ensuring differential privacy (DP) guarantees in training deep retrieval systems.
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
However, while some adoption of DP has happened in industry, attempts to apply DP to real world complex ML models are still few and far between.
Fast as CHITA: Neural Network Pruning with Combinatorial Optimization
Our approach, CHITA, extends the classical Optimal Brain Surgeon framework and results in significant improvements in speed, memory, and performance over existing optimization-based approaches for network pruning.
Mind the (optimality) Gap: A Gap-Aware Learning Rate Scheduler for Adversarial Nets
We run large-scale experiments to study the effectiveness of the scheduler on two popular applications: GANs for image generation and adversarial nets for domain adaptation.
Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance
In this paper, we show that the statistical problems with covariance estimation drive the poor performance of H-score -- a common baseline for newer metrics -- and propose shrinkage-based estimator.
Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution.
The Tree Ensemble Layer: Differentiability meets Conditional Computation
We aim to combine these advantages by introducing a new layer for neural networks, composed of an ensemble of differentiable decision trees (a. k. a.
Agent Prioritization for Autonomous Navigation
We propose a system to rank agents around an autonomous vehicle (AV) in real time.
A Survey of the Perceived Text Adaptation Needs of Adults with Autism
NLP approaches to automatic text adaptation often rely on user-need guidelines which are generic and do not account for the differences between various types of target groups.
Accelerating Gradient Boosting Machine
This is the first GBM type of algorithm with theoretically-justified accelerated convergence rate.
TF Boosted Trees: A scalable TensorFlow based framework for gradient boosting
TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees.
Compact Multi-Class Boosted Trees
Gradient boosted decision trees are a popular machine learning technique, in part because of their ability to give good accuracy with small models.
