Search Results for author:
Found 11 papers, 5 papers with code
Using Foundation Models to Detect Policy Violations with Minimal Supervision
Our contributions are: We identify a hard prompt that adapts chain-of-thought prompting to policy violation tasks.
Large-Scale Differentially Private BERT
In this work, we study the large-scale pretraining of BERT-Large with differentially private SGD (DP-SGD).
Towards Practical Second Order Optimization for Deep Learning
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent.
Scalable Second Order Optimization for Deep Learning
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent.
Memory Efficient Adaptive Optimization
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling.
Memory-Efficient Adaptive Optimization
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling.
Ranked #31 on
Machine Translation
on WMT2014 English-French
The Singular Values of Convolutional Layers
We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation.
Shampoo: Preconditioned Stochastic Tensor Optimization
Preconditioned gradient methods are among the most general and powerful tools in optimization.
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization
We describe a framework for deriving and analyzing online optimization algorithms that incorporate adaptive, data-dependent regularization, also termed preconditioning.
Random Features for Compositional Kernels
We describe and analyze a simple random feature scheme (RFS) from prescribed compositional kernels.
Communicating Semantics: Reference by Description
Messages often refer to entities such as people, places and events.
