Search Results for author: Skyler Seto

Found 10 papers, 2 papers with code

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

no code implementations29 Jan 2024 Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly

Large language models are trained on massive scrapes of the web, which are often unstructured, noisy, and poorly phrased.

Language Modelling

REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation

no code implementations7 Sep 2023 Skyler Seto, Barry-John Theobald, Federico Danieli, Navdeep Jaitly, Dan Busbridge

In online F-TTA, a pre-trained model is adapted using a stream of test samples by minimizing a self-supervised objective, such as entropy minimization.

Test-time Adaptation

Spatial LibriSpeech: An Augmented Dataset for Spatial Audio Learning

1 code implementation18 Aug 2023 Miguel Sarabia, Elena Menyaylenko, Alessandro Toso, Skyler Seto, Zakaria Aldeneh, Shadi Pirhosseinloo, Luca Zappella, Barry-John Theobald, Nicholas Apostoloff, Jonathan Sheaffer

We present Spatial LibriSpeech, a spatial audio dataset with over 650 hours of 19-channel audio, first-order ambisonics, and optional distractor noise.

8k Position

Understanding the Robustness of Multi-Exit Models under Common Corruptions

no code implementations3 Dec 2022 Akshay Mehra, Skyler Seto, Navdeep Jaitly, Barry-John Theobald

Furthermore, the lack of calibration increases the inconsistency in the predictions of the model across exits, leading to both inefficient inference and more misclassifications compared with evaluation on in-distribution data.

On the role of Lip Articulation in Visual Speech Perception

no code implementations18 Mar 2022 Zakaria Aldeneh, Masha Fedzechkina, Skyler Seto, Katherine Metcalf, Miguel Sarabia, Nicholas Apostoloff, Barry-John Theobald

Previous research has shown that traditional metrics used to optimize and assess models for generating lip motion from speech are not a good indicator of subjective opinion of animation quality.

FORML: Learning to Reweight Data for Fairness

no code implementations3 Feb 2022 Bobby Yan, Skyler Seto, Nicholas Apostoloff

Machine learning models are trained to minimize the mean loss for a single metric, and thus typically do not consider fairness and robustness.

Fairness Image Classification +2

HALO: Learning to Prune Neural Networks with Shrinkage

1 code implementation24 Aug 2020 Skyler Seto, Martin T. Wells, Wenyu Zhang

Deep neural networks achieve state-of-the-art performance in a variety of tasks by extracting a rich set of features from unstructured data, however this performance is closely tied to model size.

Network Pruning

CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context

no code implementations26 Mar 2020 Wenyu Zhang, Skyler Seto, Devesh K. Jha

The purpose of these agents is to quickly adapt and/or generalize their notion of physics of interaction in the real world based on certain features about the interacting objects that provide different contexts to the predictive models.

Meta-Learning regression +1

A Double Parametric Bootstrap Test for Topic Models

no code implementations19 Nov 2017 Skyler Seto, Sarah Tan, Giles Hooker, Martin T. Wells

Non-negative matrix factorization (NMF) is a technique for finding latent representations of data.

Topic Models

Cannot find the paper you are looking for? You can Submit a new open access paper.