Search Results for author:
Found 27 papers, 8 papers with code
MusicRL: Aligning Music Generation to Human Preferences
MusicRL is a pretrained autoregressive MusicLM (Agostinelli et al., 2023) model of discrete audio tokens finetuned with reinforcement learning to maximise sequence-level rewards.
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter
Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network.
The Symmetric Generalized Eigenvalue Problem as a Nash Equilibrium
In this work, we develop a game-theoretic formulation of the top-$k$ SGEP whose Nash equilibrium is the set of generalized eigenvectors.
Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?
Most notably, ReLICv2 is the first unsupervised representation learning method to consistently outperform the supervised baseline in a like-for-like comparison over a range of ResNet architectures.
Ranked #14 on
Semantic Segmentation
on PASCAL VOC 2012 val
Representation Learning
Self-Supervised Image Classification
+3
EigenGame Unloaded: When playing games is better than optimizing
We build on the recently proposed EigenGame that views eigendecomposition as a competitive game.
Less can be more in contrastive learning
Usually the other datapoints in the batch serve as the negatives for the given datapoint.
Representation Learning via Invariant Causal Mechanisms
Self-supervised learning has emerged as a strategy to reduce the reliance on costly supervised signal by pretraining representations only using unlabeled data.
Ranked #77 on
Self-Supervised Image Classification
on ImageNet
EigenGame: PCA as a Nash Equilibrium
We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function.
Social diversity and social preferences in mixed-motive reinforcement learning
Recent research on reinforcement learning in pure-conflict and pure-common interest games has emphasized the importance of population heterogeneity.
Spectrogram Feature Losses for Music Source Separation
In this paper we study deep learning-based music source separation, and explore using an alternative loss to the standard spectrogram pixel-level L2 loss for model training.
Neural Importance Sampling
We propose to use deep neural networks for generating samples in Monte Carlo integration.
A Fully Progressive Approach to Single-Image Super-Resolution
Recent deep learning approaches to single image super-resolution have achieved impressive results in terms of traditional error measures and perceptual quality.
Ranked #14 on
Image Super-Resolution
on BSD100 - 4x upscaling
PhaseNet for Video Frame Interpolation
We show that this is superior to the hand-crafted heuristics previously used in phase-based methods and also compares favorably to recent deep learning based approaches for video frame interpolation on challenging datasets.
Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks
To render a new scene, we sample visible points of the cloud and, for each, extract a hierarchical 3D descriptor of the cloud geometry with respect to the shading location and the light source.
Kernel-predicting convolutional networks for denoising monte carlo renderings.
In a second approach, we introduce a novel, kernel-prediction network which uses the CNN to estimate the local weighting kernels used to compute each denoised pixel from its neighbors.
Preserving Differential Privacy Between Features in Distributed Estimation
Privacy is crucial in many applications of machine learning.
The Shattered Gradients Problem: If resnets are the answer, then what is the question?
A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients.
Scalable Adaptive Stochastic Optimization Using Random Projections
We show that the regret of Ada-LR is close to the regret of full-matrix AdaGrad which can have an up-to exponentially smaller dependence on the dimension than the diagonal variant.
Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks
Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex; standard guarantees therefore do not apply.
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
The dataset, named DAVIS (Densely Annotated VIdeo Segmentation), consists of fifty high quality, Full HD video sequences, spanning multiple occurrences of common video object segmentation challenges such as occlusions, motion-blur and appearance changes.
Learning Representations for Outlier Detection on a Budget
We demonstrate the good performance of BORE compared to a variety of competing methods in the non-budgeted and the budgeted outlier detection problem on 12 real-world datasets.
Variance Reduced Stochastic Gradient Descent with Neighbors
As a side-product we provide a unified convergence analysis for a family of variance reduction algorithms, which we call memorization algorithms.
DUAL-LOCO: Distributing Statistical Estimation Using Random Projections
We present DUAL-LOCO, a communication-efficient algorithm for distributed statistical estimation.
A Variance Reduced Stochastic Newton Method
Quasi-Newton methods are widely used in practise for convex loss minimization problems.
LOCO: Distributing Ridge Regression with Random Projections
We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster.
Fast and Robust Least Squares Estimation in Corrupted Linear Models
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings.
Correlated random features for fast semi-supervised learning
Random views are justified by recent theoretical and empirical work showing that regression with random features closely approximates kernel regression, implying that random views can be expected to contain accurate estimators.
