Search Results for author:
Found 13 papers, 8 papers with code
Predicting masked tokens in stochastic locations improves masked image modeling
Specifically, we condition the model on stochastic masked token positions to guide the model toward learning features that are more robust to location uncertainties.
DINOv2: Learning Robust Visual Features without Supervision
The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision.
Ranked #1 on
Image Retrieval
on AmsterTime
(using extra training data)
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations.
The Hidden Uniform Cluster Prior in Self-Supervised Learning
A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e. g., SimCLR, VICReg, SwAV, MSN).
Masked Siamese Networks for Label-Efficient Learning
We propose Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations.
Self-Supervised Image Classification
Self-Supervised Learning
+1
Memory Augmented Optimizers for Deep Learning
Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradients for efficient parameter updates.
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assignments with support samples (PAWS).
A Closer Look at Codistillation for Distributed Training
Surprisingly, we find that even at moderate batch sizes, models trained with codistillation can perform as well as models trained with synchronous data-parallel methods, despite using a much weaker synchronization mechanism.
Advances in Asynchronous Parallel and Distributed Optimization
Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decade.
Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations
We investigate a strategy for improving the efficiency of contrastive learning of visual representations by leveraging a small amount of supervised information during pre-training.
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
We study Nesterov's accelerated gradient method with constant step-size and momentum parameters in the stochastic approximation setting (unbiased gradients with bounded variance) and the finite-sum setting (where randomness is due to sampling mini-batches).
Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning
We show that we can run several loosely coupled GALA agents in parallel on a single GPU and achieve significantly higher hardware utilization and frame-rates than vanilla A2C at comparable power draws.
Stochastic Gradient Push for Distributed Deep Learning
Distributed data-parallel algorithms aim to accelerate the training of deep neural networks by parallelizing the computation of large mini-batch gradient updates across multiple nodes.
