Search Results for author:
Found 37 papers, 21 papers with code
The Common Stability Mechanism behind most Self-Supervised Learning Approaches
Last couple of years have witnessed a tremendous progress in self-supervised learning (SSL), the success of which can be attributed to the introduction of useful inductive biases in the learning process to learn meaningful visual representations while avoiding collapse.
PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs
Vision-Language Models (VLMs), such as Flamingo and GPT-4V, have shown immense potential by integrating large language models with vision systems.
Object-Centric Diffusion for Efficient Video Editing
Diffusion-based video editing have reached impressive quality and can transform either the global style, local structure, and attributes of given video inputs, following textual edit prompts.
The LLM Surgeon
Experimentally, our method can prune rows and columns from a range of OPT models and Llamav2-7B by 20%-30%, with a negligible loss in performance, and achieve state-of-the-art results in unstructured and semi-structured pruning of large language models.
Protect Your Score: Contact Tracing With Differential Privacy Guarantees
The pandemic in 2020 and 2021 had enormous economic and societal consequences, and studies show that contact tracing algorithms can be key in the early containment of the virus.
Guided Diffusion from Self-Supervised Diffusion Features
However, recent studies have revealed that the feature representation derived from diffusion model itself is discriminative for numerous downstream tasks as well, which prompts us to propose a framework to extract guidance from, and specifically for, diffusion models.
VaLID: Variable-Length Input Diffusion for Novel View Synthesis
This is because the source-view images and corresponding poses are processed separately and injected into the model at different stages.
VeRA: Vector-based Random Matrix Adaptation
Low-rank adapation (LoRA) is a popular method that reduces the number of trainable parameters when finetuning large language models, but still faces acute storage challenges when scaling to even larger models or deploying numerous per-user or per-task adapted models.
Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video
But are we making the best use of data?
Self-Supervised Open-Ended Classification with Small Visual Language Models
We present Self-Context Adaptation (SeCAt), a self-supervised approach that unlocks few-shot abilities for open-ended classification with small visual language models.
Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations
Our paper aims to address this gap by proposing a novel approach that incorporates temporal consistency in dense self-supervised learning.
Efficient Neural PDE-Solvers using Quantization Aware Training
In the past years, the application of neural networks as an alternative to classical numerical methods to solve Partial Differential Equations has emerged as a potential paradigm shift in this century-old mathematical field.
Learning to Count without Annotations
While recent supervised methods for reference-based object counting continue to improve the performance on benchmark datasets, they have to rely on small datasets due to the cost associated with manually annotating dozens of objects in images.
BISCUIT: Causal Representation Learning from Binary Interactions
Identifying the causal variables of an environment and how to intervene on them is of core value in applications such as robotics and embodied AI.
Self-Ordering Point Clouds
In this paper we address the task of finding representative subsets of points in a 3D point cloud by means of a point-wise ordering.
Towards Label-Efficient Incremental Learning: A Survey
The current dominant paradigm when building a machine learning model is to iterate over a dataset over and over until convergence.
Skip-Attention: Improving Vision Transformers by Paying Less Attention
This work aims to improve the efficiency of vision transformers (ViT).
VTC: Improving Video-Text Retrieval with User Comments
In this paper, we a) introduce a new dataset of videos, titles and comments; b) present an attention-based mechanism that allows the model to learn from sometimes irrelevant data such as comments; c) show that by using comments, our method is able to learn better, more contextualised, representations for image, video and audio representations.
Self-Guided Diffusion Models
Diffusion models have demonstrated remarkable progress in image generation quality, especially when guidance is used to control the generative process.
Prompt Generation Networks for Input-based Adaptation of Frozen Vision Transformers
With the introduction of the transformer architecture in computer vision, increasing model scale has been demonstrated as a clear path to achieving performance and robustness gains.
Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing
Self-supervised visual representation learning has recently attracted significant research interest.
Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems
To address this issue, we propose iCITRIS, a causal representation learning method that allows for instantaneous effects in intervened temporal sequences when intervention targets can be observed, e. g., as actions of an agent.
Looking for a Handsome Carpenter! Debiasing GPT-3 Job Advertisements
The growing capability and availability of generative language models has enabled a wide range of new downstream tasks.
Self-Supervised Learning of Object Parts for Semantic Segmentation
However, learning dense representations is challenging, as in the unsupervised context it is not clear how to guide the model to learn representations that correspond to various potential object categories.
Ranked #5 on
Unsupervised Semantic Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Less than Few: Self-Shot Video Instance Segmentation
The goal of this paper is to bypass the need for labelled examples in few-shot video understanding at run time.
CITRIS: Causal Identifiability from Temporal Intervened Sequences
Understanding the latent causal factors of a dynamical system from visual observations is considered a crucial step towards agents reasoning in complex environments.
The Augmented Image Prior: Distilling 1000 Classes by Extrapolating from a Single Image
What can neural networks learn about the visual world when provided with only a single image as input?
PASS: An ImageNet replacement for self-supervised pretraining without humans
On the other hand, state-of-the-art pretraining is nowadays obtained with unsupervised methods, meaning that labelled datasets such as ImageNet may not be necessary, or perhaps not even optimal, for model pretraining.
Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset
In this paper, we collect hateful and non-hateful memes from Pinterest to evaluate out-of-sample performance on models pre-trained on the Facebook dataset.
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
In video transformers, the time dimension is often treated in the same way as the two spatial dimensions.
Ranked #15 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Self-supervised object detection from audio-visual correspondence
We tackle the problem of learning object detectors without supervision.
Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning
First, for space, we show that spatial augmentations such as cropping do work well for videos too, but that previous implementations, due to the high processing and memory cost, could not do this at a scale sufficient for it to work well.
Privacy-preserving Object Detection
Privacy considerations and bias in datasets are quickly becoming high-priority issues that the computer vision community needs to face.
Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models
Using a template-based data collection pipeline, we collect 396K sentence completions made by GPT-2 and find: (i) The machine-predicted jobs are less diverse and more stereotypical for women than for men, especially for intersections; (ii) Intersectional interactions are highly relevant for occupational associations, which we quantify by fitting 262 logistic models; (iii) For most occupations, GPT-2 reflects the skewed gender and ethnicity distribution found in US Labor Bureau data, and even pulls the societally-skewed distribution towards gender parity in cases where its predictions deviate from real labor market observations.
Labelling unlabelled videos from scratch with multi-modal self-supervision
A large part of the current success of deep learning lies in the effectiveness of data -- more precisely: labelled data.
On Compositions of Transformations in Contrastive Self-Supervised Learning
In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning.
A critical analysis of self-supervision, or what we can learn from a single image
We look critically at popular self-supervision techniques for learning deep convolutional neural networks without manual labels.
