Search Results for author:
Found 79 papers, 46 papers with code
A Large-scale Dataset for Audio-Language Representation Learning
To tackle these challenges, we present an innovative and automatic audio caption generation pipeline based on a series of public tools or APIs, and construct a large-scale, high-quality, audio-language dataset, named as Auto-ACD, comprising over 1. 9M audio-text pairs.
UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training
Magnetic resonance imaging~(MRI) have played a crucial role in brain disease diagnosis, with which a range of computer-aided artificial intelligence methods have been proposed.
The Making and Breaking of Camouflage
We also incorporate the proposed camouflage score into a generative model as an auxiliary loss and show that effective camouflage images or videos can be synthesised in a scalable manner.
Diagnosing Human-object Interaction Detectors
Although we have witnessed significant progress in human-object interaction (HOI) detection with increasingly high mAP (mean Average Precision), a single mAP score is too concise to obtain an informative summary of a model's performance and to understand why one approach is better than another.
Joint-Relation Transformer for Multi-Person Motion Prediction
Multi-person motion prediction is a challenging problem due to the dependency of motion on both individual past movements and interactions with other people.
Towards Generalist Foundation Model for Radiology
In this study, we aim to initiate the development of Radiology Foundation Model, termed as RadFM. We consider the construction of foundational models from the perspectives of dataset construction, model design, and thorough evaluation.
Boost Video Frame Interpolation via Motion Adaptation
Video frame interpolation (VFI) is a challenging task that aims to generate intermediate frames between two consecutive frames in a video.
arXiVeri: Automatic table verification with GPT
In this paper, we propose to meet this challenge through the novel task of automatic table verification (AutoTV), in which the objective is to verify the accuracy of numerical data in tables by cross-referencing cited sources.
Zero-shot Composed Text-Image Retrieval
In this paper, we consider the problem of composed image retrieval (CIR), it aims to train a model that can fuse multi-modal information, e. g., text and images, to accurately retrieve images that match the query, extending the user's expression ability.
Multi-Modal Classifiers for Open-Vocabulary Object Detection
The goal of this paper is open-vocabulary object detection (OVOD) $\unicode{x2013}$ building a model that can detect objects beyond the set of categories seen at training, thus enabling the user to specify categories of interest at inference without the need for model retraining.
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models
Generative models have recently exhibited exceptional capabilities in various scenarios, for example, image generation based on text description.
Annotation-free Audio-Visual Segmentation
The objective of Audio-Visual Segmentation (AVS) is to localise the sounding objects within visual scenes by accurately predicting pixel-wise segmentation masks.
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
In this paper, we focus on the problem of Medical Visual Question Answering (MedVQA), which is crucial in efficiently interpreting medical images with vital clinic-relevant information.
Ranked #1 on
Medical Visual Question Answering
on VQA-RAD
PMC-LLaMA: Towards Building Open-source Language Models for Medicine
Our contributions are threefold: (i) we systematically investigate the process of adapting a general-purpose foundation language model towards medical domain, this involves data-centric knowledge injection through the integration of 4. 8M biomedical academic papers and 30K medical textbooks, as well as comprehensive fine-tuning for alignment with domain-specific instructions; (ii) we contribute a large-scale, comprehensive dataset for instruction tuning.
Zero-shot Unsupervised Transfer Instance Segmentation
Segmentation is a core computer vision competency, with applications spanning a broad range of scientifically and economically valuable domains.
Towards Open-Vocabulary Video Instance Segmentation
Video Instance Segmentation (VIS) aims at segmenting and categorizing objects in videos from a closed set of training categories, lacking the generalization ability to handle novel categories in real-world videos.
AutoAD: Movie Description in Context
The objective of this paper is an automatic Audio Description (AD) model that ingests movies and outputs AD in text form.
Collaboration Helps Camera Overtake LiDAR in 3D Detection
Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems.
Multi-modal Prompting for Low-Shot Temporal Action Localization
In this paper, we consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario, with the goal of detecting and classifying the action instances from arbitrary categories within some untrimmed videos, even not seen at training time.
PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents
Foundation models trained on large-scale dataset gain a recent surge in CV and NLP.
Ranked #3 on
Medical Visual Question Answering
on PMC-VQA
Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images
While multi-modal foundation models pre-trained on large-scale data have been successful in natural language understanding and vision recognition, their use in medical domains is still limited due to the fine-grained nature of medical tasks and the high demand for domain knowledge.
K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging
In this paper, we consider the problem of disease diagnosis.
OvarNet: Towards Open-vocabulary Object Attribute Recognition
In this paper, we consider the problem of simultaneously detecting objects and inferring their visual attributes in an image, even for those with no manual annotations provided at the training stage, resembling an open-vocabulary scenario.
Ranked #1 on
Open Vocabulary Attribute Detection
on OVAD benchmark
(using extra training data)
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision
The former aims to infer all masked entities in the caption given the group tokens, that enables the model to learn fine-grained alignment between visual groups and text entities.
Open-vocabulary Object Segmentation with Diffusion Models
The goal of this paper is to extract the visual-language correspondence from a pre-trained text-to-image diffusion model, in the form of segmentation map, i. e., simultaneously generating images and segmentation masks for the corresponding visual entities described in the text prompt.
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology
In this paper, we consider enhancing medical visual-language pre-training (VLP) with domain-specific knowledge, by exploiting the paired image-text reports from the radiological daily practice.
Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models
When trained at a sufficient scale, self-supervised learning has exhibited a notable ability to solve a wide range of visual or language understanding tasks.
A Tri-Layer Plugin to Improve Occluded Detection
To this end we make the following four contributions: (1) We propose a simple 'plugin' module for the detection head of two-stage object detectors to improve the recall of partially occluded objects.
Ranked #1 on
Instance Segmentation
on Separated COCO
Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors
This contrasts with the case of synchronising videos of talking heads, where audio-visual correspondence is dense in both time and space.
Turbo Training with Token Dropout
The objective of this paper is an efficient training method for video tasks.
A Simple Plugin for Transforming Images to Arbitrary Scales
Existing models on super-resolution often specialized for one scale, fundamentally limiting their use in practical scenarios.
Motion-inductive Self-supervised Object Discovery in Videos
In this paper, we consider the task of unsupervised object discovery in videos.
Ranked #3 on
Unsupervised Object Segmentation
on DAVIS 2016
NamedMask: Distilling Segmenters from Complementary Foundation Models
Our method, termed NamedMask, begins by using CLIP to construct category-specific archives of images.
Adaptive 3D Localization of 2D Freehand Ultrasound Brain Images
Two-dimensional (2D) freehand ultrasound is the mainstay in prenatal care and fetal growth monitoring.
CounTR: Transformer-based Generalised Visual Counting
In this paper, we consider the problem of generalised visual object counting, with the goal of developing a computational model for counting the number of objects from arbitrary semantic categories, using arbitrary number of "exemplars", i. e. zero-shot or few-shot counting.
Ranked #2 on
Object Counting
on FSC147
Transforming the Interactive Segmentation for Medical Imaging
The goal of this paper is to interactively refine the automatic segmentation on challenging structures that fall behind human performance, either due to the scarcity of available annotations or the difficulty nature of the problem itself, for example, on segmenting cancer or small organs.
Aerial Monocular 3D Object Detection
To fill the gap, this work proposes a dual-view detection system named DVDET to achieve aerial monocular object detection in both the 2D image space and the 3D physical space.
Segmenting Moving Objects via an Object-Centric Layered Representation
The objective of this paper is a model that is able to discover, track and segment multiple moving objects in a video.
Ranked #3 on
Unsupervised Object Segmentation
on FBMS-59
Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisation
We present a simple yet effective self-supervised framework for audio-visual representation learning, to localize the sound source in videos.
ReCo: Retrieve and Co-segment for Zero-shot Transfer
Semantic segmentation has a broad range of applications, but its real-world impact has been significantly limited by the prohibitive annotation costs necessary to enable deployment.
Retrieval
Unsupervised Semantic Segmentation with Language-image Pre-training
K-Space Transformer for Undersampled MRI Reconstruction
This paper considers the problem of undersampled MRI reconstruction.
Temporal Alignment Networks for Long-term Video
The objective of this paper is a temporal alignment network that ingests long term video sequences, and associated text sentences, in order to: (1) determine if a sentence is alignable with the video; and (2) if it is alignable, then determine its alignment.
PromptDet: Towards Open-vocabulary Detection using Uncurated Images
The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations.
Unsupervised Salient Object Detection with Spectral Cluster Voting
In this paper, we tackle the challenging task of unsupervised salient object detection (SOD) by leveraging spectral clustering on self-supervised features.
Ranked #2 on
Unsupervised Saliency Detection
on DUTS
Label, Verify, Correct: A Simple Few Shot Object Detection Method
The objective of this paper is few-shot object detection (FSOD) -- the task of expanding an object detector for a new category given only a few instances for training.
Audio-Visual Synchronisation in the wild
Finally, we set the first benchmark for general audio-visual synchronisation with over 160 diverse classes in the new VGG-Sound Sync video dataset.
Prompting Visual-Language Models for Efficient Video Understanding
Image-based visual-language (I-VL) pre-training has shown great success for learning joint visual-textual representations from large-scale web data, revealing remarkable ability for zero-shot generalisation.
Ranked #5 on
Zero-Shot Action Detection
on ActivityNet-1.3
It's About Time: Analog Clock Reading in the Wild
In this paper, we present a framework for reading analog clocks in natural images or videos.
ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation
The objective of this work is to achieve sensorless reconstruction of a 3D volume from a set of 2D freehand ultrasound images with deep implicit representation.
Self-supervised Tumor Segmentation through Layer Decomposition
In this paper, we target self-supervised representation learning for zero-shot tumor segmentation.
Sli2Vol: Annotate a 3D Volume from a Single Slice with Self-Supervised Learning
The objective of this work is to segment any arbitrary structures of interest (SOI) in 3D volumes by only annotating a single slice, (i. e. semi-automatic 3D segmentation).
Self-supervised Video Object Segmentation by Motion Grouping
We additionally evaluate on a challenging camouflage dataset (MoCA), significantly outperforming the other self-supervised approaches, and comparing favourably to the top supervised approach, highlighting the importance of motion cues, and the potential bias towards visual appearance in existing video segmentation models.
Ranked #7 on
Unsupervised Object Segmentation
on DAVIS 2016
All you need are a few pixels: semantic segmentation with PixelPick
A central challenge for the task of semantic segmentation is the prohibitive cost of obtaining dense pixel-level annotations to supervise model training.
Localizing Visual Sounds the Hard Way
We show that our algorithm achieves state-of-the-art performance on the popular Flickr SoundNet dataset.
Quantum Self-Supervised Learning
The resurgence of self-supervised learning, whereby a deep learning model generates its own supervisory signal from the data, promises a scalable way to tackle the dramatically increasing size of real-world data sets without human annotation.
NeRF--: Neural Radiance Fields Without Known Camera Parameters
Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses.
VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge
We held the second installment of the VoxCeleb Speaker Recognition Challenge in conjunction with Interspeech 2020.
Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentation
We make the following three contributions: (i) We propose a novel architecture that consists of two essential components for breaking camouflage, namely, a differentiable registration module to align consecutive frames based on the background, which effectively emphasises the object boundary in the difference image, and a motion segmentation module with memory that discovers the moving objects, while maintaining the object permanence even when motion is absent at some point.
Self-supervised Co-training for Video Representation Learning
The objective of this paper is visual-only self-supervised video representation learning.
Ranked #12 on
Self-Supervised Action Recognition
on HMDB51 (finetuned)
Layered Neural Rendering for Retiming People in Video
We present a method for retiming people in an ordinary, natural video -- manipulating and editing the time in which different motions of individuals in the video occur.
Inducing Predictive Uncertainty Estimation for Face Recognition
We describe three use cases on the public IJB-C face verification benchmark: (i) to improve 1:1 image-based verification error rates by rejecting low-quality face images; (ii) to improve quality score based fusion performance on the 1:1 set-based verification benchmark; and (iii) its use as a quality measure for selecting high quality (unblurred, good lighting, more frontal) faces from a collection, e. g. for automatic enrolment or display.
Memory-augmented Dense Predictive Coding for Video Representation Learning
The objective of this paper is self-supervised learning from video, in particular for representations for action recognition.
Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval
Optimising a ranking-based metric, such as Average Precision (AP), is notoriously challenging due to the fact that it is non-differentiable, and hence cannot be optimised directly using gradient-descent methods.
Ranked #4 on
Vehicle Re-Identification
on VehicleID Medium
Self-supervised Video Object Segmentation
The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a. k. a.
One-shot visual object segmentation
Representation Learning
+2
VGGSound: A Large-scale Audio-Visual Dataset
Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques.
MAST: A Memory-Augmented Self-supervised Tracker
Recent interest in self-supervised dense tracking has yielded rapid progress, but performance still remains far from supervised methods.
Ranked #4 on
Unsupervised Video Object Segmentation
on DAVIS 2017 (val)
(using extra training data)
Semantic Segmentation
Semi-Supervised Video Object Segmentation
+2
VoxSRC 2019: The first VoxCeleb Speaker Recognition Challenge
The VoxCeleb Speaker Recognition Challenge 2019 aimed to assess how well current speaker recognition technology is able to identify speakers in unconstrained or `in the wild' data.
Video Representation Learning by Dense Predictive Coding
The objective of this paper is self-supervised learning of spatio-temporal embeddings from video, suitable for human action recognition.
Ranked #32 on
Self-Supervised Action Recognition
on UCF101
Representation Learning
Self-Supervised Action Recognition
+2
Geometry-Aware Video Object Detection for Static Cameras
In this paper we propose a geometry-aware model for video object detection.
AutoCorrect: Deep Inductive Alignment of Noisy Geometric Annotations
We propose AutoCorrect, a method to automatically learn object-annotation alignments from a dataset with annotations affected by geometric noise.
Self-supervised Learning for Video Correspondence Flow
Fourth, in order to shed light on the potential of self-supervised learning on the task of video correspondence flow, we probe the upper bound by training on additional data, \ie more diverse videos, further demonstrating significant improvements on video segmentation.
Self-Supervised Learning
Semi-Supervised Video Object Segmentation
+4
Utterance-level Aggregation For Speaker Recognition In The Wild
The objective of this paper is speaker recognition "in the wild"-where utterances may be of variable length and also contain irrelevant signals.
Class-Agnostic Counting
The model achieves competitive performance on cell and crowd counting datasets, and surpasses the state-of-the-art on the car dataset using only three training images.
Comparator Networks
Our contributions are: (i) We propose a Deep Comparator Network (DCN) that can ingest a pair of sets (each may contain a variable number of images) as inputs, and compute a similarity between the pair--this involves attending to multiple discriminative local regions (landmarks), and comparing local descriptors between pairs of faces; (ii) To encourage high-quality representations for each set, internal competition is introduced for recalibration based on the landmark score; (iii) Inspired by image retrieval, a novel hard sample mining regime is proposed to control the sampling process, such that the DCN is complementary to the standard image classification models.
Multicolumn Networks for Face Recognition
In this paper, we design a neural network architecture that learns to aggregate based on both "visual" quality (resolution, illumination), and "content" quality (relative importance for discriminative classification).
Ranked #5 on
Face Verification
on IJB-C
(TAR @ FAR=1e-2 metric)
Ω-Net (Omega-Net): Fully Automatic, Multi-View Cardiac MR Detection, Orientation, and Segmentation with Deep Neural Networks
Pixelwise segmentation of the left ventricular (LV) myocardium and the four cardiac chambers in 2-D steady state free precession (SSFP) cine sequences is an essential preprocessing step for a wide range of analyses.
VGGFace2: A dataset for recognising faces across pose and age
The dataset was collected with three goals in mind: (i) to have both a large number of identities and also a large number of images for each identity; (ii) to cover a large range of pose, age and ethnicity; and (iii) to minimize the label noise.
Ranked #1 on
Face Verification
on IJB-C
(training dataset metric)
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Sonography synthesis has a wide range of applications, including medical procedure simulation, clinical training and multimodality image registration.
Feature Tracking Cardiac Magnetic Resonance via Deep Learning and Spline Optimization
Feature tracking Cardiac Magnetic Resonance (CMR) has recently emerged as an area of interest for quantification of regional cardiac function from balanced, steady state free precession (SSFP) cine sequences.
