Search Results for author:
Found 105 papers, 53 papers with code
Low Bias Low Variance Gradient Estimates for Hierarchical Boolean Stochastic Networks
Stochastic neural networks with discrete random variables are an important class of models for their expressiveness and interpretability.
Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination
In this paper, we introduce a new paradigm for constructing world models that are explicit representations of the real world and its dynamics.
Redefining Normal: A Novel Object-Level Approach for Multi-Object Novelty Detection
Adapting to our object-level definition of `normal', we modify knowledge distillation frameworks, where a student network learns from a pre-trained teacher network.
From MLP to NeoMLP: Leveraging Self-Attention for Neural Fields
NeoMLP has a built-in mechanism for conditioning through the hidden and output nodes, which function as a set of latent codes, and as such, NeoMLP can be used straightforwardly as a conditional neural field.
Any-Resolution AI-Generated Image Detection by Spectral Learning
Since generated images constitute out-of-distribution samples for this model, we propose spectral reconstruction similarity to capture this divergence.
CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation
To solve this issue, we propose Cooperative Plan Optimization (CaPo) to enhance the cooperation efficiency of LLM-based embodied agents.
Language Agents Meet Causality -- Bridging LLMs and Causal World Models
We propose a framework that integrates CRLs with LLMs to enable causally-aware reasoning and planning.
NARAIM: Native Aspect Ratio Autoregressive Image Models
While vision transformers are able to solve a wide variety of computer vision tasks, no pre-training method has yet demonstrated the same scaling laws as observed in language models.
SIGMA:Sinkhorn-Guided Masked Video Modeling
To tackle this, we present Sinkhorn-guided Masked Video Modelling (SIGMA), a novel video pretraining method that jointly learns the video model in addition to a target feature space using a projection network.
GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features
These features are fed to an attention-based discriminator, which is trained to score every patch in the image.
Ranked #1 on
Anomaly Detection
on One-class CIFAR-100
VISA: Reasoning Video Object Segmentation via Large Language Models
In this paper, we introduce a new task, Reasoning Video Object Segmentation (ReasonVOS).
Ranked #3 on
Referring Video Object Segmentation
on ReVOS
Mining Open Semantics from CLIP: A Relation Transition Perspective for Few-Shot Learning
Specifically, we adopt a transformer module which takes the visual feature as "Query", the text features of the anchors as "Key" and the similarity matrix between the text features of anchor and target classes as "Value".
Space-Time Continuous PDE Forecasting using Equivariant Neural Fields
Recently, Conditional Neural Fields (NeFs) have emerged as a powerful modelling paradigm for PDEs, by learning solutions as flows in the latent space of the Conditional NeF.
Grounding Continuous Representations in Geometry: Equivariant Neural Fields
Conditional Neural Fields (CNFs) are increasingly being leveraged as continuous signal representations, by associating each data-sample with a latent variable that conditions a shared backbone Neural Field (NeF) to reconstruct the sample.
Amortized Equation Discovery in Hybrid Dynamical Systems
Hybrid dynamical systems are prevalent in science and engineering to express complex systems with continuous and discrete states.
Ai-Sampler: Adversarial Learning of Markov kernels with involutive maps
Markov chain Monte Carlo methods have become popular in statistics as versatile techniques to sample from complicated probability distributions.
Domain Adaptation with Cauchy-Schwarz Divergence
Furthermore, we illustrate that the CS divergence enables a simple estimator on the discrepancy of both marginal and conditional distributions between source and target domains in the representation space, without requiring any distributional assumptions.
Self-supervised visual learning in the low-data regime: a comparative evaluation
Self-Supervised Learning (SSL) is a valuable and robust training methodology for contemporary Deep Neural Networks (DNNs), enabling unsupervised pretraining on a 'pretext task' that does not require ground-truth labels/annotation.
Mechanistic Interpretability for AI Safety -- A Review
Understanding AI systems' inner workings is critical for ensuring value alignment and safety.
Graph Neural Networks for Learning Equivariant Representations of Neural Networks
Neural networks that process the parameters of other neural networks find applications in domains as diverse as classifying implicit neural representations, generating neural network weights, and predicting generalization errors.
Mechanistic Neural Networks for Scientific Machine Learning
This paper presents Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences.
Dynamic Prototype Adaptation with Distillation for Few-shot Point Cloud Segmentation
Existing prototype-based methods rely on support prototypes to guide the segmentation of query point clouds, but they encounter challenges when significant object variations exist between the support prototypes and query features.
How to Train Neural Field Representations: A Comprehensive Study and Benchmark
With this library, we perform a comprehensive study that investigates the effects of different hyperparameters on fitting NeFs for downstream tasks.
Motion Flow Matching for Human Motion Synthesis and Editing
In this paper, we propose \emph{Motion Flow Matching}, a novel generative model designed for human motion generation featuring efficient sampling and effectiveness in motion editing applications.
Data Augmentations in Deep Weight Spaces
Learning in weight spaces, where neural networks process the weights of other deep neural networks, has emerged as a promising research direction with applications in various fields, from analyzing and editing neural fields and implicit neural representations, to network pruning and quantization.
Latent Field Discovery In Interacting Dynamical Systems With Neural Fields
Systems of interacting objects often evolve under the influence of field effects that govern their dynamics, yet previous works have abstracted away from such effects, and assume that systems evolve in a vacuum.
Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations
Our paper aims to address this gap by proposing a novel approach that incorporates temporal consistency in dense self-supervised learning.
Neural Modulation Fields for Conditional Cone Beam Neural Tomography
Conventional Computed Tomography (CT) methods require large numbers of noise-free projections for accurate density reconstructions, limiting their applicability to the more complex class of Cone Beam Geometry CT (CBCT) reconstruction.
Learning Lie Group Symmetry Transformations with Neural Networks
The problem of detecting and quantifying the presence of symmetries in datasets is useful for model selection, generative modeling, and data analysis, amongst others.
BISCUIT: Causal Representation Learning from Binary Interactions
Identifying the causal variables of an environment and how to intervene on them is of core value in applications such as robotics and embodied AI.
Graph Switching Dynamical Systems
Dynamical systems with complex behaviours, e. g. immune system cells interacting with a pathogen, are commonly modelled by splitting the behaviour into different regimes, or modes, each with simpler dynamics, and then learning the switching behaviour from one mode to another.
PiClick: Picking the desired mask from multiple candidates in click-based interactive segmentation
Click-based interactive segmentation aims to generate target masks via human clicking, which facilitates efficient pixel-level annotation and image editing.
Towards Open-Vocabulary Video Instance Segmentation
Video Instance Segmentation (VIS) aims at segmenting and categorizing objects in videos from a closed set of training categories, lacking the generalization ability to handle novel categories in real-world videos.
Modulated Neural ODEs
Neural ordinary differential equations (NODEs) have been proven useful for learning non-linear dynamics of arbitrary trajectories.
Modelling Long Range Dependencies in $N$D: From Task-Specific to a General Purpose CNN
Performant Convolutional Neural Network (CNN) architectures must be tailored to specific tasks in order to consider the length, resolution, and dimensionality of the input data.
Few-shot Semantic Segmentation with Support-induced Graph Convolutional Network
However, the appearance variations between objects from the same category could be extremely large, leading to unreliable feature matching and query mask prediction.
Ranked #41 on
Few-Shot Semantic Segmentation
on PASCAL-5i (1-Shot)
Differentiable Mathematical Programming for Object-Centric Representation Learning
To solve the graph cuts our solution relies on an efficient, scalable, and differentiable quadratic programming approximation.
Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing
Our results suggest that reservoir computing is a promising candidate framework for the continual learning of dynamical systems.
Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systems
To address this issue, we propose iCITRIS, a causal representation learning method that allows for instantaneous effects in intervened temporal sequences when intervention targets can be observed, e. g., as actions of an agent.
Towards a General Purpose CNN for Long Range Dependencies in $N$D
The use of Convolutional Neural Networks (CNNs) is widespread in Deep Learning due to a range of desirable model properties which result in an efficient and effective machine learning framework.
Fake It Till You Make It: Towards Accurate Near-Distribution Novelty Detection
Effectiveness of our method for both the near-distribution and standard novelty detection is assessed through extensive experiments on datasets in diverse applications such as medical images, object classification, and quality control.
Ranked #3 on
Anomaly Detection
on One-class CIFAR-10
(using extra training data)
Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation
Specifically, in DPCN, a dynamic convolution module (DCM) is firstly proposed to generate dynamic kernels from support foreground, then information interaction is achieved by convolution operations over query features using these kernels.
Ranked #33 on
Few-Shot Semantic Segmentation
on PASCAL-5i (1-Shot)
NFormer: Robust Person Re-identification with Neighbor Transformer
To tackle this issue, we propose a Neighbor Transformer Network, or NFormer, which explicitly models interactions across all input images, thus suppressing outlier features and leading to more robust representations overall.
3D Equivariant Graph Implicit Functions
In this work, we address two key limitations of such representations, in failing to capture local 3D geometric fine details, and to learn from and generalize to shapes with unseen 3D transformations.
Delta Distillation for Efficient Video Processing
By extensive experiments on a wide range of architectures, including the most efficient ones, we demonstrate that delta distillation sets a new state of the art in terms of accuracy vs. efficiency trade-off for semantic segmentation and object detection in videos.
Ranked #2 on
Video Semantic Segmentation
on Cityscapes val
Reliably Re-Acting to Partner's Actions with the Social Intrinsic Motivation of Transfer Empowerment
We consider multi-agent reinforcement learning (MARL) for cooperative communication and coordination tasks.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
CITRIS: Causal Identifiability from Temporal Intervened Sequences
Understanding the latent causal factors of a dynamical system from visual observations is considered a crucial step towards agents reasoning in complex environments.
Roto-translated Local Coordinate Frames For Interacting Dynamical Systems
Modelling interactions is critical in learning complex dynamical systems, namely systems of interacting objects with highly non-linear and time-dependent behaviour.
Stability Regularization for Discrete Representation Learning
Stability regularization is method to make the output of continuous functions of Gaussian random variables close to discrete, that is binary or categorical, without the need for significant manual tuning.
WeakSTIL: Weak whole-slide image level stromal tumor infiltrating lymphocyte scores are all you need
We present WeakSTIL, an interpretable two-stage weak label deep learning pipeline for scoring the percentage of stromal tumor infiltrating lymphocytes (sTIL%) in H&E-stained whole-slide images (WSIs) of breast cancer tissue.
Efficient Neural Causal Discovery without Acyclicity Constraints
Learning the structure of a causal graphical model using both observational and interventional data is a fundamental problem in many scientific fields.
DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer
For MSI prediction in a tumor-annotated and color normalized subset of TCGA-CRC (n=360 patients), contrastive self-supervised learning improves the tile supervision baseline from 0. 77 to 0. 87 AUROC, on par with our proposed DeepSMILE method.
Federated Mixture of Experts
Federated learning (FL) has emerged as the predominant approach for collaborative training of neural network models across multiple users, without the need to gather the data at a central location.
Unsharp Mask Guided Filtering
The state-of-the-art leverages deep networks to estimate the two core coefficients of the guided filter.
Sparse-shot Learning with Exclusive Cross-Entropy for Extremely Many Localisations
However, outside the setting of regular images, we are often confronted with a different situation.
Batch Bayesian Optimization on Permutations using the Acquisition Weighted Kernel
In this work we propose a batch Bayesian optimization method for combinatorial problems on permutations, which is well suited for expensive-to-evaluate objectives.
Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
In experiments, we demonstrate the improved sample efficiency of GP BO using FM kernels (BO-FM). On synthetic problems and hyperparameter optimization problems, BO-FM outperforms competitors consistently.
Variance Reduction in Hierarchical Variational Autoencoders
Variational autoencoders with deep hierarchies of stochastic layers have been known to suffer from the problem of posterior collapse, where the top layers fall back to the prior and become independent of input.
Rotation Equivariant Siamese Networks for Tracking
We further show that this change in orientation can be used to impose an additional motion constraint in Siamese tracking through imposing restriction on the change in orientation between two consecutive frames.
Siamese Tracking with Lingual Object Constraints
Classically, visual object tracking involves following a target object throughout a given video, and it provides us the motion trajectory of the object.
Self-Selective Context for Interaction Recognition
Our experiments show that SSC leads to an important increase in interaction recognition performance, while using much fewer parameters.
PointMixup: Augmentation for Point Clouds
In this paper, we define data augmentation between point clouds as a shortest path linear interpolation.
Ranked #3 on
3D Point Cloud Data Augmentation
on ModelNet40
3D Point Cloud Classification
3D Point Cloud Data Augmentation
+2
Tackling Occlusion in Siamese Tracking with Structured Dropouts
Specifically, we present structured dropout to mimick the change in latent codes under occlusion.
Categorical Normalizing Flows via Continuous Transformations
Based on Categorical Normalizing Flows, we propose GraphCNF a permutation-invariant generative model on graphs.
PIC: Permutation Invariant Convolution for Recognizing Long-range Activities
We study the three properties of PIC and demonstrate its effectiveness in recognizing the long-range activities of Charades, Breakfast, and MultiThumos.
Siamese Tracking of Cell Behaviour Patterns
Tracking and segmentation of biological cells in video sequences is a challenging problem, especially due to the similarity of the cells and high levels of inherent noise.
Tracking-Assisted Segmentation of Biological Cells
U-Net and its variants have been demonstrated to work sufficiently well in biological cell tracking and segmentation.
SafeCritic: Collision-Aware Trajectory Prediction
The Critic network is environmentally aware to prune trajectories that are in collision or are in violation with the environment.
Increasing Expressivity of a Hyperspherical VAE
Learning suitable latent representations for observed, high-dimensional data is an important research topic underlying many recent advances in machine learning.
3D Neighborhood Convolution: Learning Depth-Aware Features for RGB-D and RGB Semantic Segmentation
We propose to model the effective receptive field of 2D convolution based on the scale and locality from the 3D neighborhood.
Low Bias Gradient Estimates for Very Deep Boolean Stochastic Networks
In this work we focus on stochastic networks with multiple layers of Boolean latent variables.
I Bet You Are Wrong: Gambling Adversarial Networks for Structured Semantic Segmentation
Adversarial training has been recently employed for realizing structured semantic segmentation, in which the aim is to preserve higher-level scene structural consistencies in dense predictions.
Model Decay in Long-Term Tracking
Updating the tracker model with adverse bounding box predictions adds an unavoidable bias term to the learning.
VideoGraph: Recognizing Minutes-Long Human Activities in Videos
To represent them, related works opt for statistical pooling, which neglects the temporal structure.
Ranked #6 on
Long-video Activity Recognition
on Breakfast
Initialized Equilibrium Propagation for Backprop-Free Training
In response to this, Scellier & Bengio (2017) proposed Equilibrium Propagation - a method for gradient-based train- ing of neural networks which uses only local learning rules and, crucially, does not rely on neurons having a mechanism for back-propagating an error gradient.
Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres
We observe many continuous output problems in computer vision are naturally contained in closed geometrical manifolds, like the Euler angles in viewpoint estimation or the normals in surface normal estimation.
Combinatorial Bayesian Optimization using the Graph Cartesian Product
On this combinatorial graph, we propose an ARD diffusion kernel with which the GP is able to model high-order interactions between variables leading to better performance.
Timeception for Complex Action Recognition
This paper focuses on the temporal aspect for recognizing human activities in videos; an important visual cue that has long been undervalued.
Ranked #7 on
Long-video Activity Recognition
on Breakfast
Relaxed Quantization for Discretized Neural Networks
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices.
Dynamic Adaptation on Non-Stationary Visual Domains
To demonstrate the effectiveness of our proposed framework, we modify associative domain adaptation to work well on source and target data batches with unequal class distributions.
Video Time: Properties, Encoders and Evaluation
Time-aware encoding of frame sequences in a video is a fundamental problem in video understanding.
BOCK : Bayesian Optimization with Cylindrical Kernels
A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space.
Long-term Tracking in the Wild: A Benchmark
We introduce the OxUvA dataset and benchmark for evaluating single-object tracking algorithms.
Explaining the Mistakes of Neural Networks with Latent Sympathetic Examples
Neural networks make mistakes.
Examining Cooperation in Visual Dialog Models
In this work we propose a blackbox intervention method for visual dialog models, with the aim of assessing the contribution of individual linguistic or visual components.
Tracking for Half an Hour
Long-term tracking requires extreme stability to the multitude of model updates and robustness to the disappearance and loss of the target as such will inevitably happen.
Tracking by Natural Language Specification
This paper strives to track a target object in a video.
Ranked #17 on
Referring Expression Segmentation
on J-HMDB
Temporally Efficient Deep Learning with Spikes
We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data - not the rate at which we process the data.
Unified Embedding and Metric Learning for Zero-Exemplar Event Detection
Given a text describing a novel event, the goal is to rank related videos accordingly.
Action Recognition with Dynamic Image Networks
This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos.
Self-Supervised Video Representation Learning With Odd-One-Out Networks
On action classification, our method obtains 60. 3\% on the UCF101 dataset using only UCF101 data for training which is approximately 10% better than current state-of-the-art self-supervised learning methods.
Ranked #47 on
Self-Supervised Action Recognition
on UCF101
VideoLSTM Convolves, Attends and Flows for Action Recognition
We present a new architecture for end-to-end sequence learning of actions in video, we call VideoLSTM.
Dynamic Image Networks for Action Recognition
We introduce the concept of dynamic image, a novel compact representation of videos useful for video analysis especially when convolutional neural networks (CNNs) are used.
Ranked #62 on
Action Recognition
on HMDB-51
Siamese Instance Search for Tracking
In this paper we present a tracker, which is radically different from state-of-the-art trackers: we apply no model updating, no occlusion detection, no combination of trackers, no geometric matching, and still deliver state-of-the-art tracking performance, as demonstrated on the popular online tracking benchmark (OTB) and six very challenging YouTube videos.
Online Action Detection
Third, the start of the action is unknown, so it is unclear over what time window the information should be integrated.
Rank Pooling for Action Recognition
We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation.
Learning to Rank Based on Subsequences
We present a supervised learning to rank algorithm that effectively orders images by exploiting the structure in image sequences.
Guiding the Long-Short Term Memory Model for Image Caption Generation
In this work we focus on the problem of image caption generation.
MidRank: Learning to rank based on subsequences
We present a supervised learning to rank algorithm that effectively orders images by exploiting the structure in image sequences.
Deep Reflectance Maps
Undoing the image formation process and therefore decomposing appearance into its intrinsic properties is a challenging task due to the under-constraint nature of this inverse problem.
Active Transfer Learning with Zero-Shot Priors: Reusing Past Datasets for Future Tasks
How can we reuse existing knowledge, in the form of available datasets, when solving a new and apparently unrelated target task from a set of unlabeled data?
Guiding Long-Short Term Memory for Image Caption Generation
In this work we focus on the problem of image caption generation.
Modeling Video Evolution for Action Recognition
We postulate that a function capable of ordering the frames of a video temporally (based on the appearance) captures well the evolution of the appearance within the video.
COSTA: Co-Occurrence Statistics for Zero-Shot Classification
In this paper we aim for zero-shot classification, that is visual recognition of an unseen class by using knowledge transfer from known classes.
Locality in Generic Instance Search from One Example
This paper aims for generic instance search from a single example.
