Search Results for author:
Found 86 papers, 36 papers with code
LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization
Temporal Action Localization (TAL) involves localizing and classifying action snippets in an untrimmed video.
BarcodeBERT: Transformers for Biodiversity Analysis
Understanding biodiversity is a global challenge, in which DNA barcodes - short snippets of DNA that cluster by species - play a pivotal role.
Bandit-Driven Batch Selection for Robust Learning under Label Noise
We introduce a novel approach for batch selection in Stochastic Gradient Descent (SGD) training, leveraging combinatorial bandit algorithms.
Which Tokens to Use? Investigating Token Reduction in Vision Transformers
While different methods have been explored to achieve this goal, we still lack understanding of the resulting reduction patterns and how those patterns differ across token reduction methods and datasets.
A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset
In an effort to catalog insect biodiversity, we propose a new large dataset of hand-labelled insect images, the BIOSCAN-Insect Dataset.
Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Data
Here, we characterize the performance of several post-hoc and training-based conformal prediction methods under these settings, providing the first empirical evaluation on large-scale datasets and models.
Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
Equipped with the learned unstructured attention pattern, sparse attention ViT (Sparsifiner) produces a superior Pareto-optimal trade-off between FLOPs and top-1 accuracy on ImageNet compared to token sparsity.
GCNet: Probing Self-Similarity Learning for Generalized Counting Network
The class-agnostic counting (CAC) problem has caught increasing attention recently due to its wide societal applications and arduous challenges.
Quantum HyperNetworks: Training Binary Neural Networks in Quantum Superposition
Binary neural networks, i. e., neural networks whose parameters and activations are constrained to only two possible values, offer a compelling avenue for the deployment of deep learning models on energy- and memory-limited devices.
Bounding generalization error with input compression: An empirical study with infinite-width networks
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data.
Monitoring Shortcut Learning using Mutual Information
The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critical domains such as healthcare, finance and autonomous vehicles.
Understanding the impact of image and input resolution on deep digital pathology patch classifiers
Our experiments show that patch classification performance can be improved by manipulating both the image and input resolution in annotation-scarce and annotation-rich environments.
DeepRNG: Towards Deep Reinforcement Learning-Assisted Generative Testing of Software
Although machine learning (ML) has been successful in automating various software engineering needs, software testing still remains a highly challenging topic.
On Evaluation Metrics for Graph Generative Models
While we focus on applying these metrics to GGM evaluation, in practice this enables the ability to easily compute the dissimilarity between any two sets of graphs regardless of domain.
Learning with Less Labels in Digital Pathology via Scribble Supervision from Natural Images
Furthermore, we show that models trained with scribble labels yield the same performance boost as full pixel-wise segmentation labels despite being significantly easier and faster to collect.
Domain-Agnostic Clustering with Self-Distillation
Recent advancements in self-supervised learning have reduced the gap between supervised and unsupervised representation learning.
Empirical analysis of representation learning and exploration in neural kernel bandits
We consider policies based on a GP and a Student's t-process (TP).
Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
Discovering a solution in a combinatorial space is prevalent in many real-world problems but it is also challenging due to diverse complex constraints and the vast number of possible combinations.
Parameter Prediction for Unseen Deep Architectures
We introduce a large-scale dataset of diverse computational graphs of neural architectures - DeepNets-1M - and use it to explore parameter prediction on CIFAR-10 and ImageNet.
Ranked #1 on
Parameter Prediction
on CIFAR10
Neural Structure Mapping For Learning Abstract Visual Analogies
Our framework uses (1) a multi-task visual relationship encoder to extract constituent concepts from raw visual input in the source domain, and (2) a neural module net analogy inference engine to reason compositionally about the inferred relation in the target domain.
Unconstrained Scene Generation with Locally Conditioned Radiance Fields
In this paper, we introduce Generative Scene Networks (GSN), which learns to decompose scenes into a collection of many local radiance fields that can be rendered from a free moving camera.
Ranked #1 on
Scene Generation
on VizDoom
The GIST and RIST of Iterative Self-Training for Semi-Supervised Segmentation
We further show that GIST and RIST can be combined with existing semi-supervised learning methods to boost performance.
LOHO: Latent Optimization of Hairstyles via Orthogonalization
Therefore, we propose Latent Optimization of Hairstyles via Orthogonalization (LOHO), an optimization-based approach using GAN inversion to infill missing hair structure details in latent space during hairstyle transfer.
SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentation
SST extracts per-pixel representations for each object in a video using sparse attention over spatiotemporal features.
Context-Aware Scene Graph Generation With Seq2Seq Transformers
In this task, the model needs to detect objects and predict visual relationships between them.
Building LEGO Using Deep Generative Models of Graphs
Generative models are now used to create a variety of high-quality digital artifacts.
Evaluating Curriculum Learning Strategies in Neural Combinatorial Optimization
Neural combinatorial optimization (NCO) aims at designing problem-independent and efficient neural network-based strategies for solving combinatorial problems.
Identifying and interpreting tuning dimensions in deep networks
While researchers have attempted to manually identify an analogue to these tuning dimensions in deep neural networks, we are unaware of an automatic way to discover them.
Instance Selection for GANs
By refining the empirical data distribution before training, we redirect model capacity towards high-density regions, which ultimately improves sample fidelity, lowers model capacity requirements, and significantly reduces training time.
Ranked #2 on
Conditional Image Generation
on ImageNet 64x64
Generative Compositional Augmentations for Scene Graph Prediction
However, test images might contain zero- and few-shot compositions of objects and relationships, e. g. <cup, on, surfboard>.
Enabling Continual Learning with Differentiable Hebbian Plasticity
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.
Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation
We show that such models can suffer the most in their ability to generalize to rare compositions, evaluating two different models on the Visual Genome dataset and its more recent, improved version, GQA.
Sample-Efficient Model-based Actor-Critic for an Interactive Dialogue Task
Human-computer interactive systems that rely on machine learning are becoming paramount to the lives of millions of people who use digital assistants on a daily basis.
ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis
Additionally, our proposed fast moving proxies also addresses small gradient issue of proxies, and this component synergizes well with low temperature scaling and Global Max Pooling.
Ranked #2 on
Image Retrieval
on CARS196
Learning with less data via Weakly Labeled Patch Classification in Digital Pathology
In Digital Pathology (DP), labeled data is generally very scarce due to the requirement that medical experts provide annotations.
Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking
Deep neural networks require collecting and annotating large amounts of data to train successfully.
Ranked #44 on
Self-Supervised Action Recognition
on UCF101
A Nonparametric Bayesian Model for Sparse Dynamic Multigraphs
As the availability and importance of temporal interaction data--such as email communication--increases, it becomes increasingly important to understand the underlying structure that underpins these interactions.
Differentiable Hebbian Consolidation for Continual Learning
Continual learning is the problem of sequentially learning new tasks or knowledge while protecting previously acquired knowledge.
Learning Temporal Attention in Dynamic Graphs with Bilinear Interactions
We consider a common case in which edges can be short term interactions (e. g., messaging) or long term structural connections (e. g., friendship).
Image Classification with Hierarchical Multigraph Networks
Graph Convolutional Networks (GCNs) are a class of general models that can learn from graph structured data.
On the Evaluation of Conditional GANs
We show that FJD can be used as a promising single metric for cGAN benchmarking and model selection.
Sequential Edge Clustering in Temporal Multigraphs
Interaction graphs, such as those recording emails between individuals or transactions between institutions, tend to be sparse yet structured, and often grow in an unbounded manner.
Understanding Attention and Generalization in Graph Neural Networks
We aim to better understand attention over nodes in graph neural networks (GNNs) and identify factors influencing its effectiveness.
Ranked #25 on
Graph Classification
on D&D
Batch Normalization is a Cause of Adversarial Vulnerability
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks.
Similarity Learning Networks for Animal Individual Re-Identification -- Beyond the Capabilities of a Human Observer
Without any species-specific modifications, our results demonstrate that similarity comparison networks can reach a performance level beyond that of humans for the task of animal re-identification.
SISC: End-to-end Interpretable Discovery Radiomics-Driven Lung Cancer Prediction via Stacked Interpretable Sequencing Cells
Conclusion: The SISC radiomic sequencer is able to achieve state-of-the-art results in lung cancer prediction, and also offers prediction interpretability in the form of critical response maps.
Adversarial Examples as an Input-Fault Tolerance Problem
We analyze the adversarial examples problem in terms of a model's fault tolerance with respect to its input.
Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instruction
Conditional text-to-image generation is an active area of research, with many possible applications.
Ranked #2 on
Text-to-Image Generation
on GeNeVA (i-CLEVR)
Spectral Multigraph Networks for Discovering and Fusing Relationships in Molecules
Spectral Graph Convolutional Networks (GCNs) are a generalization of convolutional networks to learning on graph-structured data.
Ranked #12 on
Graph Classification
on NCI109
Past, Present, and Future Approaches Using Computer Vision for Animal Re-Identification from Camera Trap Data
The ability of a researcher to re-identify (re-ID) an individual animal upon re-encounter is fundamental for addressing a broad range of questions in the study of ecosystem function, community and population dynamics, and behavioural ecology.
A Rate-Distortion Theory of Adversarial Examples
The generalization ability of deep neural networks (DNNs) is intertwined with model complexity, robustness, and capacity.
Self-Paced Learning with Adaptive Deep Visual Embeddings
Selecting the most appropriate data examples to present a deep neural network (DNN) at different stages of training is an unsolved challenge.
Stochastic Layer-Wise Precision in Deep Neural Networks
Low precision weights, activations, and gradients have been proposed as a way to improve the computational efficiency and memory footprint of deep neural networks.
Leveraging Uncertainty Estimates for Predicting Segmentation Quality
The first is producing spatial uncertainty maps, from which a clinician can observe where and why a system thinks it is failing.
Adversarial Training Versus Weight Decay
Performance-critical machine learning models should be robust to input perturbations not seen during training.
Deep Learning Object Detection Methods for Ecological Camera Trap Data
Recent advances in the field of deep learning for object detection show promise towards automating the analysis of camera trap images.
Generalized Hadamard-Product Fusion Operators for Visual Question Answering
We propose a generalized class of multimodal fusion operators for the task of visual question answering (VQA).
Real-Time End-to-End Action Detection with Two-Stream Networks
Finally, for better network initialization, we transfer from the task of action recognition to action detection by pre-training our framework using the recently released large-scale Kinetics dataset.
Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points
No spatial coherence is forced on the glimpse locations, which gives the module liberty to explore different points at each frame and better optimize the process of scrutinizing visual information.
Ranked #19 on
Skeleton Based Action Recognition
on N-UCLA
Learning Confidence for Out-of-Distribution Detection in Neural Networks
Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong.
Predicting Adversarial Examples with High Confidence
It has been suggested that adversarial examples cause deep learning models to make incorrect predictions with high confidence.
Bit-Regularized Optimization of Neural Nets
Our key idea is to control the expressive power of the network by dynamically quantizing the range and set of values that the parameters can take.
LEAP: Learning Embeddings for Adaptive Pace
The \textit{student} CNN classifier dynamically selects samples to form a mini-batch based on the \textit{easiness} from cross-entropy losses and \textit{true diverseness} of examples from the representation space sculpted by the \textit{embedding} CNN.
Attacking Binarized Neural Networks
Neural networks with low-precision weights and activations offer compelling efficiency advantages over their full-precision equivalents.
Discovery Radiomics with CLEAR-DR: Interpretable Computer Aided Diagnosis of Diabetic Retinopathy
Conclusion: We demonstrate the effectiveness and utility of the proposed CLEAR-DR system of enhancing the interpretability of diagnostic grading results for the application of diabetic retinopathy grading.
Opening the Black Box of Financial AI with CLEAR-Trade: A CLass-Enhanced Attentive Response Approach for Explaining and Visualizing Deep Learning-Driven Stock Market Prediction
However, current deep learning algorithms have been criticized as uninterpretable "black-boxes" which cannot explain their decision making processes.
Improved Regularization of Convolutional Neural Networks with Cutout
Convolutional neural networks are capable of learning powerful representational spaces, which are necessary for tackling complex learning tasks.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Structure Optimization for Deep Multimodal Fusion Networks using Graph-Induced Kernels
A popular testbed for deep learning has been multimodal recognition of human activity or gesture involving diverse inputs such as video, audio, skeletal pose and depth images.
Explaining the Unexplained: A CLass-Enhanced Attentive Response (CLEAR) Approach to Understanding Deep Neural Networks
In this work, we propose CLass-Enhanced Attentive Response (CLEAR): an approach to visualize and understand the decisions made by deep neural networks (DNNs) given a specific input.
The Ciona17 Dataset for Semantic Segmentation of Invasive Species in a Marine Aquaculture Environment
An original dataset for semantic segmentation, Ciona17, is introduced, which to the best of the authors' knowledge, is the first dataset of its kind with pixel-level annotations pertaining to invasive species in a marine environment.
Dataset Augmentation in Feature Space
Our main insight is to perform the transformation not in input space, but in a learned feature space.
An Integrated Simulator and Dataset that Combines Grasping and Vision for Deep Learning
Deep learning is an established framework for learning hierarchical data representations.
Modeling Grasp Motor Imagery through Deep Conditional Generative Models
Grasping is a complex process involving knowledge of the object, the surroundings, and of oneself.
Understanding Anatomy Classification Through Attentive Response Maps
One of the main challenges for broad adoption of deep learning based models such as convolutional neural networks (CNN), is the lack of understanding of their decisions.
Learning a metric for class-conditional KNN
To extend its applicability outside of image-based domains, we propose to learn a metric which captures perceptual similarity.
Theano-MPI: a Theano-based Distributed Training Framework
We develop a scalable and extendable training framework that can utilize GPUs across nodes in a cluster and accelerate the training of deep learning models based on data parallelism.
Deep Learning on FPGAs: Past, Present, and Future
The rapid growth of data size and accessibility in recent years has instigated a shift of philosophy in algorithm design for artificial intelligence.
ModDrop: adaptive multi-modal gesture recognition
We present a method for gesture detection and localisation based on multi-scale and multi-modal deep learning.
Scoring and Classifying with Gated Auto-encoders
In this work, we apply a dynamical systems view to GAEs, deriving a scoring function, and drawing connections to Restricted Boltzmann Machines.
Neural Network Regularization via Robust Weight Factorization
Regularization is essential when training large neural networks.
Understanding Minimum Probability Flow for RBMs Under Various Kinds of Dynamics
Here we propose a more general form for the sampling dynamics in MPF, and explore the consequences of different choices for these dynamics for training RBMs.
"Mental Rotation" by Optimizing Transforming Distance
The human visual system is able to recognize objects despite transformations that can drastically alter their appearance.
Learning Human Pose Estimation Features with Convolutional Networks
This paper introduces a new architecture for human pose estimation using a multi- layer convolutional network architecture and a modified learning technique that learns low-level features and higher-level weak spatial models.
Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines
We present a type of Temporal Restricted Boltzmann Machine that defines a probability distribution over an output sequence conditional on an input sequence.
Pose-Sensitive Embedding by Nonlinear NCA Regression
We apply our method to challenging real-world data and show that it can generalize beyond hand localization to infer a more general notion of body pose.
The Recurrent Temporal Restricted Boltzmann Machine
The Temporal Restricted Boltzmann Machine (TRBM) is a probabilistic model for sequences that is able to successfully model (i. e., generate nice-looking samples of) several very high dimensional sequences, such as motion capture data and the pixels of low resolution videos of balls bouncing in a box.
