Search Results for author:
Found 110 papers, 34 papers with code
Language Model Guided Interpretable Video Action Reasoning
Extensive experiments on two complex video action datasets, Charades & CAD-120, validates the improved performance and interpretability of our LaIAR framework.
Towards Temporally Consistent Referring Video Object Segmentation
Referring Video Object Segmentation (R-VOS) methods face challenges in maintaining consistent object segmentation due to temporal context variability and the presence of other visually similar objects.
Ranked #3 on
Referring Video Object Segmentation
on Refer-YouTube-VOS
(using extra training data)
Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation
We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from these saliency maps and the significant inter-task correlation between saliency detection and semantic segmentation.
Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models
To address these deficiencies, we introduce the Box-it-to-Bind-it (B2B) module - a novel, training-free approach for improving spatial control and semantic accuracy in text-to-image (T2I) diffusion models.
Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review
This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers.
Flowmind2Digital: The First Comprehensive Flowmind Recognition and Conversion Approach
Automated conversion methods are essential to overcome manual conversion challenges.
RFI Detection with Spiking Neural Networks
This work demonstrates the viability of SNNs as a promising avenue for machine-learning-based RFI detection in radio telescopes by establishing a minimal performance baseline on traditional and nascent satellite-based RFI sources and is the first work to our knowledge to apply SNNs in astronomy.
Ranked #1 on
Semantic Segmentation
on LOFAR RFI Detection
HOMOE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts
In our paper, we propose a novel framework that for the first time combines the Modern Hopfield Network with a Mixture of Experts (HOMOE) to classify the compositions of previously unseen objects.
Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art
Transformers have rapidly gained popularity in computer vision, especially in the field of object recognition and detection.
Multitask Deep Learning for Accurate Risk Stratification and Prediction of Next Steps for Coronary CT Angiography Patients
Diagnostic investigation has an important role in risk stratification and clinical decision making of patients with suspected and documented Coronary Artery Disease (CAD).
MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation
Building upon the observation that the attended regions of the one-class token in the standard vision transformer can contribute to a class-agnostic localization map, we explore the potential of the transformer model to capture class-specific attention for class-discriminative object localization by learning multiple class tokens.
Object Localization
Weakly supervised Semantic Segmentation
+1
Spectrum-guided Multi-granularity Referring Video Object Segmentation
To address the drift problem, we propose a Spectrum-guided Multi-granularity (SgMg) approach, which performs direct segmentation on the encoded features and employs visual details to further optimize the masks.
Ranked #1 on
Referring Expression Segmentation
on J-HMDB
(using extra training data)
A Bibliometric Review of Neuromorphic Computing and Spiking Neural Networks
In particular, spiking neural networks hold the potential to advance artificial intelligence as the basis of third-generation neural networks.
Analysis and Evaluation of Explainable Artificial Intelligence on Suicide Risk Assessment
This study investigates the effectiveness of Explainable Artificial Intelligence (XAI) techniques in predicting suicide risks and identifying the dominant causes for such behaviours.
Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization
Weakly supervised dense object localization (WSDOL) relies generally on Class Activation Mapping (CAM), which exploits the correlation between the class weights of the image classifier and the pixel-level features.
VAPCNet: Viewpoint-Aware 3D Point Cloud Completion
In this paper, we thus propose an unsupervised viewpoint representation learning scheme for 3D point cloud completion without explicit viewpoint estimation.
Utilising physics-guided deep learning to overcome data scarcity
Deep learning (DL) relies heavily on data, and the quality of data influences its performance significantly.
3D Brain and Heart Volume Generative Models: A Survey
Generative models such as generative adversarial networks and autoencoders have gained a great deal of attention in the medical field due to their excellent data generation capability.
Active-Passive SimStereo -- Benchmarking the Cross-Generalization Capabilities of Deep Learning-based Stereo Methods
In this paper, we propose the Active-Passive SimStereo dataset and a corresponding benchmark to evaluate the performance gap between passive and active stereo images for stereo matching algorithms.
Bayesian Learning for Disparity Map Refinement for Semi-Dense Active Stereo Vision
A major focus of recent developments in stereo vision has been on how to obtain accurate dense disparity maps in passive stereo vision.
Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images
Novel 3D network architectures are proposed for both the generator and discriminator of the GAN model to significantly reduce the number of parameters while maintaining the quality of image generation.
A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance
Small object detection (SOD) in optical images and videos is a challenging problem that even state-of-the-art generic object detection methods fail to accurately localize and identify such objects.
Spatial-temporal Analysis for Automated Concrete Workability Estimation
Concrete workability measure is mostly determined based on subjective assessment of a certified assessor with visual inspections.
Region Aware Video Object Segmentation with Deep Motion Modeling
Current semi-supervised video object segmentation (VOS) methods usually leverage the entire features of one frame to predict object masks and update memory.
Ranked #16 on
Semi-Supervised Video Object Segmentation
on DAVIS 2017 (val)
(using extra training data)
Pseudo-Pair based Self-Similarity Learning for Unsupervised Person Re-identification
In this paper, we present a pseudo-pair based self-similarity learning approach for unsupervised person re-ID without human annotations.
Learning Resolution-Adaptive Representations for Cross-Resolution Person Re-Identification
It is a challenging and practical problem since the query images often suffer from resolution degradation due to the different capturing conditions from real-world cameras.
Jacobian Norm with Selective Input Gradient Regularization for Improved and Interpretable Adversarial Defense
Moreover, adversarially training a defense model in general cannot produce interpretable predictions towards the inputs with perturbations, whilst a highly interpretable robust model is required by different domain experts to understand the behaviour of a DNN.
Spiking Neural Networks for Frame-based and Event-based Single Object Localization
Spiking neural networks have shown much promise as an energy-efficient alternative to artificial neural networks.
CrossFormer: Cross Spatio-Temporal Transformer for 3D Human Pose Estimation
3D human pose estimation can be handled by encoding the geometric dependencies between the body parts and enforcing the kinematic constraints.
Ranked #24 on
3D Human Pose Estimation
on Human3.6M
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation
To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens.
Unsupervised Learning on 3D Point Clouds by Clustering and Contrasting
This paper proposes a general unsupervised approach, named \textbf{ConClu}, to perform the learning of point-wise and global features by jointly leveraging point-level clustering and instance-level contrasting.
Performance of multilabel machine learning models and risk stratification schemas for predicting stroke and bleeding risk in patients with non-valvular atrial fibrillation
Conclusions Multilabel ML models can outperform clinical risk stratification scores for predicting the risk of major bleeding and death in non-valvular AF patients.
Spatio-Temporal Graph Representation Learning for Fraudster Group Detection
Then we use an RNN on the spatial relations to predict the spatio-temporal relations of reviewers in the group.
A Novel Incremental Learning Driven Instance Segmentation Framework to Recognize Highly Cluttered Instances of the Contraband Items
Screening cluttered and occluded contraband items from baggage X-ray scans is a cumbersome task even for the expert security staff.
Scene Graph Generation: A Comprehensive Survey
In this paper, we provide a comprehensive survey of recent achievements in this field brought about by deep learning techniques.
COTReg:Coupled Optimal Transport based Point Cloud Registration
Generating a set of high-quality correspondences or matches is one of the most critical steps in point cloud registration.
Explainable Artificial Intelligence for Pharmacovigilance: What Features Are Important When Predicting Adverse Outcomes?
Using XAI, we quantified the contribution that specific drugs had on these ACS predictions, thus creating an XAI-based technique for pharmacovigilance monitoring, using ACS as an example of the adverse outcome to detect.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
Generalized Closed-form Formulae for Feature-based Subpixel Alignment in Patch-based Matching
In this paper, we show that closed form formulae for subpixel disparity computation for the case of one dimensional matching, e. g., in the case of rectified stereo images where the search space is of one dimension, exists when using the standard NCC, SSD and SAD cost functions.
Social Fraud Detection Review: Methods, Challenges and Analysis
Many studies proposed approaches based on user behaviors and review text to address the challenges of fraud detection.
Training Spiking Neural Networks Using Lessons From Deep Learning
This paper serves as a tutorial and perspective showing how to apply the lessons learnt from several decades of research in deep learning, gradient descent, backpropagation and neuroscience to biologically plausible spiking neural neural networks.
Deep Bayesian Image Set Classification: A Defence Approach against Adversarial Attacks
In practice, the vulnerability of deep learning systems against carefully perturbed images, known as adversarial examples, poses a dire security threat in the physical world applications.
Tensor Pooling Driven Instance Segmentation Framework for Baggage Threat Recognition
Furthermore, to the best of our knowledge, this is the first contour instance segmentation framework that leverages multi-scale information to recognize cluttered and concealed contraband data from the colored and grayscale security X-ray imagery.
Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation
We propose a self-supervised spatio-temporal matching method, coined Motion-Aware Mask Propagation (MAMP), for video object segmentation.
Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation
Motivated by the significant inter-task correlation, we propose a novel weakly supervised multi-task framework termed as AuxSegNet, to leverage saliency detection and multi-label image classification as auxiliary tasks to improve the primary task of semantic segmentation using only image-level ground-truth labels.
Unsupervised Anomaly Instance Segmentation for Baggage Threat Recognition
Identifying potential threats concealed within the baggage is of prime concern for the security staff.
A Systematic Collection of Medical Image Datasets for Deep Learning
Thus, as comprehensive as possible, this paper provides a collection of medical image datasets with their associated challenges for deep learning research.
CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency
Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input.
Attack to Fool and Explain Deep Networks
Exploring this phenomenon further, we alter the `adversarial' objective of our attack to use it as a tool to `explain' deep visual representation.
q-RBFNN:A Quantum Calculus-based RBF Neural Network
In this research a novel stochastic gradient descent based learning approach for the radial basis function neural networks (RBFNN) is proposed.
HIN-RNN: A Graph Representation Learning Neural Network for Fraudster Group Detection With No Handcrafted Features
Social reviews are indispensable resources for modern consumers' decision making.
4D Atlas: Statistical Analysis of the Spatiotemporal Variability in Longitudinal 3D Shape Data
With this formulation, the statistical analysis of 4D surfaces can be cast as the problem of analyzing trajectories embedded in a nonlinear Riemannian manifold.
Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction
To handle occlusions, solutions based on the reconstruction of the occluded part of the face have been proposed.
Dynamic Facial Expression Recognition
Facial Expression Recognition
+2
WEmbSim: A Simple yet Effective Metric for Image Captioning
The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements.
LCEval: Learned Composite Metric for Caption Evaluation
Automatic evaluation metrics hold a fundamental importance in the development and fine-grained analysis of captioning systems.
SubICap: Towards Subword-informed Image Captioning
In this work we address this common limitation of IC systems in dealing with rare words in the corpora.
Human Action Recognition from Various Data Modalities: A Review
Human Action Recognition (HAR) aims to understand human behavior and assign a label to each action.
Imputation of Missing Data with Class Imbalance using Conditional Generative Adversarial Networks
State-of-the-art imputation approaches, such as Generative Adversarial Imputation Nets (GAIN), model the distribution of observed data to approximate the missing values.
A Practical Tutorial on Graph Neural Networks
Graph neural networks (GNNs) have recently grown in popularity in the field of artificial intelligence (AI) due to their unique ability to ingest relatively unstructured data types as input data.
Trainable Structure Tensors for Autonomous Baggage Threat Detection Under Extreme Occlusion
Detecting baggage threats is one of the most difficult tasks, even for expert officers.
Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users
Modern deep learning methods constitute incredibly powerful tools to tackle a myriad of challenging problems.
Multi-Kernel Fusion for RBF Neural Networks
A simple yet effective architectural design of radial basis function neural networks (RBFNN) makes them amongst the most popular conventional neural networks.
Orthogonal Deep Models As Defense Against Black-Box Attacks
On the other, deep learning has also been found vulnerable to adversarial attacks, which calls for new techniques to defend deep models against these attacks.
ScoreGAN: A Fraud Review Detector based on Multi Task Learning of Regulated GAN with Data Augmentation
However, the lack of trusted labeled data has limited the performance of the current solutions in detecting fraud reviews.
DFraud3- Multi-Component Fraud Detection freeof Cold-start
In this research, instead of focusing only on one component, detecting either fraud reviews or fraud users (fraudsters), vector representations are learnt for each component, enabling multi-component classification.
A Survey on Deep Learning Techniques for Stereo-based Depth Estimation
Motivated by their growing success in solving various 2D and 3D vision problems, deep learning for stereo-based depth estimation has attracted growing interest from the community, with more than 150 papers published in this area between 2014 and 2019.
Ranked #1 on
Monocular Depth Estimation
on Make3D
(RMSE metric)
Unconstrained Matching of 2D and 3D Descriptors for 6-DOF Pose Estimation
We use this concept to directly localize images in a 3D point cloud.
Cascaded Structure Tensor Framework for Robust Identification of Heavily Occluded Baggage Items from X-ray Scans
In the last two decades, baggage scanning has globally become one of the prime aviation security concerns.
Efficient Scene Text Detection with Textual Attention Tower
Scene text detection has received attention for years and achieved an impressive performance across various benchmarks.
Structure-Feature based Graph Self-adaptive Pooling
To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes.
Deep Learning for 3D Point Clouds: A Survey
To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds.
Cascaded Structure Tensor Framework for Robust Identification of Heavily Occluded Baggage Items from Multi-Vendor X-ray Scans
In the last two decades, luggage scanning has globally become one of the prime aviation security concerns.
Biometrics Recognition Using Deep Learning: A Survey
Deep learning-based models have been very successful in achieving state-of-the-art results in many of the computer vision, speech recognition, and natural language processing tasks in the last few years.
Learning Latent Global Network for Skeleton-based Action Prediction
Human actions represented with 3D skeleton sequences are robust to clustered backgrounds and illumination changes.
Ranked #4 on
Skeleton Based Action Recognition
on SYSU 3D
RGB-D image-based Object Detection: from Traditional Methods to Deep Learning Techniques
Deep learning techniques, coupled with the availability of large training datasets, have now revolutionized the field of computer vision, including RGB-D object detection, achieving an unprecedented level of performance.
Automatic Hierarchical Classification of Kelps using Deep Residual Features
Across the globe, remote image data is rapidly being collected for the assessment of benthic communities from shallow to extremely deep waters on continental slopes to the abyssal seas.
Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
Given this new era of rapid evolution, this article provides a comprehensive survey of the recent developments in this field.
Direct Image to Point Cloud Descriptors Matching for 6-DOF Camera Localization in Dense 3D Point Cloud
We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds.
Label Universal Targeted Attack
We introduce Label Universal Targeted Attack (LUTA) that makes a deep model predict a label of attacker's choice for `any' sample of a given source class with high probability.
A Novel Adaptive Kernel for the RBF Neural Networks
In this paper, we propose a novel adaptive kernel for the radial basis function (RBF) neural networks.
Improving Image-Based Localization with Deep Learning: The Impact of the Loss Function
This work investigates the impact of the loss function on the performance of Neural Networks, in the context of a monocular, RGB-only, image localization task.
Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study
Optical flow techniques are becoming increasingly performant and robust when estimating motion in a scene, but their performance has yet to be proven in the area of facial expression recognition.
Imaging and Classification Techniques for Seagrass Mapping and Monitoring: A Comprehensive Survey
The detection and mapping of underwater vegetation, especially seagrass has drawn the attention of the research community as early as the nineteen eighties.
Attention in Convolutional LSTM for Gesture Recognition
On this basis, a new variant of LSTM is derived, in which the convolutional structures are only embedded into the input-to-state transition of LSTM.
RAFP-Pred: Robust Prediction of Antifreeze Proteins using Localized Analysis of n-Peptide Compositions
In extreme cold weather, living organisms produce Antifreeze Proteins (AFPs) to counter the otherwise lethal intracellular formation of ice.
NNEval: Neural Network based Evaluation Metric for Image Captioning
The automatic evaluation of image descriptions is an intricate task, and it is highly important in the development and fine-grained analysis of captioning systems.
DataDeps.jl: Repeatable Data Setup for Replicable Data Science
We present DataDeps. jl: a julia package for the reproducible handling of static datasets to enhance the repeatability of scripts used in the data and computational sciences.
Software Engineering
NovelPerspective: Identifying Point of View Characters
Our tool detects the main character that each section is from the POV of, and allows the user to generate a new ebook with only those sections.
Learning-based Composite Metrics for Improved Caption Evaluation
The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency.
Real Time Surveillance for Low Resolution and Limited-Data Scenarios: An Image Set Classification Approach
Class specific gallery subspaces are used to estimate regression models for each image of the test image set.
Learning clip representations for skeleton-based 3d action recognition
This paper presents a new representation of skeleton sequences for 3D action recognition.
Ranked #60 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Exploiting Layerwise Convexity of Rectifier Networks with Sign Constrained Weights
By introducing sign constraints on the weights, this paper proposes sign constrained rectifier networks (SCRNs), whose training can be solved efficiently by the well known majorization-minimization (MM) algorithms.
Learning Action Recognition Model From Depth and Skeleton Videos
Depth sensors open up possibilities of dealing with the human action recognition problem by providing 3D human skeleton data and depth images of the scene.
Learning of Colors from Color Names: Distribution and Point Estimation
Color names are often made up of multiple words.
On the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries
To show the superior compressive power of deep rectifier networks over shallow rectifier networks, we prove that the maximum boundary resolution of a single hidden layer rectifier network classifier grows exponentially with the number of units when this number is smaller than the dimension of the patterns.
Skeletonnet: Mining deep part features for 3-d action recognition
Given a skeleton sequence, the spatial structure of the skeleton joints in each frame and the temporal information between multiple frames are two important factors for action recognition.
Ranked #106 on
Skeleton Based Action Recognition
on NTU RGB+D
From Deep to Shallow: Transformations of Deep Rectifier Networks
Similarly, for a residual net and a conventional rectifier net with the same structure except for the skip connections in the residual net, the corresponding single hidden layer representation of the residual net is much more complex than the corresponding single hidden layer representation of the conventional net.
A New Representation of Skeleton Sequences for 3D Action Recognition
This paper presents a new method for 3D action recognition with skeleton sequences (i. e., 3D trajectories of human skeleton joints).
Ranked #65 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Efficient Image Set Classification using Linear Regression based Image Reconstruction
We estimate regression models for each test image using the class specific gallery subspaces.
ResFeats: Residual Network Based Features for Image Classification
Deep residual networks have recently emerged as the state-of-the-art architecture in image segmentation and object detection.
Leveraging Structural Context Models and Ranking Score Fusion for Human Interaction Prediction
The structural models, including the spatial and the temporal models, are learned with Long Short Term Memory (LSTM) networks to capture the dependency of the global and local contexts of each RGB frame and each optical flow image, respectively.
Learning deep structured network for weakly supervised change detection
We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework.
Listening With Your Eyes: Towards a Practical Visual Speech Recognition System Using Deep Boltzmann Machines
This paper presents a novel feature learning method for visual speech recognition using Deep Boltzmann Machines (DBM).
Contractive Rectifier Networks for Nonlinear Maximum Margin Classification
The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer.
Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data
Class imbalance is a common problem in the case of real-world object detection and classification tasks.
A Spatial Layout and Scale Invariant Feature Representation for Indoor Scene Classification
This paper introduces a new learnable feature descriptor called "spatial layout and scale invariant convolutional activations" to deal with these challenges.
A Discriminative Representation of Convolutional Features for Indoor Scene Recognition
To this end, we introduce a new large-scale dataset of 1300 object categories which are commonly present in indoor scenes.
Separating Objects and Clutter in Indoor Scenes
Objects' spatial layout estimation and clutter identification are two important tasks to understand indoor scenes.
Learning Non-Linear Reconstruction Models for Image Set Classification
We propose a deep learning framework for image set classification with application to face recognition.
Automatic Feature Learning for Robust Shadow Detection
We present a practical framework to automatically detect shadows in real world scenes from a single photograph.
Rotational Projection Statistics for 3D Local Surface Description and Object Recognition
The performance of the proposed LRF, RoPS descriptor and object recognition algorithm was rigorously tested on a number of popular and publicly available datasets.
