Search Results for author:
Found 95 papers, 48 papers with code
TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition
We demonstrate that using both RNNs (using LSTMs) and Temporal-ConvNets on spatiotemporal feature matrices are able to exploit spatiotemporal dynamics to improve the overall performance.
Ranked #54 on
Action Recognition
on UCF101
Grounded Objects and Interactions for Video Captioning
We address the problem of video captioning by grounding language generation on object interactions in the video.
Attend and Interact: Higher-Order Object Interactions for Video Understanding
Human actions often involve complex interactions across several inter-related objects in the scene.
CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition
We benchmark the performance of existing solutions in real-world scenarios and analyze the performance variation with respect to challenging conditions.
Perceptual Image Quality Assessment through Spectral Analysis of Error Representations
To overcome these shortcomings, we introduce an image quality assessment algorithm based on the Spectral Understanding of Multi-scale and Multi-channel Error Representations, denoted as SUMMER.
Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions]
Robust and reliable traffic sign detection is necessary to bring autonomous vehicles onto our roads.
CSV: Image Quality Assessment Based on Color, Structure, and Visual System
This paper presents a full-reference image quality estimator based on color, structure, and visual system characteristics denoted as CSV.
CURE-OR: Challenging Unreal and Real Environments for Object Recognition
Moreover, we investigate the relationship between object recognition and image quality and show that objective quality algorithms can estimate recognition performance under certain photometric challenging conditions.
HeartBEAT: Heart Beat Estimation through Adaptive Tracking
In this paper, we propose an algorithm denoted as HeartBEAT that tracks heart rate from wrist-type photoplethysmography (PPG) signals and simultaneously recorded three-axis acceleration data.
Content-adaptive non-parametric texture similarity measure
In this paper, we introduce a non-parametric texture similarity measure based on the singular value decomposition of the curvelet coefficients followed by a content-based truncation of the singular values.
Image and Video Processing
ReSIFT: Reliability-Weighted SIFT-based Image Quality Assessment
In terms of the Pearson and the Spearman correlation, ReSIFT is the best performing quality estimator in the overall databases.
BLeSS: Bio-inspired Low-level Spatiochromatic Similarity Assisted Image Quality Assessment
Moreover, BleSS significantly enhances the performance of SR-SIM and FSIM in the full TID 2013 database.
PerSIM: Multi-resolution Image Quality Assessment in the Perceptually Uniform Color Domain
An average observer perceives the world in color instead of black and white.
A Comparative Study of Computational Aesthetics
We show that generic descriptors can perform as well as state of the art hand-crafted aesthetics models that use global features.
Boosting in Image Quality Assessment
In addition to support vector machines that are commonly used in the multi-method fusion, we propose using neural networks in the boosting.
A Comparative Study of Quality and Content-Based Spatial Pooling Strategies in Image Quality Assessment
In this work, we compare the state of the art quality and content-based spatial pooling strategies and show that although features are the key in any image quality assessment, pooling also matters.
MS-UNIQUE: Multi-model and Sharpness-weighted Unsupervised Image Quality Estimation
We use multiple linear decoders to capture different abstraction levels of the image patches.
Generating Adaptive and Robust Filter Sets Using an Unsupervised Learning Framework
While assessing image quality, the filters need to capture perceptual differences based on dissimilarities between a reference image and its distorted version.
Image Quality Assessment and Color Difference
In this work, we combine these approaches by extending CIEDE2000 formula with perceptual color difference to assess image quality.
Power of Tempospatially Unified Spectral Density for Perceptual Video Quality Assessment
This is a full-reference tempospatial approach that considers both temporal and spatial PSD characteristics.
A comparative study of texture attributes for characterizing subsurface structures in seismic volumes
We focus on spatial attributes in this study and examine them in a new application for seismic interpretation, i. e., seismic volume labeling.
The role of visual saliency in the automation of seismic interpretation
In this paper, we propose a workflow based on SalSi for the detection and delineation of geological structures such as salt domes.
Unsupervised uncertainty estimation using spatiotemporal cues in video saliency detection
Based on the study, we then develop an algorithm that estimates a pixel-wise uncertainty map that reflects our confidence in the associated computational saliency map by relating a pixel's saliency to the saliency of its neighbors.
SalSi: A new seismic attribute for salt dome detection
In this paper, we propose a saliency-based attribute, SalSi, to detect salt dome bodies within seismic volumes.
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
The Vision-and-Language Navigation (VLN) task entails an agent following navigational instruction in photo-realistic unknown environments.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
Natural Language Visual Grounding
Vision and Language Navigation
+2
A Machine Learning Benchmark for Facies Classification
In addition to making the dataset and the code publicly available, this work helps advance research in this area by creating an objective benchmark for comparing the results of different machine learning approaches for facies classification.
Multiresolution Analysis and Learning for Computational Seismic Interpretation
Moreover, directional multiresolution attributes, such as the curvelet transform, are more effective than the non-directional attributes in distinguishing different subsurface structures in large seismic datasets, and can greatly help the interpretation process.
Image and Video Processing Geophysics
Petrophysical Property Estimation from Seismic Data Using Recurrent Neural Networks
Recent advances in machine learning have shown promising results for recurrent neural networks (RNN) in modeling complex sequential data such as videos and speech signals.
Saliency detection for seismic applications using multi-dimensional spectral projections and directional comparisons
In this paper, we propose a novel approach for saliency detection for seismic applications using 3D-FFT local spectra and multi-dimensional plane projections.
Understanding spatial correlation in eye-fixation maps for visual attention in videos
In this paper, we present an analysis of recorded eye-fixation data from human subjects viewing video sequences.
Characterization of migrated seismic volumes using texture attributes: a comparative study
It is our hope that this comparative study will help acquaint the seismic interpretation community with the many available powerful image texture analysis techniques, providing more alternative attributes for their seismic exploration.
Semantically Interpretable and Controllable Filter Sets
In this paper, we generate and control semantically interpretable filters that are directly learned from natural images in an unsupervised fashion.
Object Recognition under Multifarious Conditions: A Reliability Analysis and A Feature Similarity-based Performance Estimation
Experimental results show that deep learning-based image representations can estimate the recognition performance variation with a Spearman's rank-order correlation of 0. 94 under multifarious acquisition conditions.
Challenging Environments for Traffic Sign Detection: Reliability Assessment under Inclement Conditions
Experimental results show that benchmarked algorithms are highly sensitive to tested challenging conditions, which result in an average performance drop of 0. 17 in terms of precision and a performance drop of 0. 28 in recall under severe conditions.
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
The Regretful Navigation Agent for Vision-and-Language Navigation
As deep learning continues to make progress for challenging perception tasks, there is increased interest in combining vision, language, and decision-making.
Structure Label Prediction Using Similarity-Based Retrieval and Weakly-Supervised Label Mapping
By having an interpreter select a very small number of exemplar images for every class of subsurface structures, we use a novel similarity-based retrieval technique to extract thousands of images that contain similar subsurface structures from the seismic volume.
Automated Pupillary Light Reflex Test on a Portable Platform
In this paper, we introduce a portable eye imaging device denoted as lab-on-a-headset, which can automatically perform a swinging flashlight test.
Implicit Background Estimation for Semantic Segmentation
Scene understanding and semantic segmentation are at the core of many computer vision tasks, many of which, involve interacting with humans in potentially dangerous ways.
Multi-level Texture Encoding and Representation (MuLTER) based on Deep Neural Networks
In this paper, we propose a multi-level texture encoding and representation network (MuLTER) for texture-related applications.
Texture retrieval using periodically extended and adaptive curvelets
Image retrieval is an important problem in the area of multimedia processing.
Temporal Attentive Alignment for Video Domain Adaptation
Finally, we propose Temporal Attentive Adversarial Adaptation Network (TA3N), which explicitly attends to the temporal dynamics using domain discrepancy for more effective domain alignment, achieving state-of-the-art performance on three video DA datasets.
Ranked #1 on
Domain Adaptation
on UCF-to-Olympic
Semi-supervised Learning for Acoustic Impedance Inversion
Then, a neural-network-based inversion model comprising convolutional and recurrent neural layers is used to invert seismic data for acoustic impedance.
Image and Video Processing Signal Processing Geophysics
Learning to Generate Grounded Visual Captions without Localization Supervision
When automatically generating a sentence description for an image or video, it often remains unclear how well the generated caption is grounded, that is whether the model uses the correct image regions to output particular words, or if the model is hallucinating based on priors in the dataset and/or the language model.
Estimation of Acoustic Impedance from Seismic Data using Temporal Convolutional Network
In exploration seismology, seismic inversion refers to the process of inferring physical properties of the subsurface from seismic data.
Temporal Attentive Alignment for Large-Scale Video Domain Adaptation
Finally, we propose Temporal Attentive Adversarial Adaptation Network (TA3N), which explicitly attends to the temporal dynamics using domain discrepancy for more effective domain alignment, achieving state-of-the-art performance on four video DA datasets (e. g. 7. 9% accuracy gain over "Source only" from 73. 9% to 81. 8% on "HMDB --> UCF", and 10. 3% gain on "Kinetics --> Gameplay").
Relative Afferent Pupillary Defect Screening through Transfer Learning
Based on the conducted experiments, proposed algorithm RAPDNet can achieve a sensitivity and a specificity of 90. 6% over 64 test cases in a balanced set, which corresponds to an AUC of 0. 929 in ROC analysis.
Semi-supervised Sequence Modeling for Elastic Impedance Inversion
The proposed workflow uses well-log data to guide the inversion.
Distorted Representation Space Characterization Through Backpropagated Gradients
In this paper, we utilize weight gradients from backpropagation to characterize the representation space learned by deep learning algorithms.
Traffic Sign Detection under Challenging Conditions: A Deeper Look Into Performance Variations and Spectral Characteristics
We investigate the effect of challenging conditions through spectral analysis and show that challenging conditions can lead to distinct magnitude spectrum characteristics.
Leveraging Adversarial Examples to Obtain Robust Second-Order Representations
Such a positioning scheme is based on a data point’s second-order property.
Characterizing Missing Information in Deep Networks Using Backpropagated Gradients
To complement the learned information from activation-based representation, we propose utilizing a gradient-based representation that explicitly focuses on missing information.
Texture Classification using Block Intensity and Gradient Difference (BIGD) Descriptor
In this paper, we present an efficient and distinctive local descriptor, namely block intensity and gradient difference (BIGD).
Action Segmentation with Joint Self-Supervised Temporal Domain Adaptation
Despite the recent progress of fully-supervised action segmentation techniques, the performance is still not fully satisfactory.
Ranked #11 on
Action Segmentation
on GTEA
Fabric Surface Characterization: Assessment of Deep Learning-based Texture Representations Using a Challenging Dataset
We formulate the problem as a very fine-grained texture classification problem, and study how deep learning-based texture representation techniques can help tackle the task.
Spatiotemporal Modeling of Seismic Images for Acoustic Impedance Estimation
We empirically compare our proposed workflow with some other sequence modeling-based neural networks that model seismic data only temporally.
Backpropagated Gradient Representations for Anomaly Detection
Anomalies require more drastic model updates to fully represent them compared to normal data.
Contrastive Explanations in Neural Networks
Current modes of visual explanations answer questions of the form $`Why \text{ } P?'$.
Implicit Saliency in Deep Neural Networks
In this paper, we show that existing recognition and localization deep architectures, that have not been exposed to eye tracking data or any saliency datasets, are capable of predicting the human visual saliency.
Novelty Detection Through Model-Based Characterization of Neural Networks
To articulate the significance of the model perspective in novelty detection, we utilize backpropagated gradients.
Gradients as a Measure of Uncertainty in Neural Networks
We demonstrate the effectiveness of gradients as a measure of model uncertainty in applications of detecting unfamiliar inputs, including out-of-distribution and corrupted samples.
Robustness and Overfitting Behavior of Implicit Background Models
In this paper, we examine the overfitting behavior of image classification models modified with Implicit Background Estimation (SCrIBE), which transforms them into weakly supervised segmentation models that provide spatial domain visualizations without affecting performance.
On the Structures of Representation for the Robustness of Semantic Segmentation to Input Corruption
Semantic segmentation is a scene understanding task at the heart of safety-critical applications where robustness to corrupted inputs is essential.
Self-Supervised Annotation of Seismic Images using Latent Space Factorization
Annotating seismic data is expensive, laborious and subjective due to the number of years required for seismic interpreters to attain proficiency in interpretation.
Extracting Causal Visual Features for Limited label Classification
Neural networks trained to classify images do so by identifying features that allow them to distinguish between classes.
Contrastive Reasoning in Neural Networks
In this paper, we formalize the structure of contrastive reasoning and propose a methodology to extract a neural network's notion of contrast.
Action Segmentation with Mixed Temporal Domain Adaptation
The main progress for action segmentation comes from densely-annotated data for fully-supervised learning.
Ranked #14 on
Action Segmentation
on Breakfast
Self-Supervised Delineation of Geological Structures using Orthogonal Latent Space Projection
Images from the selected clusters are used to train the encoder-decoder network.
A novel attention model for salient structure detection in seismic volumes
Subsequently, a novel directional center-surround attention model is proposed to incorporate directional comparisons around each voxel for saliency detection within each projected dimension.
Explanatory Paradigms in Neural Networks
With $P$ as the prediction from a neural network, these questions are `Why P?
A Gating Model for Bias Calibration in Generalized Zero-shot Learning
Also, the two-stream autoencoder works as a unified framework for the gating model and the unseen expert, which makes the proposed method computationally efficient.
Open-Set Recognition with Gradient-Based Representations
Neural networks for image classification tasks assume that any given image during inference belongs to one of the training classes.
Gradient-Based Adversarial and Out-of-Distribution Detection
We propose to utilize gradients for detecting adversarial and out-of-distribution samples.
Volumetric Supervised Contrastive Learning for Seismic Semantic Segmentation
In seismic interpretation, pixel-level labels of various rock structures can be time-consuming and expensive to obtain.
DECAL: DEployable Clinical Active Learning
Hence, active learning techniques that are developed for natural images are insufficient for handling medical data.
Patient Aware Active Learning for Fine-Grained OCT Classification
For this purpose, we propose a framework that incorporates clinical insights into the sample selection process of active learning that can be incorporated with existing algorithms.
Introspective Learning : A Two-Stage Approach for Inference in Neural Networks
Finally, we ground the proposed machine introspection to human introspection for the application of image quality assessment.
OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics
The dataset consists of 1268 near-IR fundus images each with at least 49 OCT scans, and 16 biomarkers, along with 4 clinical labels and a disease diagnosis of DR or DME.
Clinical Contrastive Learning for Biomarker Detection
This is accomplished by leveraging the larger amount of clinical data as pseudo-labels for our data without biomarker labels in order to choose positive and negative instances for training a backbone network with a supervised contrastive loss.
On the Ramifications of Human Label Uncertainty
Our evaluation of existing uncertainty estimation algorithms, with the presence of HLU, indicates the limitations of existing uncertainty metrics and algorithms themselves in response to HLU.
Man-recon: manifold learning for reconstruction with deep autoencoder for smart seismic interpretation
Deep learning can extract rich data representations if provided sufficient quantities of labeled training data.
Explainable Machine Learning for Hydrocarbon Prospect Risking
On a dataset of well outcomes and corresponding geophysical attribute data, we show how LIME can induce trust in model's decisions by revealing the decision-making process to be aligned to domain knowledge.
Explaining Deep Models through Forgettable Learning Dynamics
Inspired by this phenomenon, we present a novel segmentation method that actively uses this information to alter the data representation within the model by increasing the variety of difficult regions.
Forgetful Active Learning with Switch Events: Efficient Sampling for Out-of-Distribution Data
However, existing strategies directly base the data selection on the data representation of the unlabeled data which is random for OOD samples by definition.
Learning Trajectory-Conditioned Relations to Predict Pedestrian Crossing Behavior
In smart transportation, intelligent systems avoid potential collisions by predicting the intent of traffic agents, especially pedestrians.
Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networks
This paper conjectures and validates a framework that allows for action during inference in supervised neural networks.
Gaussian Switch Sampling: A Second Order Approach to Active Learning
To alleviate this issue, we propose a grounded second-order definition of information content and sample importance within the context of active learning.
Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation
In recent years, deep neural networks have significantly impacted the seismic interpretation process.
Probing the Purview of Neural Networks via Gradient Analysis
We define purview as the additional capacity necessary to characterize inference samples that differ from the training data.
Exploiting the Distortion-Semantic Interaction in Fisheye Data
In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data modality.
Clinically Labeled Contrastive Learning for OCT Biomarker Classification
We exploit this relationship by using the clinical data as pseudo-labels for our data without biomarker labels in order to choose positive and negative instances for training a backbone network with a supervised contrastive loss.
Clinical Trial Active Learning
There exists two types of clinical trials: retrospective and prospective.
FOCAL: A Cost-Aware Video Dataset for Active Learning
In this paper, we introduce the FOCAL (Ford-OLIVES Collaboration on Active Learning) dataset which enables the study of the impact of annotation-cost within a video active learning setting.
Perceptual Quality-based Model Training under Annotator Label Uncertainty
Annotator label uncertainty manifests in variations of labeling quality.
TrajPRed: Trajectory Prediction with Region-based Relation Learning
We integrate multi-goal estimation and region-based relation learning to model the two stimuli, social interactions, and stochastic goals, in a prediction framework.
