Search Results for author: John K. Tsotsos

Found 47 papers, 11 papers with code

Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation

no code implementations13 Oct 2023 Iuliia Kotseruba, John K. Tsotsos

Therefore, to enable analysis and modeling of these factors for drivers' gaze prediction, we propose the following: 1) address some shortcomings of the popular DR(eye)VE dataset and extend it with per-frame annotations for driving task and context; 2) benchmark a number of baseline and SOTA models for saliency and driver gaze prediction and analyze them w. r. t.

Gaze Prediction

The Psychophysics of Human Three-Dimensional Active Visuospatial Problem-Solving

no code implementations19 Jun 2023 Markus D. Solbach, John K. Tsotsos

How well the visual system performs while interacting with the visual environment and how vision is used in the real world have not been well studied, especially in humans.

Self-attention in Vision Transformers Performs Perceptual Grouping, Not Attention

no code implementations2 Mar 2023 Paria Mehrani, John K. Tsotsos

To answer this question, we revisited the attention formulation in these models and found that despite the name, computationally, these models perform a special class of relaxation labeling with similarity grouping effects.

Saliency Detection

Learning a model of shape selectivity in V4 cells reveals shape encoding mechanisms in the brain

no code implementations28 Nov 2021 Paria Mehrani, John K. Tsotsos

We propose a hierarchical model that reveals V1/V2 encodings that are essential components for this transformation to the reported curvature representations in V4.

Active Observer Visual Problem-Solving Methods are Dynamically Hypothesized, Deployed and Tested

no code implementations18 Aug 2021 Markus D. Solbach, John K. Tsotsos

The importance of active observation is striking as is the lack of any learning effect.

Industry and Academic Research in Computer Vision

no code implementations10 Jul 2021 Iuliia Kotseruba, Manos Papagelis, John K. Tsotsos

The results indicate that the distribution of the research topics is similar in industry and academic papers.

Probing the Effect of Selection Bias on Generalization: A Thought Experiment

no code implementations20 May 2021 John K. Tsotsos, Jun Luo

The point of the thought experiment is not to demonstrate problems with all learned systems.

Epidemiology Selection bias

Behavioral Research and Practical Models of Drivers' Attention

1 code implementation12 Apr 2021 Iuliia Kotseruba, John K. Tsotsos

Drivers deal with multiple concurrent tasks, such as keeping the vehicle in the lane, observing and anticipating the actions of other road users, reacting to hazards, and dealing with distractions inside and outside the vehicle.

Blocks World Revisited: The Effect of Self-Occlusion on Classification by Convolutional Neural Networks

no code implementations25 Feb 2021 Markus D. Solbach, John K. Tsotsos

We present baseline evaluations with five well-known classification deep neural networks and show that TEOS poses a significant challenge for all of them.

General Classification Object

Video action recognition for lane-change classification and prediction of surrounding vehicles

no code implementations13 Jan 2021 Mahdi Biparva, David Fernández-Llorca, Rubén Izquierdo-Gonzalo, John K. Tsotsos

Up to four different two-stream-based approaches, that have been successfully applied to address human action recognition, are adapted here by stacking visual cues from forward-looking video cameras to recognize and anticipate lane-changes of target vehicles.

Action Recognition Autonomous Vehicles +2

On the Control of Attentional Processes in Vision

no code implementations5 Jan 2021 John K. Tsotsos, Omar Abid, Iuliia Kotseruba, Markus D. Solbach

The key conclusions of this paper are that an executive controller is necessary for human attentional function in vision, and that there is a 'first principles' computational approach to its understanding that is complementary to the previous approaches that focus on modelling or learning from experimental observations directly.

An Empirical Method to Quantify the Peripheral Performance Degradation in Deep Networks

no code implementations4 Dec 2020 Calden Wloka, John K. Tsotsos

When applying a convolutional kernel to an image, if the output is to remain the same size as the input then some form of padding is required around the image boundary, meaning that for each layer of convolution in a convolutional neural network (CNN), a strip of pixels equal to the half-width of the kernel size is produced with a non-veridical representation.

PESAO: Psychophysical Experimental Setup for Active Observers

no code implementations15 Sep 2020 Markus D. Solbach, John K. Tsotsos

Nonetheless, how exactly active observation occurs in humans so that it can inform the design of active computer vision systems is an open problem.

Two-Stream Networks for Lane-Change Prediction of Surrounding Vehicles

no code implementations25 Aug 2020 David Fernández-Llorca, Mahdi Biparva, Rubén Izquierdo-Gonzalo, John K. Tsotsos

Different sizes of the regions around the vehicles are analyzed, evaluating the importance of the interaction between vehicles and the context information in the performance.

Action Recognition Temporal Action Localization +1

Do Saliency Models Detect Odd-One-Out Targets? New Datasets and Evaluations

2 code implementations13 May 2020 Iuliia Kotseruba, Calden Wloka, Amir Rasouli, John K. Tsotsos

Furthermore, we investigate the effect of training state-of-the-art CNN-based saliency models on these types of stimuli and conclude that the additional training data does not lead to a significant improvement of their ability to find odd-one-out targets.

Odd One Out

Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs

1 code implementation13 May 2020 Amir Rasouli, Iuliia Kotseruba, John K. Tsotsos

To this end, we propose a solution for the problem of pedestrian action anticipation at the point of crossing.

Action Anticipation Autonomous Vehicles

A Possible Reason for why Data-Driven Beats Theory-Driven Computer Vision

no code implementations28 Aug 2019 John K. Tsotsos, Iuliia Kotseruba, Alexander Andreopoulos, Yulong Wu

This reveals a strong mismatch between optimal performance ranges of classical theory-driven algorithms and sensor setting distributions in the common vision datasets, while data-driven models were trained for those datasets.

Fast Visual Object Tracking with Rotated Bounding Boxes

1 code implementation8 Jul 2019 Bao Xin Chen, John K. Tsotsos

In this paper, we demonstrate a novel algorithm that uses ellipse fitting to estimate the bounding box rotation angle and size with the segmentation(mask) on the target for online and real-time visual object tracking.

Object Visual Object Tracking

Multiplicative modulations in hue-selective cells enhance unique hue representation

no code implementations3 Jul 2019 Paria Mehrani, Andrei Mouraviev, John K. Tsotsos

Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that model V4 neurons have the capacity to encode unique hues.

Rapid Visual Categorization is not Guided by Early Salience-Based Selection

no code implementations15 Jan 2019 John K. Tsotsos, Iuliia Kotseruba, Calden Wloka

The current dominant visual processing paradigm in both human and machine research is the feedforward, layered hierarchy of neural-like processing elements.

Early recurrence enables figure border ownership

no code implementations10 Jan 2019 Paria Mehrani, John K. Tsotsos

Various models proposed that feedback from higher ventral areas or lateral connections could provide the required contextual information.


SMILER: Saliency Model Implementation Library for Experimental Research

1 code implementation20 Dec 2018 Calden Wloka, Toni Kunić, Iuliia Kotseruba, Ramin Fahimi, Nicholas Frosst, Neil D. B. Bruce, John K. Tsotsos

The Saliency Model Implementation Library for Experimental Research (SMILER) is a new software package which provides an open, standardized, and extensible framework for maintaining and executing computational saliency models.

The Elephant in the Room

1 code implementation9 Aug 2018 Amir Rosenfeld, Richard Zemel, John K. Tsotsos

We showcase a family of common failures of state-of-the art object detectors.

Object object-detection +2

Attention-based Active Visual Search for Mobile Robots

no code implementations27 Jul 2018 Amir Rasouli, Pablo Lanillos, Gordon Cheng, John K. Tsotsos

In this paper, we propose a new model that actively extracts visual information via visual attention techniques and, in conjunction with a non-myopic decision-making algorithm, leads the robot to search more relevant areas of the environment.

Decision Making

Visual Attention and its Intimate Links to Spatial Cognition

no code implementations29 Jun 2018 John K. Tsotsos, Iuliia Kotseruba, Amir Rasouli, Markus D. Solbach

It is almost universal to regard attention as the facility that permits an agent, human or machine, to give priority processing resources to relevant stimuli while ignoring the irrelevant.

Active Fixation Control to Predict Saccade Sequences

2 code implementations CVPR 2018 Calden Wloka, Iuliia Kotseruba, John K. Tsotsos

However, on static images the emphasis of these models has largely been based on non-ordered prediction of fixations through a saliency map.

Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice

no code implementations30 May 2018 Amir Rasouli, John K. Tsotsos

To make it a reality, autonomous vehicles require the ability to communicate with other road users and understand their intentions.

Autonomous Vehicles

Random Polyhedral Scenes: An Image Generator for Active Vision System Experiments

1 code implementation27 Mar 2018 Markus D. Solbach, Stephen Voland, Jeff Edmonds, John K. Tsotsos

We present a Polyhedral Scene Generator system which creates a random scene based on a few user parameters, renders the scene from random view points and creates a dataset containing the renderings and corresponding annotation files.

Totally Looks Like - How Humans Compare, Compared to Machines

no code implementations5 Mar 2018 Amir Rosenfeld, Markus D. Solbach, John K. Tsotsos

Perceptual judgment of image similarity by humans relies on rich internal representations ranging from low-level features to high-level concepts, scene properties and even cultural associations.

Bridging Cognitive Programs and Machine Learning

no code implementations16 Feb 2018 Amir Rosenfeld, John K. Tsotsos

While great advances are made in pattern recognition and machine learning, the successes of such fields remain restricted to narrow applications and seem to break down when training data is scarce, a shift in domain occurs, or when intelligent reasoning is required for rapid adaptation to new environments.

BIG-bench Machine Learning reinforcement-learning +1

Challenging Images For Minds and Machines

no code implementations13 Feb 2018 Amir Rosenfeld, John K. Tsotsos

There is no denying the tremendous leap in the performance of machine learning methods in the past half-decade.

BIG-bench Machine Learning Position

Joint Attention in Driver-Pedestrian Interaction: from Theory to Practice

no code implementations7 Feb 2018 Amir Rasouli, John K. Tsotsos

Today, one of the major challenges that autonomous vehicles are facing is the ability to drive in urban environments.

Autonomous Vehicles

Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next to Nothing

no code implementations2 Feb 2018 Amir Rosenfeld, John K. Tsotsos

The implications of this intriguing property of deep neural networks are discussed and we suggest ways to harness it to create more robust representations.

Saccade Sequence Prediction: Beyond Static Saliency Maps

no code implementations29 Nov 2017 Calden Wloka, Iuliia Kotseruba, John K. Tsotsos

The accuracy of such models has dramatically increased recently due to deep learning.

STAR-RT: Visual attention for real-time video game playing

no code implementations26 Nov 2017 Iuliia Kotseruba, John K. Tsotsos

In this paper we present STAR-RT - the first working prototype of Selective Tuning Attention Reference (STAR) model and Cognitive Programs (CPs).


Priming Neural Networks

1 code implementation16 Nov 2017 Amir Rosenfeld, Mahdi Biparva, John K. Tsotsos

This process has been shown to be an effect of top-down signaling in the visual system triggered by the said cue.

Object object-detection +2

Vision-Based Fallen Person Detection for the Elderly

1 code implementation24 Jul 2017 Markus D. Solbach, John K. Tsotsos

Furthermore, our system consists of a reasoning module which formulates a number of measures to reason whether a person is fallen.

Human Detection

Incremental Learning Through Deep Adaptation

no code implementations ICLR 2018 Amir Rosenfeld, John K. Tsotsos

Given an existing trained neural network, it is often desirable to learn new capabilities without hindering performance of those already learned.

Continual Learning Image Classification +2

Integrating Three Mechanisms of Visual Attention for Active Visual Search

no code implementations14 Feb 2017 Amir Rasouli, John K. Tsotsos

Algorithms for robotic visual search can benefit from the use of visual attention methods in order to reduce computational costs.

The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects

no code implementations14 Feb 2017 Amir Rasouli, John K. Tsotsos

The results indicate that on average color space C1C2C3 followed by HSI and XYZ achieve the best time in searching for objects of various colors.

A Review of 40 Years of Cognitive Architecture Research: Core Cognitive Abilities and Practical Applications

no code implementations27 Oct 2016 Iuliia Kotseruba, John K. Tsotsos

Thus, in this survey we wanted to shift the focus towards a more inclusive and high-level overview of the research on cognitive architectures.

Joint Attention in Autonomous Driving (JAAD)

no code implementations15 Sep 2016 Iuliia Kotseruba, Amir Rasouli, John K. Tsotsos

In this paper we present a novel dataset for a critical aspect of autonomous driving, the joint attention that must occur between drivers and of pedestrians, cyclists or other drivers.


Revisiting Active Perception

no code implementations8 Mar 2016 Ruzena Bajcsy, Yiannis Aloimonos, John K. Tsotsos

Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception.

Cannot find the paper you are looking for? You can Submit a new open access paper.