Search Results for author:
Found 49 papers, 20 papers with code
Zero-BEV: Zero-shot Projection of Any First-Person Modality to BEV Maps
Bird's-eye view (BEV) maps are an important geometrically structured representation widely used in robotics, in particular self-driving vehicles and terrestrial robots.
Task-conditioned adaptation of visual features in multi-task policy learning
We evaluate the method on a wide variety of tasks from the CortexBench benchmark and show that, compared to existing work, it can be addressed with a single policy.
Learning to navigate efficiently and precisely in real environments
In the context of autonomous navigation of terrestrial robots, the creation of realistic models for agent dynamics and sensing is a widespread habit in the robotics literature and in commercial applications, where they are used for model based control and/or for localization and mapping.
Multi-Object Navigation in real environments using hybrid policies
More recently, beyond waypoint planning, problems involving significant components of (visual) high-level reasoning have been explored in simulated environments, mostly addressed with large-scale machine learning, in particular RL, offline-RL or imitation learning.
End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon
The main challenge lies in learning compact representations generalizable to unseen environments and in learning high-capacity perception modules capable of reasoning on high-dimensional input.
Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction
Agents navigating in 3D environments require some form of memory, which should hold a compact and actionable representation of the history of observations useful for decision taking and planning.
AutoNeRF: Training Implicit Scene Representations with Autonomous Agents
Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment, and can be used for several downstream robotic tasks, and that modular trained exploration models outperform other classical and end-to-end baselines.
Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformers
To perform future forecasting of pressure and velocity on the challenging EAGLE dataset, we introduce a new mesh transformer.
Multi-Object Navigation with dynamically learned neural implicit representations
Understanding and mapping a new environment are core abilities of any autonomously navigating agent.
Learning Reduced Nonlinear State-Space Models: an Output-Error Based Canonical Approach
The identification of a nonlinear dynamic model is an open topic in control theory, especially from sparse input-output measurements.
An experimental study of the vision-bottleneck in VQA
We also study the impact of two methods to incorporate the information about objects necessary for answering a question, in the reasoning module directly, and earlier in the object selection stage.
Filtered-CoPhy: Unsupervised Learning of Counterfactual Physics in Pixel Space
Learning causal relationships in high-dimensional data (images, videos) is a hard task, as they are often defined on low dimensional manifolds and must be extracted from complex signals dominated by appearance, lighting, textures and also spurious correlations in the data.
Graph augmented Deep Reinforcement Learning in the GameRLand3D environment
We address planning and navigation in challenging 3D video games featuring maps with disconnected regions reachable by agents using special actions.
Godot Reinforcement Learning Agents
We present Godot Reinforcement Learning (RL) Agents, an open-source interface for developing environments and agents in the Godot Game Engine.
An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environments
In this work we present an in-depth study of the performance and reasoning capacities of real physical agents, trained in simulation and deployed to two different physical environments.
Satellite Image Semantic Segmentation
In this paper, we propose a method for the automatic semantic segmentation of satellite images into six classes (sparse forest, dense forest, moor, herbaceous formation, building, and road).
SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation
The Robotics community has started to heavily rely on increasingly realistic 3D simulators for large-scale training of robots on massive amounts of data.
Underwater Acoustic Networks for Security Risk Assessment in Public Drinking Water Reservoirs
We have built a novel system for the surveillance of drinking water reservoirs using underwater sensor networks.
Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation
In the context of visual navigation, the capacity to map a novel environment is necessary for an agent to exploit its observation history in the considered place and efficiently reach known goals.
Universal Domain Adaptation in Ordinal Regression
We address the problem of universal domain adaptation (UDA) in ordinal regression (OR), which attempts to solve classification problems in which labels are not independent, but follow a natural order.
Supervising the Transfer of Reasoning Patterns in VQA
Methods for Visual Question Anwering (VQA) are notorious for leveraging dataset biases rather than performing reasoning, hindering generalization.
How Transferable are Reasoning Patterns in VQA?
Since its inception, Visual Question Answering (VQA) is notoriously known as a task, where models are prone to exploit biases in datasets to find shortcuts instead of performing high-level reasoning.
VisQA: X-raying Vision and Language Reasoning in Transformers
First, as a result of a collaboration of three fields, machine learning, vision and language reasoning, and data analytics, the work lead to a better understanding of bias exploitation of neural models for VQA, which eventually resulted in an impact on its design and training through the proposition of a method for the transfer of reasoning patterns from an oracle model.
Deep KKL: Data-driven Output Prediction for Non-Linear Systems
Building on this formulation and problem definition, we propose a predictor structure based on the Kazantzis-Kravaris/Luenberger (KKL) observer and we show that KKL fits well into our general framework.
SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentation
SST extracts per-pixel representations for each object in a video using sparse attention over spatiotemporal features.
Learning to plan with uncertain topological maps
We train an agent to navigate in 3D environments using a hierarchical strategy including a high-level graph based planner and a local policy.
Estimating semantic structure for the VQA answer space
Since its appearance, Visual Question Answering (VQA, i. e. answering a question posed over an image), has always been treated as a classification problem over a set of predefined answers.
Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?
Models for Visual Question Answering (VQA) are notorious for their tendency to rely on dataset biases, as the large and unbalanced diversity of questions and concepts involved and tends to prevent models from learning to reason, leading them to perform educated guesses instead.
EgoMap: Projective mapping and structured egocentric memory for Deep RL
The EgoMap architecture incorporates several inductive biases including a differentiable inverse projection of CNN feature vectors onto a top-down spatially structured map.
Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks
The large adoption of the self-attention (i. e. transformer model) and BERT-like training principles has recently resulted in a number of high performing models on a large panoply of vision-and-language problems (such as Visual Question Answering (VQA), image retrieval, etc.).
CoPhy: Counterfactual Learning of Physical Dynamics
Understanding causes and effects in mechanical systems is an essential component of reasoning in the physical world.
DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explainability in the field of visual analytics.
Attentional PointNet for 3D-Object Detection in Point Clouds
In this study, we propose Attentional Point- Net, which is a novel end-to-end trainable deep architecture for object detection in point clouds.
Learning 3D Navigation Protocols on Touch Interfaces with Cooperative Multi-Agent Reinforcement Learning
The second agent acts as the interaction protocol, interpreting and translating to 3D operations the 2D finger trajectories from the first agent.
Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer
In this paper we argue that research on training agents capable of complex reasoning can be simplified by decoupling from the requirement of high fidelity photographic observations.
Object Level Visual Reasoning in Videos
Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context.
Ranked #1 on
Semantic Object Interaction Classification
on VLOG
Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points
No spatial coherence is forced on the glimpse locations, which gives the module liberty to explore different points at each frame and better optimize the process of scrutinizing visual information.
Ranked #19 on
Skeleton Based Action Recognition
on N-UCLA
Learning to recognize touch gestures: recurrent vs. convolutional features and dynamic sampling
We propose a fully automatic method for learning gestures on big touch devices in a potentially multi-user context.
Human Action Recognition: Pose-based Attention draws focus to Hands
We propose a new spatio-temporal attention based mechanism for human action recognition able to automatically attend to the hands most involved into the studied action and detect the most discriminative moments in an action.
Multi-view pose estimation with mixtures-of-parts and adaptive viewpoint selection
We propose a new method for human pose estimation which leverages information from multiple views to impose a strong prior on articulated pose.
Residual Conv-Deconv Grid Network for Semantic Segmentation
However, for semantic image segmentation, where the task consists in providing a semantic class to each pixel of an image, feature maps reduction is harmful because it leads to a resolution loss in the output prediction.
KiDS-450 + 2dFLenS: Cosmological parameter constraints from weak gravitational lensing tomography and overlapping redshift-space galaxy clustering
The complementarity of our observables allows for constraints on modified gravity degrees of freedom that are not simultaneously bounded with either probe alone, and up to a factor of three improvement in the $S_8$ constraint in the extended cosmology compared to KiDS alone.
Cosmology and Nongalactic Astrophysics
Full-Page Text Recognition: Learning Where to Start and When to Stop
Text line detection and localization is a crucial step for full page document analysis, but still suffers from heterogeneity of real life documents.
Pose-conditioned Spatio-Temporal Attention for Human Action Recognition
We show that it is of high interest to shift the attention to different hands at different time steps depending on the activity itself.
Learning to detect and localize many objects from few examples
The current trend in object detection and localization is to learn predictions with high capacity deep neural networks trained on a very large amount of annotated data and using a high amount of processing power.
Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning
We propose a method for hand pose estimation based on a deep regressor trained on two different kinds of input.
Learning Human Identity from Motion Patterns
We present a large-scale study exploring the capability of temporal deep neural networks to interpret natural human kinematics and introduce the first method for active biometric authentication with mobile inertial sensors.
Activity recognition from videos with parallel hypergraph matching on GPUs
In this paper, we propose a method for activity recognition from videos based on sparse local features and hypergraph matching.
ModDrop: adaptive multi-modal gesture recognition
We present a method for gesture detection and localisation based on multi-scale and multi-modal deep learning.
