Search Results for author:
Found 22 papers, 8 papers with code
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks
Videos are often used to learn or extract the necessary information to complete tasks in ways different than what text and static imagery alone can provide.
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale
To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi.
Data Generation Using Large Language Models for Text Classification: An Empirical Case Study
Using Large Language Models (LLMs) to generate synthetic data for model training has become increasingly popular in recent years.
EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields
We present EvDNeRF, a pipeline for generating event data and training an event-based dynamic NeRF, for the purpose of faithfully reconstructing eventstreams on scenes with rigid and non-rigid deformations that may be too fast to capture with a standard camera.
Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training
We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning.
ConBaT: Control Barrier Transformer for Safe Policy Learning
In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion.
ChatGPT for Robotics: Design Principles and Model Abilities
This paper presents an experimental study regarding the use of OpenAI's ChatGPT for robotics applications.
SMART: Self-supervised Multi-task pretrAining with contRol Transformers
Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels.
PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training
Robotics has long been a field riddled with complex systems architectures whose modules and connections, whether traditional or learning-based, require significant human expertise and prior knowledge.
LATTE: LAnguage Trajectory TransformEr
Natural language is one of the most intuitive ways to express human intent.
Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers
However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands.
3D Human Reconstruction in the Wild with Collaborative Aerial Cameras
Aerial vehicles are revolutionizing applications that require capturing the 3D structure of dynamic targets in the wild, such as sports, medicine, and entertainment.
Batteries, camera, action! Learning a semantic control space for expressive robot cinematography
First, we generate a database of video clips with a diverse range of shots in a photo-realistic simulator, and use hundreds of participants in a crowd-sourcing framework to obtain scores for a set of semantic descriptors for each clip.
Do You See What I See? Coordinating Multiple Aerial Cameras for Robot Cinematography
We validate our approach in multiple cluttered environments of a photo-realistic simulator, and deploy the system using two UAVs in real-world experiments.
Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Making
In this work, we address the problem in its entirety and propose a complete system for real-time aerial cinematography that for the first time combines: (1) vision-based target estimation; (2) 3D signed-distance mapping for occlusion estimation; (3) efficient trajectory optimization for long time-horizon camera motion; and (4) learning-based artistic shot selection.
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors.
Development of email classifier in Brazilian Portuguese using feature selection for automatic response
We study the automatic reply of email business messages in Brazilian Portuguese.
Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography
Aerial filming is constantly gaining importance due to the recent advances in drone technology.
Towards a Robust Aerial Cinematography Platform: Localizing and Tracking Moving Targets in Unstructured Environments
In this work, we overcome such limitations and propose a complete system for aerial cinematography that combines: (1) a vision-based algorithm for target localization; (2) a real-time incremental 3D signed-distance map algorithm for occlusion and safety computation; and (3) a real-time camera motion planner that optimizes smoothness, collisions, occlusions and artistic guidelines.
Improved Generalization of Heading Direction Estimation for Aerial Filming Using Semi-supervised Regression
We show that by leveraging unlabeled sequences, the amount of labeled data required can be significantly reduced.
Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories
The first-stage network learns feature representations of the environment using low-level LiDAR statistics and the second-stage network combines those learned features with kinematics data.
Autonomous drone cinematographer: Using artistic principles to create smooth, safe, occlusion-free trajectories for aerial filming
Autonomous aerial cinematography has the potential to enable automatic capture of aesthetically pleasing videos without requiring human intervention, empowering individuals with the capability of high-end film studios.
