Search Results for author:
Found 32 papers, 8 papers with code
A Challenge to Build Neuro-Symbolic Video Agents
Modern video understanding systems excel at tasks such as scene classification, object detection, and short video retrieval.
We'll Fix it in Post: Improving Text-to-Video Generation with Neuro-Symbolic Feedback
Current text-to-video (T2V) generation models are increasingly popular due to their ability to produce coherent videos from textual prompts.
Learning Human Perception Dynamics for Informative Robot Communication
We introduce CoNav-Maze, a simulated robotics environment where a robot navigates using local perception while a human operator provides guidance based on an inaccurate map.
Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations
Many continuous control problems can be formulated as sparse-reward reinforcement learning (RL) tasks.
SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Models
Additionally, we demonstrate that our SynDiff-AD pipeline enhances the driving performance of end-to-end autonomous driving models, like AIM-2D and AIM-BEV, by up to 20% across diverse environmental conditions in the CARLA autonomous driving simulator, providing a more robust model.
Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification
As these models become prevalent, various metrics and benchmarks have emerged to evaluate the quality of the generated videos.
Any2Any: Incomplete Multimodal Retrieval with Conformal Prediction
Autonomous agents perceive and interpret their surroundings by integrating multimodal inputs, such as vision, audio, and LiDAR.
Human-Agent Coordination in Games under Incomplete Information via Multi-Step Intent
To synthesize cooperative policies for the agent in this extended game, we propose an approach featuring a memory module for a running probabilistic belief of the environment dynamics and an online planning algorithm called IntentMCTS.
Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learning
In reinforcement learning, conducting task composition by forming cohesive, executable sequences from multiple tasks remains challenging.
Robot-Enabled Machine Learning-Based Diagnosis of Gastric Cancer Polyps Using Partial Surface Tactile Imaging
In this paper, to collectively address the existing limitations on endoscopic diagnosis of Advanced Gastric Cancer (AGC) Tumors, for the first time, we propose (i) utilization and evaluation of our recently developed Vision-based Tactile Sensor (VTS), and (ii) a complementary Machine Learning (ML) algorithm for classifying tumors using their textural features.
Should we use model-free or model-based control? A case study of battery management systems
Reinforcement learning (RL) and model predictive control (MPC) each offer distinct advantages and limitations when applied to control problems in power and energy systems.
Towards Neuro-Symbolic Video Understanding
The unprecedented surge in video data production in recent years necessitates efficient tools to extract meaningful frames from videos for downstream tasks.
Time Weaver: A Conditional Time Series Generation Model
Current approaches to time series generation often ignore this paired metadata, and its heterogeneity poses several practical challenges in adapting existing conditional generation approaches from the image, audio, and video domains to the time series domain.
Online Foundation Model Selection in Robotics
We thus formulate a user-centric online model selection problem and propose a novel solution that combines an open-source encoder to output context and an online learning algorithm that processes this context.
Specification-Driven Video Search via Foundation Models and Formal Verification
The increasing abundance of video data enables users to search for events of interest, e. g., emergency incidents.
Task-aware Distributed Source Coding under Dynamic Bandwidth
A decoder at the central node decompresses and passes the data to a pre-trained machine learning-based task to generate the final output.
Data Games: A Game-Theoretic Approach to Swarm Robotic Data Collection
Instead, we propose a cooperative data sampling strategy where geo-distributed AVs collaborate to collect a diverse ML training dataset in the cloud.
Forecaster-aided User Association and Load Balancing in Multi-band Mobile Networks
Cellular networks are becoming increasingly heterogeneous with higher base station (BS) densities and ever more frequency bands, making BS selection and band assignment key decisions in terms of rate and coverage.
A Control Theoretic Approach to Infrastructure-Centric Blockchain Tokenomics
As such, the associated token rewards should gracefully scale with the size of the decentralized system, but should be carefully balanced with consumer demand to manage inflation and be designed to ultimately reach an equilibrium.
Online Poisoning Attacks Against Data-Driven Predictive Control
Data-driven predictive control (DPC) is a feedback control method for systems with unknown dynamics.
Joint Learning of Reward Machines and Policies in Environments with Partially Known Semantics
We develop a reinforcement-learning algorithm that infers a reward machine that encodes the underlying task while learning how to execute it, despite the uncertainties of the propositions' truth values.
Task-Aware Network Coding Over Butterfly Network
Classical network coding is largely task-agnostic -- the coding schemes mainly aim to faithfully reconstruct data at the receivers, regardless of what ultimate task the received data is used for.
Class-Aware Adversarial Transformers for Medical Image Segmentation
In this work, we present CASTformer, a novel type of adversarial transformers, for 2D medical image segmentation.
Personalized Federated Learning of Driver Prediction Models for Autonomous Driving
Autonomous vehicles (AVs) must interact with a diverse set of human drivers in heterogeneous geographic areas.
ML-EXray: Visibility into ML Deployment on the Edge
The key challenge is that there is not much visibility into ML inference execution on edge devices, and very little awareness of potential issues during the edge deployment process.
Task-aware Privacy Preservation for Multi-dimensional Data
Local differential privacy (LDP) can be adopted to anonymize richer user data attributes that will be input to sophisticated machine learning (ML) tasks.
Data Sharing and Compression for Cooperative Networked Control
Sharing forecasts of network timeseries data, such as cellular or electricity load patterns, can improve independent control applications ranging from traffic scheduling to power generation.
Sampling Training Data for Continual Learning Between Robots and the Cloud
Today's robotic fleets are increasingly measuring high-volume video and LIDAR sensory streams, which can be mined for valuable training data, such as rare scenes of road construction sites, to steadily improve robotic perception models.
Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging
We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp.
Task-relevant Representation Learning for Networked Robotic Perception
However, today's representations for sensory data are mostly designed for human, not robotic, perception and thus often waste precious compute or wireless network resources to transmit unimportant parts of a scene that are unnecessary for a high-level robotic task.
Multi-agent Reinforcement Learning for Networked System Control
This paper considers multi-agent reinforcement learning (MARL) in networked system control.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Network Offloading Policies for Cloud Robotics: a Learning-based Approach
In this paper, we formulate a novel Robot Offloading Problem --- how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication?
