Search Results for author:
Found 41 papers, 14 papers with code
StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated.
StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated.
Social Attention: Modeling Attention in Human Crowds
In this work, we propose Social Attention, a novel trajectory prediction model that captures the relative importance of each person when navigating in the crowd, irrespective of their proximity.
EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting
In this paper, we present EigenTrajectory ($\mathbb{ET}$), a trajectory prediction approach that uses a novel trajectory descriptor to form a compact space, known here as $\mathbb{ET}$ space, in place of Euclidean space, for representing pedestrian movements.
Robot Synesthesia: A Sound and Emotion Guided AI Painter
For speech, we decouple speech into its transcribed text and the tone of the speech.
Complementary Random Masking for RGB-Thermal Semantic Segmentation
Also, the proposed self-distillation loss encourages the network to extract complementary and meaningful representations from a single modality or complementary masked modalities.
Ranked #2 on
Thermal Image Segmentation
on MFN Dataset
Trajformer: Trajectory Prediction with Local Self-Attentive Contexts for Autonomous Driving
Effective feature-extraction is critical to models' contextual understanding, particularly for applications to robotics and autonomous driving, such as multimodal trajectory prediction.
Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimization
We introduce an approach to generating videos based on a series of given language descriptions.
Path Planning in Dynamic Environments with Adaptive Dimensionality
In this paper, we apply the idea of adaptive dimensionality to speed up path planning in dynamic environments for a robot with no assumptions on its dynamic model.
Robotics
Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agent
The objective of most Reinforcement Learning painting agents is to minimize the loss between a target image and the paint canvas.
T2FPV: Dataset and Method for Correcting First-Person View Errors in Pedestrian Trajectory Prediction
To support first-person view trajectory prediction research, we present T2FPV, a method for constructing high-fidelity first-person view (FPV) datasets given a real-world, top-down trajectory dataset; we showcase our approach on the ETH/UCY pedestrian dataset to generate the egocentric visual data of all interacting pedestrians, creating the T2FPV-ETH dataset.
Artistic Style in Robotic Painting; a Machine Learning Approach to Learning Brushstroke from Human Artists
We propose a method to integrate an artistic style to the brushstrokes and the painting process through collaboration with a human artist.
Towards Equitable Representation in Text-to-Image Synthesis Models with the Cross-Cultural Understanding Benchmark (CCUB) Dataset
It has been shown that accurate representation in media improves the well-being of the people who consume it.
Cultural Vocal Bursts Intensity Prediction
Image Generation
+3
Learning Lexical Entries for Robotic Commands using Crowdsourcing
Robotic commands in natural language usually contain various spatial descriptions that are semantically similar but syntactically different.
Explainable Semantic Mapping for First Responders
One of the key challenges in the semantic mapping problem in postdisaster environments is how to analyze a large amount of data efficiently with minimal supervision.
Following Social Groups: Socially Compliant Autonomous Navigation in Dense Crowds
The underlying system incorporates a deep neural network to track social groups and join the flow of a social group in facilitating the navigation.
Adjusting Image Attributes of Localized Regions with Low-level Dialogue
Our system grounds language on the level of edit operations, and suggests options for a user to choose from.
A Multimodal Dialogue System for Conversational Image Editing
In this paper, we present a multimodal dialogue system for Conversational Image Editing.
Noticing Motion Patterns: Temporal CNN with a Novel Convolution Operator for Human Trajectory Prediction
We propose a Convolutional Neural Network-based approach to learn, detect, and extract patterns in sequential trajectory data, known here as Social Pattern Extraction Convolution (Social-PEC).
Image Captioning with Compositional Neural Module Networks
In image captioning where fluency is an important factor in evaluation, e. g., $n$-gram metrics, sequential models are commonly used; however, sequential models generally result in overgeneralized expressions that lack the details that may be present in an input image.
Anytime 3D Object Reconstruction using Multi-modal Variational Autoencoder
In this context, we propose a method for imputation of latent variables whose elements are partially lost.
Anchor Distance for 3D Multi-Object Distance Estimation from 2D Single Shot
Given a 2D Bounding Box (BBox) and object parameters, a 3D distance to the object can be calculated directly using 3D reprojection; however, such methods are prone to significant errors because an error from the 2D detection can be amplified in 3D.
Domain Adaptive Monocular Depth Estimation With Semantic Information
The advent of deep learning has brought an impressive advance to monocular depth estimation, e. g., supervised monocular depth estimation has been thoroughly investigated.
Self-supervised Learning of 3D Object Understanding by Data Association and Landmark Estimation for Image Sequence
Therefore, we propose a strategy to exploit multipleobservations of the object in the image sequence in orderto surpass the self-performance: first, the landmarks for theglobal object map are estimated through network predic-tion and data association, and the corrected annotation fora single frame is obtained.
Language Understanding for Field and Service Robots in a Priori Unknown Environments
This progress now creates an opportunity for robots to operate not only in isolation, but also with and alongside humans in our complex environments.
Core Challenges in Embodied Vision-Language Planning
Recent advances in the areas of multimodal machine learning and artificial intelligence (AI) have led to the development of challenging tasks at the intersection of Computer Vision, Natural Language Processing, and Embodied AI.
Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling
LOGOS leverages two grounding tasks to better localize the key information of the image, utilizes scene text clustering to group individual OCR tokens, and learns to select the best answer from different sources of OCR (Optical Character Recognition) texts.
Unsupervised Domain Adaptation Via Pseudo-labels And Objectness Constraints
Crucially, the objectness constraint is agnostic to the ground-truth semantic segmentation labels and, therefore, remains appropriate for unsupervised adaptation settings.
Translating Robot Skills: Learning Unsupervised Skill Correspondences Across Robots
In this paper, we explore how we can endow robots with the ability to learn correspondences between their own skills, and those of morphologically different robots in different domains, in an entirely unsupervised manner.
Safe Autonomous Racing via Approximate Reachability on Ego-vision
Given the nature of the task, autonomous agents need to be able to 1) identify and avoid unsafe scenarios under the complex vehicle dynamics, and 2) make sub-second decision in a fast-changing environment.
Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing
In the first stage of the challenge, we evaluate an autonomous agent's ability to drive as fast as possible, while adhering to safety constraints.
RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning
Under shared autonomy, wheelchair users expect vehicles to provide safe and comfortable rides while following users high-level navigation plans.
Distribution-aware Goal Prediction and Conformant Model-based Planning for Safe Autonomous Driving
The feasibility of collecting a large amount of expert demonstrations has inspired growing research interests in learning-to-drive settings, where models learn by imitating the driving behaviour from experts.
Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation
We also define a new audio-visual navigation sub-task, where agents are evaluated on novel sounding objects, as opposed to unheard clips of known objects.
Learned Tree Search for Long-Horizon Social Robot Navigation in Shared Airspace
In this work, we propose Social Robot Tree Search (SoRTS), an algorithm for the safe navigation of mobile robots in social domains.
Core Challenges in Embodied Vision-Language Planning
Recent advances in the areas of Multimodal Machine Learning and Artificial Intelligence (AI) have led to the development of challenging tasks at the intersection of Computer Vision, Natural Language Processing, and Robotics.
Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints
Self-training based on pseudo-labels has emerged as a dominant approach for addressing conditional distribution shifts in unsupervised domain adaptation (UDA) for semantic segmentation problems.
FishRecGAN: An End to End GAN Based Network for Fisheye Rectification and Calibration
We propose an end-to-end deep learning approach to rectify fisheye images and simultaneously calibrate camera intrinsic and distortion parameters.
SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation
Accurate representation in media is known to improve the well-being of the people who consume it.
Towards Human-Centered Construction Robotics: An RL-Driven Companion Robot For Contextually Assisting Carpentry Workers
In the dynamic construction industry, traditional robotic integration has primarily focused on automating specific tasks, often overlooking the complexity and variability of human aspects in construction workflows.
