Search Results for author:
Found 39 papers, 13 papers with code
Retrieval, Analogy, and Composition: A framework for Compositional Generalization in Image Captioning
Image captioning systems are expected to have the ability to combine individual concepts when describing scenes with concept combinations that are not observed during training.
Semantic Aligned Multi-modal Transformer for Vision-LanguageUnderstanding: A Preliminary Study on Visual QA
Recent vision-language understanding approaches adopt a multi-modal transformer pre-training and finetuning paradigm.
Language Models Can Reduce Asymmetry in Information Markets
This work addresses the buyer's inspection paradox for information markets.
Compositional 3D Scene Synthesis with Scene Graph Guided Layout-Shape Generation
Recent progresses have been made in shape generation with powerful generative models, such as diffusion models, which increases the shape fidelity.
Learning 3D object-centric representation through prediction
As part of human core knowledge, the representation of objects is the building block of mental representation that supports high-level concepts and symbolic reasoning.
ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling
By combining natural language understanding and the generation capabilities and breadth of knowledge of large language models with image perception, recent large vision language models (LVLMs) have shown unprecedented reasoning capabilities in the real world.
AffordanceLLM: Grounding Affordance from Vision Language Models
Affordance grounding refers to the task of finding the area of an object with which one can interact.
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models
Compositionality is a common property in many modalities including natural languages and images, but the compositional generalization of multi-modal models is not well-understood.
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention
On the one hand, using dense attention in ViT leads to excessive memory and computational cost, and features can be influenced by irrelevant parts that are beyond the region of interests.
Ranked #4 on
Object Detection
on COCO 2017
GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fields
To incorporate semantics in 3D, the reconstruction module utilizes a vision-language foundation model ($\textit{e. g.}$, Stable Diffusion) to distill rich semantic information into the deep 3D voxel.
HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models
A human evaluation aligned with 95% of our evaluations on average was conducted to probe the effectiveness of HRS-Bench.
For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
Our study yields a series of intriguing results, including the discovery that the effectiveness of pre-training is highly dependent on the choice of the downstream policy learning algorithm.
ImageCaptioner$^2$: Image Captioner for Image Captioning Bias Amplification Assessment
In addition, we design a formulation for measuring the bias of generated captions as prompt-based image captioning instead of using language classifiers.
LiDAR-Based 3D Object Detection via Hybrid 2D Semantic Scene Generation
Bird's-Eye View (BEV) features are popular intermediate scene representations shared by the 3D backbone and the detector head in LiDAR-based object detectors.
Evaluating Step-by-Step Reasoning through Symbolic Verification
Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations or chain-of-thoughts (CoT)) for in-context learning.
Improving self-supervised representation learning via sequential adversarial masking
Recent methods in self-supervised learning have demonstrated that masking-based pretext tasks extend beyond NLP, serving as useful pretraining objectives in computer vision.
Policy Adaptation from Foundation Model Feedback
When deploying the trained policy to a new task or a new environment, we first let the policy play with randomly generated instructions to record the demonstrations.
A General Purpose Neural Architecture for Geospatial Systems
Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications.
Neural Attentive Circuits
Recent work has seen the development of general purpose neural architectures that can be trained to perform tasks across diverse data modalities.
Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning
In some complex environments with continuous state-action spaces, sparse rewards, and/or long temporal horizons, learning a good policy in the original environments can be difficult.
Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation
Previous works on Treatment Effect Estimation (TEE) are not in widespread use because they are predominantly theoretical, where strong parametric assumptions are made but untractable for practical application.
A Causal Lens for Controllable Text Generation
Controllable text generation concerns two fundamental tasks of wide applications, namely generating text of given attributes (i. e., attribute-conditional generation), and minimally editing existing text to possess desired attributes (i. e., text attribute transfer).
Vision Transformer with Deformable Attention
On the one hand, using dense attention e. g., in ViT, leads to excessive memory and computational cost, and features can be influenced by irrelevant parts which are beyond the region of interests.
Ranked #107 on
Object Detection
on COCO test-dev
Learning to perceive objects by prediction
Infants develop the notion of objects without supervision.
CausalDyna: Improving Generalization of Dyna-style Reinforcement Learning via Counterfactual-Based Data Augmentation
Deep reinforcement learning agents trained in real-world environments with a limited diversity of object properties to learn manipulation tasks tend to suffer overfitting and fail to generalize to unseen testing environments.
Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving
The ability to predict unseen vehicles is critical for safety in autonomous driving.
StruMonoNet: Structure-Aware Monocular 3D Prediction
Monocular 3D prediction is one of the fundamental problems in 3D vision.
Robust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals
Vehicle detection with visual sensors like lidar and camera is one of the critical functions enabling autonomous driving.
Correcting Automated and Manual Speech Transcription Errors using Warped Language Models
Masked language models have revolutionized natural language processing systems in the past few years.
Disentangled Recurrent Wasserstein Autoencoder
Learning disentangled representations leads to interpretable models and facilitates data generation with style transfer, which has been extensively studied on static data such as images in an unsupervised learning framework.
Motion Forecasting with Unlikelihood Training
We propose a new objective, unlikelihood training, which forces generated trajectories that conflicts with contextual information to be assigned a lower probability by our model.
HalentNet: Multimodal Trajectory Forecasting with Hallucinative Intents
Our model's learned representation leads to better and more semantically meaningful coverage of the trajectory distribution.
3D Object Detection with Pointformer
In this paper, we propose Pointformer, a Transformer backbone designed for 3D point clouds to learn features effectively.
Video Depth Estimation by Fusing Flow-to-Depth Proposals
Our flow-to-depth layer is differentiable, and thus we can refine camera poses by maximizing the aggregated confidence in the camera pose refinement module.
Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness
In contrast, we propose adaptive thin volumes (ATVs); in an ATV, the depth hypothesis of each plane is spatially varying, which adapts to the uncertainties of previous per-pixel depth predictions.
Ranked #13 on
3D Reconstruction
on DTU
Towards Safety-Aware Computing System Design in Autonomous Vehicles
To address this issue, we propose a `safety score' as a primary metric for measuring the level of safety in AV computing system design.
Disentangled Deep Autoencoding Regularization for Robust Image Classification
In spite of achieving revolutionary successes in machine learning, deep convolutional neural networks have been recently found to be vulnerable to adversarial attacks and difficult to generalize to novel test images with reasonably large geometric transformations.
Fast and Accurate Performance Analysis of LTE Radio Access Networks
Our choice of this domain is influenced by its commonalities with several other domains that produce real-time data, our access to a large live dataset, and their real-time nature and dimensionality which makes it a natural fit for a popular analysis technique, machine learning (ML).
