Search Results for author:
Found 21 papers, 12 papers with code
Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V
Autonomous robot navigation and manipulation in open environments require reasoning and replanning with closed-loop feedback.
PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI
With recent developments in Embodied Artificial Intelligence (EAI) research, there has been a growing demand for high-quality, large-scale interactive scene generation.
Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance
Despite significant advancements in text-to-motion synthesis, generating language-guided human motion within 3D environments poses substantial challenges.
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
In comparison to recent advancements in the 2D domain, grounding language in 3D scenes faces several significant challenges: (i) the inherent complexity of 3D scenes due to the diverse object configurations, their rich attributes, and intricate relationships; (ii) the scarcity of paired 3D vision-language data to support grounded learning; and (iii) the absence of a unified learning framework to distill knowledge from grounded 3D data.
An Embodied Generalist Agent in 3D World
Leveraging massive knowledge and learning schemes from large language models (LLMs), recent machine learning models show notable successes in building generalist agents that exhibit the capability of general-purpose task solving in diverse domains, including natural language processing, computer vision, and robotics.
X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events
Intuitive physics is pivotal for human understanding of the physical world, enabling prediction and interpretation of events even in infancy.
ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes
To tackle these challenges, we present ARNOLD, a benchmark that evaluates language-grounded task learning with continuous states in realistic 3D scenes.
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
SceneDiffuser provides a unified model for solving scene-conditioned generation, optimization, and planning.
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation
Current computer vision models, unlike the human visual system, cannot yet achieve general-purpose visual understanding.
Improving Object-centric Learning with Query Optimization
The ability to decompose complex natural scenes into meaningful object-centric abstractions lies at the core of human perception and reasoning.
EgoTaskQA: Understanding Human Tasks in Egocentric Videos
The challenges of such capability lie in the difficulty of generating a detailed understanding of situated actions, their effects on object states (i. e., state changes), and their causal dependencies.
Latent Diffusion Energy-Based Model for Interpretable Text Modeling
Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling.
Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization.
Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution
To fill in this gap, we propose a neuro-symbolic Probabilistic Abduction and Execution (PrAE) learner; central to the PrAE learner is the process of probabilistic abduction and execution on a probabilistic scene representation, akin to the mental manipulation of objects.
ACRE: Abstract Causal REasoning Beyond Covariation
Causal induction, i. e., identifying unobservable mechanisms that lead to the observable relations among variables, has played a pivotal role in modern scientific discovery, especially in scenarios with only sparse and limited data.
Learning Algebraic Representation for Abstract Spatial-Temporal Reasoning
We further show that the algebraic representation learned can be decoded by isomorphism and used to generate an answer.
LEMMA: A Multi-view Dataset for Learning Multi-agent Multi-task Activities
Understanding and interpreting human actions is a long-standing challenge and a critical indicator of perception in artificial intelligence.
Learning Perceptual Inference by Contrasting
"Thinking in pictures," [1] i. e., spatial-temporal reasoning, effortless and instantaneous for humans, is believed to be a significant ability to perform logical induction and a crucial factor in the intellectual history of technology development.
RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
In this work, we propose a new dataset, built in the context of Raven's Progressive Matrices (RPM) and aimed at lifting machine intelligence by associating vision with structural, relational, and analogical reasoning in a hierarchical representation.
Learning Human-Object Interactions by Graph Parsing Neural Networks
For a given scene, GPNN infers a parse graph that includes i) the HOI graph structure represented by an adjacency matrix, and ii) the node labels.
Ranked #32 on
Human-Object Interaction Detection
on V-COCO
Generalized Earley Parser: Bridging Symbolic Grammars and Sequence Data for Future Prediction
Future predictions on sequence data (e. g., videos or audios) require the algorithms to capture non-Markovian and compositional properties of high-level semantics.
