Search Results for author:
Found 26 papers, 15 papers with code
Towards Socially Intelligent Agents with Mental State Transition and Human Value
One of which is to track the agent’s mental state transition and teach the agent to make decisions guided by its value like a human.
TheoremQA: A Theorem-driven Question Answering dataset
We evaluate a wide spectrum of 16 large language and code models with different prompting strategies like Chain-of-Thoughts and Program-of-Thoughts.
Multimodal Procedural Planning via Dual Text-Image Prompting
The key challenges of MPP are to ensure the informativeness, temporal coherence, and accuracy of plans across modalities.
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
This strategy effectively alleviates the interference between the two tasks of image-text alignment and instruction following and achieves strong multi-modal reasoning with only a small-scale image-text and instruction dataset.
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
Chameleon synthesizes programs by composing various tools (e. g., LLMs, off-the-shelf vision models, web search engines, Python functions, and heuristic-based modules) for accomplishing complex reasoning tasks.
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
A Survey of Deep Learning for Mathematical Reasoning
Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life.
UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression
Naturally, we also present a unified multi-task Geometric Transformer framework, Geoformer, to tackle calculation and proving problems simultaneously in the form of sequence generation, which finally shows the reasoning ability can be improved on both two tasks by unifying formulation.
Ranked #3 on
Mathematical Reasoning
on PGPS9K
Lila: A Unified Benchmark for Mathematical Reasoning
Mathematical reasoning skills are essential for general-purpose intelligent systems to perform tasks from grocery shopping to climate modeling.
Ranked #1 on
Mathematical Reasoning
on Lila (OOD)
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data.
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
We further design language models to learn to generate lectures and explanations as the chain of thought (CoT) to mimic the multi-hop reasoning process when answering ScienceQA questions.
Ranked #3 on
Science Question Answering
on ScienceQA
Triangular Character Animation Sampling with Motion, Emotion, and Relation
We propose a Spatial-Temporal And-Or graph (ST-AOG), a stochastic grammar model, to encode the contextual relationship between motion, emotion, and relation, forming a triangle in a conditional random field.
ValueNet: A New Dataset for Human Value Driven Dialogue System
To the best of our knowledge, ValueNet is the first large-scale text dataset for human value modeling, and we are the first one trying to incorporate a value model into emotionally intelligent dialogue systems.
Learning from the Tangram to Solve Mini Visual Tasks
Current pre-training methods in computer vision focus on natural images in the daily-life context.
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Also, we develop a strong IconQA baseline Patch-TRM that applies a pyramid cross-modal Transformer with input diagram embeddings pre-trained on the icon dataset.
Ranked #1 on
Visual Question Answering (VQA)
on IconQA
SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues
Inferring social relations from dialogues is vital for building emotionally intelligent robots to interpret human language better and act accordingly.
Ranked #5 on
Dialog Relation Extraction
on DialogRE
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning
We further propose a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem Solver (Inter-GPS).
Ranked #1 on
Mathematical Question Answering
on GeoS
Towards Socially Intelligent Agents with Mental State Transition and Human Utility
One of which is to track the agent's mental state transition and teach the agent to make decisions guided by its value like a human.
Learning Long- and Short-Term User Literal-Preference with Multimodal Hierarchical Transformer Network for Personalized Image Caption
Personalized image caption, a natural extension of the standard image caption task, requires to generate brief image descriptions tailored for users’ writing style and traits, and is more practical to meet users’ real demands.
Knowledge Aware Semantic Concept Expansion for Image-Text Matching
In this paper, we develop a Scene Concept Graph (SCG) by aggregating image scene graphs and extracting frequently co-occurred concept pairs as scene common-sense knowledge.
Knowledge-Aware Deep Dual Networks for Text-Based Mortality Prediction
To address the above issues, we propose novel Knowledge-aware Deep Dual Networks (K-DDN) for the text-based mortality prediction task.
A Novel Hybrid Sequential Model for Review-based Rating Prediction
Nowadays, the online interactions between users and items become diverse, and may include textual reviews as well as numerical ratings.
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
Question-Guided Hybrid Convolution for Visual Question Answering
Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features. To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage.
R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering
To better utilize semantic knowledge in images, we propose a novel framework to learn visual relation facts for VQA.
Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Existing VQA methods mainly adopt the visual attention mechanism to associate the input question with corresponding image regions for effective question answering.
