Search Results for author:
Found 36 papers, 23 papers with code
Building Goal-oriented Document-grounded Dialogue Systems
In this paper, we describe our systems for solving the two Doc2Dial shared task: knowledge identification and response generation.
OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization
In this paper, we introduce an open-source framework designed to facilitate the development of multimodal web agent that can autonomously conduct real-world exploration and improve itself.
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
Unlike traditional function-level or file-level coding tasks, AI software engineering requires not only basic coding proficiency but also advanced skills in managing and interacting with code repositories.
DivScene: Benchmarking LVLMs for Object Navigation with Diverse Scenes and Objects
In this paper, we study a new task of navigating to diverse target objects in a large number of scene types.
Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks
Tasks involving multiple text-rich images are especially challenging, as they require not only understanding the content of individual images but reasoning about inter-relationships and logical flows across multiple visual inputs.
Cognitive Kernel: An Open-source Agent System towards Generalist Autopilots
Cognitive Kernel adopts a model-centric design.
DSBench: How Far Are Data Science Agents to Becoming Data Science Experts?
To bridge this gap, we introduce DSBench, a comprehensive benchmark designed to evaluate data science agents with realistic tasks.
COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes
Then, we develop COLUMBUS, a synthetic benchmark that applies the task pipeline to create QA sets with text and icon rebus puzzles based on publicly available collections of compounds and common phrases.
DOCBENCH: A Benchmark for Evaluating LLM-based Document Reading Systems
In this paper, we introduce DocBench, a new benchmark designed to evaluate LLM-based document reading systems.
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
In this paper, we split the original benchmark to also support fine-tuning setting and present SemEval Task 9: BRAIN-TEASER(S), the first task at this competition designed to test the system's reasoning and lateral thinking ability.
MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning
Further analysis of perception questions reveals that MLLMs struggle to comprehend the visual features (near-random performance) and even count the panels in the puzzle ( <45%), hindering their ability for abstract reasoning.
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
The rapid advancement of large language models (LLMs) has led to a new era marked by the development of autonomous applications in real-world scenarios, which drives innovation in creating advanced web agents.
Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models
In response to these challenges, we introduces Chain-of-Noting (CoN), a novel approach aimed at improving the robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios.
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms.
LASER: LLM Agent with State-Space Exploration for Web Navigation
We evaluate our proposed LLM Agent with State-Space ExploRation (LASER) on both the WebShop task and amazon. com.
A Study of Situational Reasoning for Traffic Understanding
Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure.
Knowledge-enhanced Agents for Interactive Text Games
We experiment with four models on the 10 tasks in the ScienceWorld text-based game environment, to illustrate the impact of knowledge injection on various model configurations and challenging task settings.
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Our approach outperforms recent self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.
Ranked #4 on
Question Answering
on HotpotQA
Transferring Procedural Knowledge across Commonsense Tasks
Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories.
Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
Understanding novel situations in the traffic domain requires an intricate combination of domain-specific and causal commonsense knowledge.
Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
We propose a novel open-domain question answering (ODQA) framework for answering single/multi-hop questions across heterogeneous knowledge sources.
Coalescing Global and Local Information for Procedural Text Understanding
In this paper, we propose Coalescing Global and Local Information (CGLI), a new model that builds entity- and timestep-aware input representations (local input) considering the whole context (global input), and we jointly model the entity states with a structured prediction objective (global output).
An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs
In this paper, we study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
Generalizable Neuro-symbolic Systems for Commonsense Question Answering
This chapter illustrates how suitable neuro-symbolic models for language understanding can enable domain generalizability and robustness in downstream tasks.
Open Domain Question Answering with A Unified Knowledge Interface
The retriever-reader framework is popular for open-domain question answering (ODQA) due to its ability to use explicit knowledge.
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models
In this paper, we investigate what models learn from commonsense reasoning datasets.
Dimensions of Commonsense Knowledge
Recently, the focus has been on large text-based sources, which facilitate easier integration with neural (language) models and application to textual tasks, typically at the expense of the semantics of the sources and their harmonization.
Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection
Conditional text generation has been a challenging task that is yet to see human-level performance from state-of-the-art models.
Audio-Visual Event Recognition through the lens of Adversary
This work aims to study several key questions related to multimodal learning through the lens of adversarial noises: 1) The trade-off between early/middle/late fusion affecting its robustness and accuracy 2) How do different frequency/time domain features contribute to the robustness?
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering
Guided by a set of hypotheses, the framework studies how to transform various pre-existing knowledge resources into a form that is most effective for pre-training models.
Neuro-symbolic Architectures for Context Understanding
Computational context understanding refers to an agent's ability to fuse disparate sources of information for decision-making and is, therefore, generally regarded as a prerequisite for sophisticated machine reasoning capabilities, such as in artificial intelligence (AI).
Bend but Don't Break? Multi-Challenge Stress Test for QA Models
The field of question answering (QA) has seen rapid growth in new tasks and modeling approaches in recent years.
Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering
Non-extractive commonsense QA remains a challenging AI task, as it requires systems to reason about, synthesize, and gather disparate pieces of information, in order to generate responses to queries.
Ranked #16 on
Common Sense Reasoning
on CommonsenseQA
Challenging Reading Comprehension on Daily Conversation: Passage Completion on Multiparty Dialog
This paper presents a new corpus and a robust deep learning architecture for a task in reading comprehension, passage completion, on multiparty dialog.
