Search Results for author:
Found 19 papers, 15 papers with code
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity.
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Through extensive quantitative and qualitative analysis, we identify several limitations of text-only LLM agents, and reveal gaps in the capabilities of state-of-the-art multimodal language agents.
WebArena: A Realistic Web Environment for Building Autonomous Agents
Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions.
Hierarchical Prompting Assists Large Language Model on Web Navigation
Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks.
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Many recent advances in natural language generation have been fueled by training large language models on internet-scale data.
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
We release five language-specific pretrained models to use with our publicly available code.
Causal Reasoning of Entities and Events in Procedural Texts
By injecting the causal relations between entities and events as intermediate reasoning steps in our representation, we further boost the performance to . 67 F1.
Execution-Based Evaluation for Open-Domain Code Generation
To extend the scope of coding queries to more realistic settings, we propose ODEX, the first Open-Domain EXecution-based natural language (NL) to Python code generation dataset.
PAL: Program-aided Language Models
Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem.
Ranked #17 on
Arithmetic Reasoning
on GSM8K
Language Models of Code are Few-Shot Commonsense Learners
In all these natural language tasks, we show that using our approach, a code generation LM (CODEX) outperforms natural-LMs that are fine-tuned on the target task (e. g., T5) and other strong LMs such as GPT-3 in the few-shot setting.
DocPrompting: Generating Code by Retrieving the Docs
Publicly available source-code libraries are continuously growing and changing.
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages
While there has been a recent burgeoning of applications at the intersection of natural and programming languages, such as code generation and code summarization, these applications are usually English-centric.
Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
To this end, we develop a simple and efficient method that links steps (e. g., "purchase a camera") in an article to other articles with similar goals (e. g., "how to choose a camera"), recursively constructing the KB.
Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language
When humans conceive how to perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks.
Soft Gazetteers for Low-Resource Named Entity Recognition
However, designing such features for low-resource languages is challenging, because exhaustive entity gazetteers do not exist in these languages.
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts.
Towards Zero-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) grounds named entities in a source language to an English Knowledge Base (KB), such as Wikipedia.
Improving Robustness of Neural Machine Translation with Multi-task Learning
While neural machine translation (NMT) achieves remarkable performance on clean, in-domain text, performance is known to degrade drastically when facing text which is full of typos, grammatical errors and other varieties of noise.
Aggregated Semantic Matching for Short Text Entity Linking
The task of entity linking aims to identify concepts mentioned in a text fragments and link them to a reference knowledge base.
