Search Results for author:
Found 25 papers, 20 papers with code
CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation
CowPilot reduces the number of steps humans need to perform by allowing agents to propose next steps, while users are able to pause, reject, or take alternative actions.
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
We interact with computers on an everyday basis, be it in everyday life or work, and many aspects of work can be done entirely with access to a computer and the Internet.
Beyond Browsing: API-Based Web Agents
To do so, we propose two varieties of agents: (1) an API-calling agent that attempts to perform online tasks through APIs only, similar to traditional coding agents, and (2) a Hybrid Agent that can interact with online data through both web browsing and APIs.
Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents
We study an open question in this work: does the desired safety refusal, typically enforced in chat contexts, generalize to non-chat and agentic use cases?
Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale
LLMs can now act as autonomous agents that interact with digital environments and complete specific objectives (e. g., arranging an online meeting).
WebCanvas: Benchmarking Web Agents in Online Environments
To bridge this gap, we introduce WebCanvas, an innovative online evaluation framework for web agents that effectively addresses the dynamic nature of web interactions.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity.
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Through extensive quantitative and qualitative analysis, we identify several limitations of text-only LLM agents, and reveal gaps in the capabilities of state-of-the-art multimodal language agents.
WebArena: A Realistic Web Environment for Building Autonomous Agents
Building upon our environment, we release a set of benchmark tasks focusing on evaluating the functional correctness of task completions.
Hierarchical Prompting Assists Large Language Model on Web Navigation
Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks.
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation
Many recent advances in natural language generation have been fueled by training large language models on internet-scale data.
CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
We release five language-specific pretrained models to use with our publicly available code.
Causal Reasoning of Entities and Events in Procedural Texts
By injecting the causal relations between entities and events as intermediate reasoning steps in our representation, we further boost the performance to . 67 F1.
Execution-Based Evaluation for Open-Domain Code Generation
To extend the scope of coding queries to more realistic settings, we propose ODEX, the first Open-Domain EXecution-based natural language (NL) to Python code generation dataset.
PAL: Program-aided Language Models
Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem.
Ranked #43 on
Math Word Problem Solving
on MATH
Language Models of Code are Few-Shot Commonsense Learners
In all these natural language tasks, we show that using our approach, a code generation LM (CODEX) outperforms natural-LMs that are fine-tuned on the target task (e. g., T5) and other strong LMs such as GPT-3 in the few-shot setting.
DocPrompting: Generating Code by Retrieving the Docs
Publicly available source-code libraries are continuously growing and changing.
MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages
While there has been a recent burgeoning of applications at the intersection of natural and programming languages, such as code generation and code summarization, these applications are usually English-centric.
Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
To this end, we develop a simple and efficient method that links steps (e. g., "purchase a camera") in an article to other articles with similar goals (e. g., "how to choose a camera"), recursively constructing the KB.
Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language
When humans conceive how to perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks.
Soft Gazetteers for Low-Resource Named Entity Recognition
However, designing such features for low-resource languages is challenging, because exhaustive entity gazetteers do not exist in these languages.
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts.
Towards Zero-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) grounds named entities in a source language to an English Knowledge Base (KB), such as Wikipedia.
Improving Robustness of Neural Machine Translation with Multi-task Learning
While neural machine translation (NMT) achieves remarkable performance on clean, in-domain text, performance is known to degrade drastically when facing text which is full of typos, grammatical errors and other varieties of noise.
Aggregated Semantic Matching for Short Text Entity Linking
The task of entity linking aims to identify concepts mentioned in a text fragments and link them to a reference knowledge base.
