Search Results for author:
Found 27 papers, 5 papers with code
Geometric-Averaged Preference Optimization for Soft Preference Labels
In this work, we introduce the distributional soft preference labels and improve Direct Preference Optimization (DPO) with a weighted geometric average of the LLM output likelihood in the loss function.
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability
We find that for a fixed dataset, larger and longer-trained LMs hallucinate less.
Scaling Exponents Across Parameterizations and Optimizers
Robust and effective scaling of models from small to large width typically requires the precise adjustment of many algorithmic and architectural details, such as parameterization and optimizer choices.
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
To do so, we investigate a simple self-training method based on expectation-maximization, which we call ReST$^{EM}$, where we (1) generate samples from the model and filter them using binary feedback, (2) fine-tune the model on these samples, and (3) repeat this process a few times.
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web
We show that while existing prompted LMAs (gpt-3. 5-turbo or gpt-4) achieve 94. 0% average success rate on base tasks, their performance degrades to 24. 9% success rate on compositional tasks.
Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?
We introduce and study the problem of adversarial arithmetic, which provides a simple yet challenging testbed for language model alignment.
Small-scale proxies for large-scale Transformer training instabilities
In this work, we seek ways to reproduce and study training stability and instability at smaller scales.
A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web automation.
Ranked #1 on
on Mind2Web
Multimodal Web Navigation with Instruction-Finetuned Foundation Models
The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make it difficult to leverage generalization from rich out-of-domain data.
Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration
It is unclear how scalable single-agent formulations are as we increase the complexity of the design space (e. g., full stack System-on-Chip design).
CLUTR: Curriculum Learning via Unsupervised Task Representation Learning
Recently, Unsupervised Environment Design (UED) emerged as a new paradigm for zero-shot generalization by simultaneously learning a task distribution and agent policies on the generated tasks.
Understanding HTML with Large Language Models
We contribute HTML understanding models (fine-tuned LLMs) and an in-depth analysis of their capabilities under three tasks: (i) Semantic Classification of HTML elements, (ii) Description Generation for HTML inputs, and (iii) Autonomous Web Navigation of HTML pages.
Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Different from the previous meta-rl methods trying to directly infer the unstructured task embedding, our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks, and use it as a prior to improve the task inference in testing.
Hierarchical Reinforcement Learning
Meta Reinforcement Learning
+2
Environment Generation for Zero-Shot Compositional Reinforcement Learning
We learn to generate environments composed of multiple pages or rooms, and train RL agents capable of completing wide-range of complex tasks in those environments.
Less is More: Generating Grounded Navigation Instructions from Landmarks
We study the automatic generation of navigation instructions from 360-degree images captured on indoor routes.
Targeted Environment Design from Offline Data
We formalize our approach as offline targeted environment design(OTED), which automatically learns a distribution over simulator parameters to match a provided offline dataset, and then uses the learned simulator to train an RL agent in standard online fashion.
SparseDice: Imitation Learning for Temporally Sparse Data via Regularization
We are concerned with a setting where the demonstrations comprise only a subset of state-action pairs (as opposed to the whole trajectories).
Adversarial Environment Generation for Learning to Navigate the Web
The regret objective trains the adversary to design a curriculum of environments that are "just-the-right-challenge" for the navigator agents; our results show that over time, the adversary learns to generate increasingly complex web navigation tasks.
Assessing Post-Disaster Damage from Satellite Imagery using Semi-Supervised Learning Techniques
To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected.
Learning to Navigate the Web
Even though recent approaches improve the success rate on relatively simple environments with the help of human demonstrations to guide the exploration, they still fail in environments where the set of possible instructions can reach millions.
User Modeling for Task Oriented Dialogues
We further develop several variants by utilizing a latent variable model to inject random variations into user responses to promote diversity in simulated user responses and a novel goal regularization mechanism to penalize divergence of user responses from the initial user goal.
What It Takes to Achieve 100\% Condition Accuracy on WikiSQL
The SQL queries in WikiSQL are simple: Each involves one relation and does not have any join operation.
DialSQL: Dialogue Based Structured Query Generation
The recent advance in deep learning and semantic parsing has significantly improved the translation accuracy of natural language questions to structured queries.
Recovering Question Answering Errors via Query Revision
The existing factoid QA systems often lack a post-inspection component that can help models recover from their own mistakes.
Accurate Supervised and Semi-Supervised Machine Reading for Long Documents
We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows.
Global Relation Embedding for Relation Extraction
We study the problem of textual relation embedding with distant supervision.
