Search Results for author:
Found 25 papers, 12 papers with code
Online Adaptation of Language Models with a Memory of Amortized Contexts
We propose an amortized feature extraction and memory-augmentation approach to compress and extract information from new documents into compact modulations stored in a memory bank.
A Critical Evaluation of AI Feedback for Aligning Large Language Models
RLAIF first performs supervised fine-tuning (SFT) using demonstrations from a teacher model and then further fine-tunes the model with reinforcement learning (RL), using feedback from a critic model.
RLVF: Learning from Verbal Feedback without Overgeneralization
The diversity of contexts in which large language models (LLMs) are deployed requires the ability to modify or customize default model behaviors to incorporate nuanced requirements and preferences.
Fine-tuning Language Models for Factuality
The fluency and creativity of large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines.
An Emulator for Fine-Tuning Large Language Models using Small Language Models
To aid in doing so, we introduce a novel technique for decoupling the knowledge and skills gained in these two stages, enabling a direct answer to the question, "What would happen if we combined the knowledge learned by a large model during pre-training with the knowledge learned by a small model during fine-tuning (or vice versa)?"
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF).
Meta-Learning Online Adaptation of Language Models
We meta-train a small, autoregressive model to reweight the language modeling loss for each token during online fine-tuning, with the objective of maximizing the out-of-date base question-answering model's ability to answer questions about a document after a single weighted gradient step.
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
A trustworthy real-world prediction system should produce well-calibrated confidence scores; that is, its confidence in an answer should be indicative of the likelihood that the answer is correct, enabling deferral to an expert in cases of low-confidence predictions.
RECKONING: Reasoning through Dynamic Knowledge Encoding
In the outer loop, the model learns to use the updated weights to reproduce and answer reasoning questions about the memorized knowledge.
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
In this paper, we identify a property of the structure of an LLM's probability function that is useful for such detection.
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models
A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems.
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference
While large pre-trained language models are powerful, their predictions often lack logical consistency across test inputs.
Memory-Based Model Editing at Scale
We find that only SERAC achieves high performance on all three problems, consistently outperforming existing approaches to model editing by a significant margin.
Fast Model Editing at Scale
To enable easy post-hoc editing at scale, we propose Model Editor Networks using Gradient Decomposition (MEND), a collection of small auxiliary editing networks that use a single desired input-output pair to make fast, local edits to a pre-trained model's behavior.
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation
However, goal images also have a number of drawbacks: they are inconvenient for humans to provide, they can over-specify the desired behavior leading to a sparse reward signal, or under-specify task information in the case of non-goal reaching tasks.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Offline Meta-Reinforcement Learning with Advantage Weighting
That is, in offline meta-RL, we meta-train on fixed, pre-collected data from several tasks in order to adapt to a new task with a very small amount (less than 5 trajectories) of data from the new task.
Higher Order Function Networks for View Planning and Multi-View Reconstruction
In contrast to offline methods which require a 3D model of the object as input or online methods which rely on only local measurements, our method uses a neural network which encodes shape information for a large number of objects.
Mint: Matrix-Interleaving for Multi-Task Learning
Deep learning enables training of large and flexible function approximators from scratch at the cost of large amounts of data.
QXplore: Q-Learning Exploration by Maximizing Temporal Difference Error
We implement the objective with an adversarial Q-learning method in which Q and Qx are the action-value functions for extrinsic and secondary rewards, respectively.
Higher-Order Function Networks for Learning Composable 3D Object Representations
We present a new approach to 3D object representation where a neural network encodes the geometry of an object directly into the weights and biases of a second 'mapping' network.
Reward Prediction Error as an Exploration Objective in Deep RL
We then propose a deep reinforcement learning method, QXplore, which exploits the temporal difference error of a Q-function to solve hard exploration tasks in high-dimensional MDPs.
Siamese Encoding and Alignment by Multiscale Learning with Self-Supervision
We show that siamese encoding enables more accurate alignment than the image pyramids of SPyNet, a previous deep learning approach to coarse-to-fine alignment.
Q-Learning for Continuous Actions with Cross-Entropy Guided Policies
CGP aims to combine the stability and performance of iterative sampling policies with the low computational cost of a policy network.
