Search Results for author:
Found 32 papers, 11 papers with code
The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank
While previous work has tackled this problem with models trained specifically to do fill in the blank, a more useful model is one that can effectively perform _both_ FitB and continuation tasks.
Scalable Extraction of Training Data from (Production) Language Models
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset.
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System
Neural language models are increasingly deployed into APIs and websites that allow a user to pass in a prompt and receive generated text.
Prompts Should not be Seen as Secrets: Systematically Measuring Prompt Extraction Attack Success
The prompts used by companies to guide their models are often treated as secrets, to be hidden from the user making the query.
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
Second, we explore the effect of quality and toxicity filters, showing a trade-off between performance on standard benchmarks and risk of toxic generations.
Extracting Training Data from Diffusion Models
Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.
Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text
As text generated by large language models proliferates, it becomes vital to understand how humans engage with such text, and whether or not they are able to detect when the text they are reading did not originate with a human writer.
Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers
Recent developments in natural language generation (NLG) using neural language models have brought us closer than ever to the goal of building AI-powered creative writing tools.
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Studying data memorization in neural language models helps us understand the risks (e. g., to privacy or copyright) associated with models regurgitating training data and aids in the development of countermeasures.
Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence
In this paper, we frame D&D specifically as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history.
Measuring Forgetting of Memorized Training Examples
In memorization, models overfit specific training examples and become susceptible to privacy attacks.
The Case for a Single Model that can Both Generate Continuations and Fill in the Blank
The task of inserting text into a specified position in a passage, known as fill in the blank (FitB), is useful for a variety of applications where writers interact with a natural language generation (NLG) system to craft text.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Natural Language Inference
on RTE
Creating Multimedia Summaries Using Tweets and Videos
While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time.
Quantifying Memorization Across Neural Language Models
Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim.
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets
We use our method to curate SynthBio - a new evaluation set for WikiBio - composed of structured attribute lists describing fictional individuals, mapped to natural language biographies.
A Recipe For Arbitrary Text Style Transfer with Large Language Models
In this paper, we leverage large language models (LMs) to perform zero-shot text style transfer.
Wordcraft: a Human-AI Collaborative Editor for Story Writing
As neural language models grow in effectiveness, they are increasingly being applied in real-world settings.
Deduplicating Training Data Makes Language Models Better
As a result, over 1% of the unprompted output of language models trained on these datasets is copied verbatim from the training data.
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text
In recent years, large neural networks for natural language generation (NLG) have made leaps and bounds in their ability to generate fluent text.
COVI White Paper
Manual contact tracing of Covid-19 cases has significant challenges that limit the ability of public health authorities to minimize community infections.
Toward Better Storylines with Sentence-Level Language Models
We propose a sentence-level language model which selects the next sentence in a story from a finite set of fluent alternatives.
Trading Off Diversity and Quality in Natural Language Generation
For open-ended language generation tasks such as storytelling and dialogue, choosing the right decoding algorithm is critical to controlling the tradeoff between generation quality and diversity.
Contact Tracing Mobile Apps for COVID-19: Privacy Considerations and Related Trade-offs
Importantly, though we discuss potential modifications, this document is not meant as a formal research paper, but instead is a response to some of the privacy characteristics of direct contact tracing apps like TraceTogether and an early-stage Request for Comments to the community.
Cryptography and Security
Automatic Detection of Generated Text is Easiest when Humans are Fooled
Recent advancements in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text.
Comparison of Diverse Decoding Methods from Conditional Language Models
While conditional language models have greatly improved in their ability to output high-quality natural language, many NLP applications benefit from being able to generate a diverse set of candidate sequences.
ChatEval: A Tool for Chatbot Evaluation
We introduce a unified framework for human evaluation of chatbots that augments existing tools and provides a web-based hub for researchers to share and compare their dialog systems.
Unsupervised Hierarchical Story Infilling
Story infilling involves predicting words to go into a missing span from a story.
Learning Translations via Images with a Massively Multilingual Image Dataset
To facilitate research on the task, we introduce a large-scale multilingual corpus of images, each labeled with the word it represents.
Understanding image motion with group representations
Our results demonstrate that this representation is useful for learning motion in the general setting where explicit labels are difficult to obtain.
