Search Results for author:
Found 37 papers, 19 papers with code
PaLM: Scaling Language Modeling with Pathways
To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
Ranked #1 on
Question Answering
on MultiRC
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
We summarize, categorize, and discuss how researchers have been addressing these issues and what their findings mean for the current state of model evaluations.
Diagnosing AI Explanation Methods with Folk Concepts of Behavior
When explaining AI behavior to humans, how is the communicated information being comprehended by the human explainee, and does it match what the explanation attempted to communicate?
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.
SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets
We use our method to curate SynthBio - a new evaluation set for WikiBio - composed of structured attribute lists describing fictional individuals, mapped to natural language biographies.
LMdiff: A Visual Diff Tool to Compare Language Models
While different language models are ubiquitous in NLP, it is hard to contrast their outputs and identify which contexts one can handle better than the other.
Learning Compact Metrics for MT
Recent developments in machine translation and multilingual text generation have led researchers to adopt trained metrics such as COMET or BLEURT, which treat evaluation as a regression problem and use representations from multilingual pre-trained models such as XLM-RoBERTa or mBERT.
Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards
Developing documentation guidelines and easy-to-use templates for datasets and models is a challenging task, especially given the variety of backgrounds, skills, and incentives of the people involved in the building of natural language processing (NLP) tools.
Automatic Construction of Evaluation Suites for Natural Language Generation Datasets
By applying this framework to the GEM generation benchmark, we propose an evaluation suite made of 80 challenge sets, demonstrate the kinds of analyses that it enables and shed light onto the limits of current generation models.
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models
Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts.
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Ranked #1 on
Data-to-Text Generation
on WebNLG ru
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Investigating Gender Bias in Language Models Using Causal Mediation Analysis
As a case study, we apply this methodology to analyzing gender bias in pre-trained Transformer language models.
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
The quality of machine translation systems has dramatically improved over the last decade, and as a result, evaluation has become an increasingly challenging problem.
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models.
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models
Large Transformer-based language models can route and reshape complex information via their multi-headed attention mechanism.
Interpretability and Analysis in Neural NLP
While deep learning has transformed the natural language processing (NLP) field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior.
Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics
De novo therapeutic design is challenged by a vast chemical repertoire and multiple constraints, e. g., high broad-spectrum potency and low toxicity.
ToTTo: A Controlled Table-To-Text Generation Dataset
We present ToTTo, an open-domain English table-to-text dataset with over 120, 000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.
Ranked #2 on
Data-to-Text Generation
on ToTTo
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
Common methods for interpreting neural models in natural language processing typically examine either their structure or their behavior, but not both.
A Corpus for Detecting High-Context Medical Conditions in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients
In this paper, we introduce a dataset for patient phenotyping, a task that is defined as the identification of whether a patient has a given medical condition (also referred to as clinical indication or phenotype) based on their patient note.
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
We introduce three memory-augmented Recurrent Neural Networks (MARNNs) and explore their capabilities on a series of simple language modeling tasks whose solutions require stack-based mechanisms.
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models
We present exBERT, an interactive tool named after the popular BERT language model, that provides insights into the meaning of the contextual representations by matching a human-specified input to similar contexts in a large annotated dataset.
Generating Abstractive Summaries with Finetuned Language Models
Neural abstractive document summarization is commonly approached by models that exhibit a mostly extractive behavior.
Margin Call: an Accessible Web-based Text Viewer with Generated Paragraph Summaries in the Margin
We present Margin Call, a web-based text viewer that automatically generates short summaries for each paragraph of the text and displays the summaries in the margin of the text next to the corresponding paragraph.
Encoder-Agnostic Adaptation for Conditional Language Generation
Large pretrained language models have changed the way researchers approach discriminative natural language understanding tasks, leading to the dominance of approaches that adapt a pretrained model for arbitrary downstream tasks.
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference
Automation of tasks can have critical consequences when humans lose agency over decision processes.
GLTR: Statistical Detection and Visualization of Generated Text
The rapid improvement of language models has raised the specter of abuse of text generation systems.
LSTM Networks Can Perform Dynamic Counting
In this paper, we systematically assess the ability of standard recurrent networks to perform dynamic counting and to encode hierarchical representations.
Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation
Titles of short sections within long documents support readers by guiding their focus towards relevant passages and by providing anchor-points that help to understand the progression of the document.
Interactive Visual Exploration of Latent Space (IVELS) for peptide auto-encoder model selection
We present the pipeline in an interactive visual tool to enable the exploration of the metrics, analysis of the learned latent space, and selection of the best model for a given task.
E2E NLG Challenge Submission: Towards Controllable Generation of Diverse Natural Language
In natural language generation (NLG), the task is to generate utterances from a more abstract input, such as structured data.
Debugging Sequence-to-Sequence Models with Seq2Seq-Vis
Neural attention-based sequence-to-sequence models (seq2seq) (Sutskever et al., 2014; Bahdanau et al., 2014) have proven to be accurate and robust for many sequence prediction tasks.
End-to-End Content and Plan Selection for Data-to-Text Generation
Learning to generate fluent natural language from structured data with neural networks has become an common approach for NLG.
Bottom-Up Abstractive Summarization
We use this selector as a bottom-up attention step to constrain the model to likely phrases.
Ranked #4 on
Multi-Document Summarization
on Multi-News
Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
In this work, we present a visual analysis tool that allows interaction with a trained sequence-to-sequence model through each stage of the translation process.
Comparing Rule-Based and Deep Learning Models for Patient Phenotyping
We assess the performance of deep learning algorithms and compare them with classical NLP approaches.
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics.
