Search Results for author: Emma Strubell

Found 37 papers, 12 papers with code

On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling

1 code implementation EMNLP 2021 Zhisong Zhang, Emma Strubell, Eduard Hovy

Although recent developments in neural architectures and pre-trained representations have greatly increased state-of-the-art model performance on fully-supervised semantic role labeling (SRL), the task remains challenging for languages where supervised SRL training data are not abundant.

Cross-Lingual Transfer Semantic Role Labeling

Evaluating Gender Bias Transfer from Film Data

no code implementations NAACL (GeBNLP) 2022 Amanda Bertsch, Ashley Oh, Sanika Natu, Swetha Gangu, Alan W. black, Emma Strubell

We extend our analysis to a longitudinal study of bias in film dialogue over the last 110 years and find that continued pre-training on OpenSubtitles encodes additional bias into BERT.

Dialogue Generation Machine Translation +3

Comparing Span Extraction Methods for Semantic Role Labeling

1 code implementation ACL (spnlp) 2021 Zhisong Zhang, Emma Strubell, Eduard Hovy

In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL).

Semantic Role Labeling

Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models

no code implementations30 Jun 2023 Harnoor Dhingra, Preetiha Jayashanker, Sayali Moghe, Emma Strubell

Large Language Models (LLMs) are trained primarily on minimally processed web text, which exhibits the same wide range of social biases held by the humans who created that content.

Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research

no code implementations29 Jun 2023 Ji-Ung Lee, Haritz Puerto, Betty van Aken, Yuki Arase, Jessica Zosa Forde, Leon Derczynski, Andreas Rücklé, Iryna Gurevych, Roy Schwartz, Emma Strubell, Jesse Dodge

Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters.

Large Language Model Distillation Doesn't Need a Teacher

no code implementations24 May 2023 Ananya Harsh Jha, Dirk Groeneveld, Emma Strubell, Iz Beltagy

Knowledge distillation trains a smaller student model to match the output distribution of a larger teacher to maximize the end-task performance under computational constraints.

Knowledge Distillation Language Modelling +1

Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training

no code implementations22 May 2023 Zhisong Zhang, Emma Strubell, Eduard Hovy

To address this challenge we adopt an error estimator to decide the partial selection ratio adaptively according to the current model's capability.

Active Learning Structured Prediction

Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints

no code implementations29 Apr 2023 Rajshekhar Das, Jonathan Francis, Sanket Vaibhav Mehta, Jean Oh, Emma Strubell, Jose Moura

Self-training based on pseudo-labels has emerged as a dominant approach for addressing conditional distribution shifts in unsupervised domain adaptation (UDA) for semantic segmentation problems.

Semantic Segmentation Unsupervised Domain Adaptation

The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment

no code implementations13 Feb 2023 Jared Fernandez, Jacob Kahn, Clara Na, Yonatan Bisk, Emma Strubell

Increased focus on the deployment of machine learning systems has led to rapid improvements in hardware accelerator performance and neural network model efficiency.

DSI++: Updating Transformer Memory with New Documents

no code implementations19 Dec 2022 Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Jinfeng Rao, Marc Najork, Emma Strubell, Donald Metzler

In this work, we introduce DSI++, a continual learning challenge for DSI to incrementally index new documents while being able to answer queries related to both previously and newly indexed documents.

Continual Learning Natural Questions +1

Error-aware Quantization through Noise Tempering

no code implementations11 Dec 2022 Zheng Wang, Juncheng B Li, Shuhui Qu, Florian Metze, Emma Strubell

In this work, we incorporate exponentially decaying quantization-error-aware noise together with a learnable scale of task loss gradient to approximate the effect of a quantization operator.

Model Compression Quantization

Bridging Fairness and Environmental Sustainability in Natural Language Processing

no code implementations8 Nov 2022 Marius Hessenthaler, Emma Strubell, Dirk Hovy, Anne Lauscher

Fairness and environmental impact are important research directions for the sustainable development of artificial intelligence.

Dimensionality Reduction Fairness +4

A Survey of Active Learning for Natural Language Processing

1 code implementation18 Oct 2022 Zhisong Zhang, Emma Strubell, Eduard Hovy

In this work, we provide a survey of active learning (AL) for its applications in natural language processing (NLP).

Active Learning Structured Prediction

Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolution

1 code implementation14 Oct 2022 Nupoor Gandhi, Anjalie Field, Emma Strubell

Although recent neural models for coreference resolution have led to substantial improvements on benchmark datasets, transferring these models to new target domains containing out-of-vocabulary spans and requiring differing annotation schemes remains challenging.

coreference-resolution Domain Adaptation

SQuAT: Sharpness- and Quantization-Aware Training for BERT

no code implementations13 Oct 2022 Zheng Wang, Juncheng B Li, Shuhui Qu, Florian Metze, Emma Strubell

Quantization is an effective technique to reduce memory footprint, inference latency, and power consumption of deep learning models.


Measuring the Carbon Intensity of AI in Cloud Instances

no code implementations10 Jun 2022 Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, Will Buchanan

By providing unprecedented access to computational resources, cloud computing has enabled rapid growth in technologies such as machine learning, the computational demands of which incur a high energy cost and a commensurate carbon footprint.

Cloud Computing Language Modelling

Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models

no code implementations25 May 2022 Clara Na, Sanket Vaibhav Mehta, Emma Strubell

Model compression by way of parameter pruning, quantization, or distillation has recently gained popularity as an approach for reducing the computational requirements of modern deep neural network models for NLP.

Model Compression Quantization +3

An Empirical Investigation of the Role of Pre-training in Lifelong Learning

1 code implementation16 Dec 2021 Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, Emma Strubell

The lifelong learning paradigm in machine learning is an attractive alternative to the more prominent isolated learning scheme not only due to its resemblance to biological learning but also its potential to reduce energy waste by obviating excessive model re-training.

Continual Learning Image Classification

Unsupervised Domain Adaptation Via Pseudo-labels And Objectness Constraints

no code implementations29 Sep 2021 Rajshekhar Das, Jonathan Francis, Sanket Vaibhav Mehta, Jean Oh, Emma Strubell, Jose Moura

Crucially, the objectness constraint is agnostic to the ground-truth semantic segmentation labels and, therefore, remains appropriate for unsupervised adaptation settings.

Pseudo Label Semantic Segmentation +2

End-to-end Quantized Training via Log-Barrier Extensions

no code implementations1 Jan 2021 Juncheng B Li, Shuhui Qu, Xinjian Li, Emma Strubell, Florian Metze

Quantization of neural network parameters and activations has emerged as a successful approach to reducing the model size and inference time on hardware that sup-ports native low-precision arithmetic.


Energy and Policy Considerations for Deep Learning in NLP

3 code implementations ACL 2019 Emma Strubell, Ananya Ganesh, Andrew McCallum

Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.

Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?

no code implementations WS 2018 Emma Strubell, Andrew McCallum

Do unsupervised methods for learning rich, contextualized token representations obviate the need for explicit modeling of linguistic structure in neural network models for semantic role labeling (SRL)?

Semantic Role Labeling Word Embeddings

Linguistically-Informed Self-Attention for Semantic Role Labeling

1 code implementation EMNLP 2018 Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, Andrew McCallum

Unlike previous models which require significant pre-processing to prepare linguistic features, LISA can incorporate syntax using merely raw tokens as input, encoding the sequence only once to simultaneously perform parsing, predicate detection and role labeling for all predicates.

Dependency Parsing Multi-Task Learning +4

Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction

1 code implementation NAACL 2018 Patrick Verga, Emma Strubell, Andrew McCallum

Most work in relation extraction forms a prediction by looking at a short span of text within a single sentence containing a single entity pair mention.

Relation Extraction

Automatically Extracting Action Graphs from Materials Science Synthesis Procedures

no code implementations18 Nov 2017 Sheshera Mysore, Edward Kim, Emma Strubell, Ao Liu, Haw-Shiuan Chang, Srikrishna Kompella, Kevin Huang, Andrew McCallum, Elsa Olivetti

In this work, we present a system for automatically extracting structured representations of synthesis procedures from the texts of materials science journal articles that describe explicit, experimental syntheses of inorganic compounds.

Attending to All Mention Pairs for Full Abstract Biological Relation Extraction

no code implementations23 Oct 2017 Patrick Verga, Emma Strubell, Ofer Shai, Andrew McCallum

We propose a model to consider all mention and entity pairs simultaneously in order to make a prediction.

Relation Extraction

Dependency Parsing with Dilated Iterated Graph CNNs

no code implementations WS 2017 Emma Strubell, Andrew McCallum

Dependency parses are an effective way to inject linguistic knowledge into many downstream tasks, and many practitioners wish to efficiently parse sentences at scale.

Dependency Parsing

Fast and Accurate Entity Recognition with Iterated Dilated Convolutions

4 code implementations EMNLP 2017 Emma Strubell, Patrick Verga, David Belanger, Andrew McCallum

Today when many practitioners run basic NLP on the entire web and large-volume traffic, faster methods are paramount to saving time and energy costs.

NER Structured Prediction

Multilingual Relation Extraction using Compositional Universal Schema

1 code implementation NAACL 2016 Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, Andrew McCallum

In response, this paper introduces significant further improvements to the coverage and flexibility of universal schema relation extraction: predictions for entities unseen in training and multilingual transfer learning to domains with no annotation.

Relation Extraction slot-filling +3

Learning Dynamic Feature Selection for Fast Sequential Prediction

no code implementations IJCNLP 2015 Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.

Benchmarking feature selection +6

Training for Fast Sequential Prediction Using Dynamic Feature Selection

no code implementations30 Oct 2014 Emma Strubell, Luke Vilnis, Andrew McCallum

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.

feature selection Part-Of-Speech Tagging

Cannot find the paper you are looking for? You can Submit a new open access paper.