Search Results for author:
Found 60 papers, 31 papers with code
IrEne-viz: Visualizing Energy Consumption of Transformer Models
IrEne is an energy prediction system that accurately predicts the interpretable inference energy consumption of a wide range of Transformer-based NLP models.
SpecNFS: A Challenge Dataset Towards Extracting Formal Models from Natural Language Specifications
Evaluations show that even when using a state-of-the-art language model, there is significant room for improvement, with the best models achieving an F1 score of only 60. 5 and 33. 3 in the named-entity-recognition and dependency-link-prediction sub-tasks, respectively.
Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks
We find that representing documents as an average of the token hidden states performs the best generally.
Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective
Such attention sinks may lead to models' over-smoothing in single-task learning and interference in sequential tasks' learning, which may compromise the models' CL performance despite their high pre-trained capabilities.
Look Hear: Gaze Prediction for Speech-directed Human Attention
To predict the gaze scanpaths in this incremental object referral task, we developed the Attention in Referral Transformer model or ART, which predicts the human fixations spurred by each word in a referring expression.
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
To remedy this gap, we built $\textbf{AppWorld Engine}$, a high-quality execution environment (60K lines of code) of 9 day-to-day apps operable via 457 APIs and populated with realistic digital activities simulating the lives of ~100 fictitious users.
On Initializing Transformers with Pre-trained Embeddings
Indeed, we find that pre-trained word embeddings from GloVe, and some sub-word embeddings extracted from language models such as T5 and mT5 fare much worse compared to random initialization.
CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans
A fundamental aspect of plans is the temporal order in which their steps needs to be executed, which reflects the underlying causal dependencies between them.
Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both?
Pre-trained language models consider the context of neighboring words and documents but lack any author context of the human generating the text.
Large Human Language Models: A Need and the Challenges
As research in human-centered NLP advances, there is a growing recognition of the importance of incorporating human and social factors into NLP models.
Modeling Complex Event Scenarios via Simple Entity-focused Questions
Event scenarios are often complex and involve multiple event sequences connected through different entity participants.
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
While using the question to retrieve relevant text from an external knowledge source helps LLMs, we observe that this one-step retrieve-and-read approach is insufficient for multi-step QA.
POQue: Asking Participant-specific Outcome Questions for a Deeper Understanding of Complex Events
Knowledge about outcomes is critical for complex event understanding but is hard to acquire.
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples
We introduce a novel semi-supervised procedure that bootstraps an NLI dataset from existing biomedical dataset that pairs mechanisms with experimental evidence in abstracts.
Ranked #1 on
Natural Language Inference
on BioNLI
Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation
Anticipating future actions in a video is useful for many autonomous and assistive technologies.
Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
PASTA: A Dataset for Modeling Participant States in Narratives
This dataset contains inferable participant states; a counterfactual perturbation to each state; and the changes to the story that would be necessary if the counterfactual were true.
Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contexts
We show how to use question decompositions to teach language models these broad reasoning skills in a robust fashion.
Enhancing Continual Learning with Global Prototypes: Counteracting Negative Representation Drift
Continual learning (CL) aims to learn a sequence of tasks over time, with data distributions shifting from one task to another.
SuMe: A Dataset Towards Summarizing Biomedical Mechanisms
We leverage this structure and create a summarization task, where the input is a collection of sentences and the main entities in an abstract, and the output includes the relationship and a sentence that summarizes the mechanism.
Human Language Modeling
Natural language is generated by people, yet traditional language modeling views words or documents as if generated independently.
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection
Much of natural language processing is focused on leveraging large capacity language models, typically trained over single messages with a task of predicting one or more tokens.
Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension
Instead, we advocate for an abstractive approach, where we propose to generate a question-focused, abstractive summary of input paragraphs and then feed it to an RC system.
MuSiQue: Multihop Questions via Single-hop Question Composition
Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts.
Don't Let Discourse Confine Your Model: Sequence Perturbations for Improved Event Language Models
Event language models represent plausible sequences of events.
Toward Diverse Precondition Generation
We propose DiP, a Diverse Precondition generation system that can generate unique and diverse preconditions.
TellMeWhy: A Dataset for Answering Why-Questions in Narratives
They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.
IrEne: Interpretable Energy Prediction for Transformers
We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models.
On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers
This informs the design of an inference-time quantization technique using both pruning and log-scaled mapping which produces only a few (e. g. $2^3$) unique values.
Bew: Towards Answering Business-Entity-Related Web Questions
We present a practical approach, called BewQA, that can answer Bew queries by mining a template of the business-related webpages and using the template to guide the search.
Open4Business(O4B): An Open Access Dataset for Summarizing Business Documents
The dataset introduces a new challenge for summarization in the business domain, requiring highly abstractive and more concise summaries as compared to other existing datasets.
Author's Sentiment Prediction
We introduce PerSenT, a dataset of crowd-sourced annotations of the sentiment expressed by the authors towards the main entities in news articles.
LANNS: A Web-Scale Approximate Nearest Neighbor Lookup System
Nearest neighbor search (NNS) has a wide range of applications in information retrieval, computer vision, machine learning, databases, and other areas.
Towards Accurate and Reliable Energy Measurement of NLP Models
In this work, we show that existing software-based energy measurements are not accurate because they do not take into account hardware differences and how resource utilization affects energy consumption.
Modeling Preconditions in Text with a Crowd-sourced Dataset
This paper introduces PeKo, a crowd-sourced annotation of preconditions between event pairs in newswire, an order of magnitude larger than prior text annotations.
Hierarchical Modeling for User Personality Prediction: The Role of Message-Level Attention
Not all documents are equally important.
Modeling Label Semantics for Predicting Emotional Reactions
Predicting how events induce emotions in the characters of a story is typically seen as a standard multi-label classification task, which usually treats labels as anonymous classes to predict.
Ranked #1 on
Emotion Classification
on ROCStories
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
It turns out that we can get by without input-wide self-attention at all layers, especially in the lower layers.
Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning
For a recent large-scale model (XLNet), we show that only 18 points out of its answer F1 score of 72 on HotpotQA are obtained through multifact reasoning, roughly the same as that of a simpler RNN baseline.
Generating Narrative Text in a Switching Dynamical System
A SLDS is a dynamical system in which the latent dynamics of the system (i. e. how the state vector transforms over time) is controlled by top-level discrete switching variables.
Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning
In this paper, we propose a novel deep learning model called Task Adaptive Activation Network (TAAN) that can automatically learn the optimal network architecture for MTL.
Tweet Classification without the Tweet: An Empirical Examination of User versus Document Attributes
NLP naturally puts a primary focus on leveraging document language, occasionally considering user attributes as supplemental.
Repurposing Entailment for Multi-Hop Question Answering Tasks
We introduce Multee, a general architecture that can effectively use entailment models for multi-hop QA tasks.
PoMo: Generating Entity-Specific Post-Modifiers in Context
Given a sentence about a target entity, the task is to automatically generate a post-modifier phrase that provides contextually relevant information about the entity.
Residualized Factor Adaptation for Community Social Media Prediction Tasks
Predictive models over social media language have shown promise in capturing community outcomes, but approaches thus far largely neglect the socio-demographic context (e. g. age, education rates, race) of the community from which the language originates.
Hierarchical Quantized Representations for Script Generation
This permits the decoder to softly decide what portions of the latent hierarchy to condition on by attending over the value embeddings for a given setting.
Fake Sentence Detection as a Training Task for Sentence Encoding
We automatically generate fake sentences by corrupting original sentences from a source collection and train the encoders to produce representations that are effective at detecting fake sentences.
The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models
Seq2Seq based neural architectures have become the go-to architecture to apply to sequence to sequence language tasks.
Controlling Decoding for More Abstractive Summaries with Copy-Based Networks
Attention-based neural abstractive summarization systems equipped with copy mechanisms have shown promising results.
Event Representations with Tensor-based Compositions
Robust and flexible event representations are important to many core areas in language understanding.
Human Centered NLP with User-Factor Adaptation
We pose the general task of user-factor adaptation {--} adapting supervised learning models to real-valued user factors inferred from a background of their language, reflecting the idea that a piece of text should be understood within the context of the user that wrote it.
MobiRNN: Efficient Recurrent Neural Network Execution on Mobile GPU
In this paper, we explore optimizations to run Recurrent Neural Network (RNN) models locally on mobile devices.
What's in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams
These explanations are used to create a fine-grained categorization of the requirements.
Markov Logic Networks for Natural Language Question Answering
In the first, we simply use the extracted science rules directly as MLN clauses.
