2 code implementations • EMNLP 2016 • Ye Zhang, Iain Marshall, Byron C. Wallace
We present a new Convolutional Neural Network (CNN) model for text classification that jointly exploits labels on documents and their component sentences.
1 code implementation • 14 Jun 2016 • Ye Zhang, Matthew Lease, Byron C. Wallace
We also show that, as expected, the method quickly learns discriminative word embeddings.
3 code implementations • CONLL 2016 • Silvio Amir, Byron C. Wallace, Hao Lyu, Paula Carvalho Mário J. Silva
We introduce a deep neural network for automated sarcasm detection.
no code implementations • 18 Nov 2016 • Ye Zhang, Md Mustafizur Rahman, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek Khetan, Tyler McDonnell, An Thanh Nguyen, Dan Xu, Byron C. Wallace, Matthew Lease
A recent "third wave" of Neural Network (NN) approaches now delivers state-of-the-art performance in many machine learning tasks, spanning speech recognition, computer vision, and natural language processing.
no code implementations • ACL 2017 • Ye Zhang, Matthew Lease, Byron C. Wallace
A fundamental advantage of neural models for NLP is their ability to learn representations from scratch.
1 code implementation • 30 Apr 2017 • Silvio Amir, Glen Coppersmith, Paula Carvalho, Mário J. Silva, Byron C. Wallace
Our experimental results demonstrate that the user embeddings capture similarities between users with respect to mental conditions, and are predictive of mental health.
no code implementations • 21 Sep 2017 • Zhiguo Yu, Byron C. Wallace, Todd Johnson, Trevor Cohen
In this paper, we present a method that retrofits distributional context vector representations of biomedical concepts using structural information from the UMLS Metathesaurus, such that the similarity between vector representations of linked concepts is augmented.
1 code implementation • EMNLP 2018 • Sarthak Jain, Edward Banner, Jan-Willem van de Meent, Iain J. Marshall, Byron C. Wallace
We propose a method for learning disentangled representations of texts that code for distinct and complementary aspects, with the aim of affording efficient model transfer and interpretability.
2 code implementations • ACL 2018 • Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain J. Marshall, Ani Nenkova, Byron C. Wallace
We present a corpus of 5, 000 richly annotated abstracts of medical articles describing clinical randomized controlled trials.
no code implementations • IJCNLP 2019 • David Lowell, Zachary C. Lipton, Byron C. Wallace
Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget.
1 code implementation • EMNLP 2018 • Gaurav Singh, James Thomas, Iain J. Marshall, John Shawe-Taylor, Byron C. Wallace
We propose a model for tagging unstructured texts with an arbitrary number of terms drawn from a tree-structured vocabulary (i. e., an ontology).
no code implementations • 12 Dec 2018 • Babak Esmaeili, Hongyi Huang, Byron C. Wallace, Jan-Willem van de Meent
We present Variational Aspect-based Latent Topic Allocation (VALTA), a family of autoencoding topic models that learn aspect-based representations of reviews.
8 code implementations • NAACL 2019 • Sarthak Jain, Byron C. Wallace
Attention mechanisms have seen wide adoption in neural NLP models.
2 code implementations • NAACL 2019 • Eric Lehman, Jay DeYoung, Regina Barzilay, Byron C. Wallace
In this paper, we present a new task and corpus for making this unstructured evidence actionable.
no code implementations • WS 2019 • Sarthak Jain, Ramin Mohammadi, Byron C. Wallace
In this work we perform experiments to explore this question using two EMR corpora and four different predictive tasks, that: (i) inclusion of attention mechanisms is critical for neural encoder modules that operate over notes fields in order to yield competitive performance, but, (ii) unfortunately, while these boost predictive performance, it is decidedly less clear whether they provide meaningful support for predictions.
no code implementations • NAACL 2019 • Yinfei Yang, Oshin Agarwal, Chris Tar, Byron C. Wallace, Ani Nenkova
Experiments on a complex biomedical information extraction task using expert and lay annotators show that: (i) simply excluding from the training data instances predicted to be difficult yields a small boost in performance; (ii) using difficulty scores to weight instances during training provides further, consistent gains; (iii) assigning instances predicted to be difficult to domain experts is an effective strategy for task routing.
no code implementations • 28 Jun 2019 • Ramin Mohammadi, Sarthak Jain, Stephen Agboola, Ramya Palacholla, Sagar Kamarthi, Byron C. Wallace
We develop machine learning models (logistic regression and recurrent neural networks) to stratify patients with respect to the risk of exhibiting uncontrolled hypertension within the coming three-month period.
2 code implementations • ACL 2020 • Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, Byron C. Wallace
We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i. e., the degree to which provided rationales influenced the corresponding predictions).
1 code implementation • 8 Apr 2020 • Oshin Agarwal, Yinfei Yang, Byron C. Wallace, Ani Nenkova
We propose a method for auditing the in-domain robustness of systems, focusing specifically on differences in performance due to the national origin of entities.
no code implementations • 9 Apr 2020 • Denis Jered McInerney, Borna Dabiri, Anne-Sophie Touret, Geoffrey Young, Jan-Willem van de Meent, Byron C. Wallace
We propose and evaluate models that extract relevant text snippets from patient records to provide a rough case summary intended to aid physicians considering one or more diagnoses.
no code implementations • CL (ACL) 2021 • Oshin Agarwal, Yinfei Yang, Byron C. Wallace, Ani Nenkova
We examine these questions by contrasting the performance of several variants of LSTM-CRF architectures for named entity recognition, with some provided only representations of the context as features.
2 code implementations • ACL 2020 • Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, Byron C. Wallace
In NLP this often entails extracting snippets of an input text `responsible for' corresponding model output; when such a snippet comprises tokens that indeed informed the model's prediction, it is a faithful explanation.
1 code implementation • WS 2020 • Jay DeYoung, Eric Lehman, Ben Nye, Iain J. Marshall, Byron C. Wallace
Ideally, we could consult a database of evidence gleaned from clinical trials to answer such questions.
1 code implementation • ACL 2020 • Xiaochuang Han, Byron C. Wallace, Yulia Tsvetkov
In this work, we investigate the use of influence functions for NLP, providing an alternative approach to interpreting neural text classifiers.
no code implementations • AKBC 2020 • Somin Wadhwa, Kanhua Yin, Kevin S. Hughes, Byron C. Wallace
We propose and evaluate several model variants, including a transformer-based joint entity and relation extraction model to extract <germline mutation, risk-estimate>} pairs.
1 code implementation • ACL 2020 • Benjamin E. Nye, Ani Nenkova, Iain J. Marshall, Byron C. Wallace
We apply the system at scale to all reports of randomized controlled trials indexed in MEDLINE, powering the automatic generation of evidence maps, which provide a global view of the efficacy of different interventions combining data from all relevant clinical trials on a topic.
2 code implementations • 25 Aug 2020 • Byron C. Wallace, Sayantan Saha, Frank Soboczenski, Iain J. Marshall
We enlist medical professionals to evaluate generated summaries, and we find that modern summarization systems yield consistently fluent and relevant synopses, but that they are not always factual.
no code implementations • 7 Oct 2020 • Benjamin E. Nye, Jay DeYoung, Eric Lehman, Ani Nenkova, Iain J. Marshall, Byron C. Wallace
Here we consider the end-to-end task of both (a) extracting treatments and outcomes from full-text articles describing clinical trials (entity identification) and, (b) inferring the reported results for the former with respect to the latter (relation extraction).
no code implementations • EMNLP 2021 • David Lowell, Brian E. Howard, Zachary C. Lipton, Byron C. Wallace
Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding 'noised' examples produced via data augmentation.
1 code implementation • NAACL 2021 • Pouya Pezeshkpour, Sarthak Jain, Byron C. Wallace, Sameer Singh
Instance attribution methods constitute one means of accomplishing these goals by retrieving training instances that (may have) led to a particular prediction.
1 code implementation • NAACL 2021 • Ashwin Devaraj, Iain J. Marshall, Byron C. Wallace, Junyi Jessy Li
In this work we introduce a new corpus of parallel texts in English comprising technical and lay summaries of all published evidence pertaining to different clinical topics.
no code implementations • NAACL 2021 • Silvio Amir, Jan-Willem van de Meent, Byron C. Wallace
Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s).
no code implementations • EMNLP 2021 • Xiongyi Zhang, Jan-Willem van de Meent, Byron C. Wallace
Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a multitude of downstream tasks.
4 code implementations • NAACL 2021 • Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, Byron C. Wallace
The cost of training such models (and the necessity of data access to do so) coupled with their utility motivates parameter sharing, i. e., the release of pretrained models such as ClinicalBERT.
2 code implementations • Findings (ACL) 2021 • Diego Garcia-Olano, Yasumasa Onoe, Ioana Baldini, Joydeep Ghosh, Byron C. Wallace, Kush R. Varshney
Pre-trained language models induce dense entity representations that offer strong performance on entity-centric NLP tasks, but such representations are not immediately interpretable.
no code implementations • Findings (ACL) 2022 • Pouya Pezeshkpour, Sarthak Jain, Sameer Singh, Byron C. Wallace
In this paper we evaluate use of different attribution methods for aiding identification of training data artifacts.
no code implementations • EMNLP (MRQA) 2021 • Gregory Kell, Iain J. Marshall, Byron C. Wallace, Andre Jaun
Medical question answering (QA) systems have the potential to answer clinicians uncertainties about treatment and diagnosis on demand, informed by the latest evidence.
1 code implementation • ACL 2022 • Ashwin Devaraj, William Sheffield, Byron C. Wallace, Junyi Jessy Li
We find that errors often appear in both that are not captured by existing evaluation metrics, motivating a need for research into ensuring the factual accuracy of automated simplification models.
1 code implementation • NAACL (ClinicalNLP) 2022 • Eric Lehman, Vladislav Lialin, Katelyn Y. Legaspi, Anne Janelle R. Sy, Patricia Therese S. Pile, Nicole Rose I. Alberto, Richard Raymund R. Ragasa, Corinna Victoria M. Puyat, Isabelle Rose I. Alberto, Pia Gabrielle I. Alfonso, Marianne Taliño, Dana Moukheiber, Byron C. Wallace, Anna Rumshisky, Jenifer J. Liang, Preethi Raghavan, Leo Anthony Celi, Peter Szolovits
The questions are generated by medical experts from 100+ MIMIC-III discharge summaries.
no code implementations • 12 Oct 2022 • Denis Jered McInerney, Geoffrey Young, Jan-Willem van de Meent, Byron C. Wallace
We compare alignments from a state-of-the-art multimodal (image and text) model for EHR with human annotations that link image regions to sentences.
no code implementations • 14 Oct 2022 • Nikita Salkar, Thomas Trikalinos, Byron C. Wallace, Ani Nenkova
In a regression analysis, we find that the three architectures have different propensities for repeating content across output summaries for inputs, with BART being particularly prone to self-repetition.
1 code implementation • 22 Oct 2022 • Zhaoyue Sun, Jiazheng Li, Gabriele Pergola, Byron C. Wallace, Bino John, Nigel Greene, Joseph Kim, Yulan He
The primary goal of drug safety researchers and regulators is to promptly identify adverse drug reactions.
1 code implementation • 25 Oct 2022 • Sarthak Jain, Varun Manjunatha, Byron C. Wallace, Ani Nenkova
We show the practical utility of segment influence by using the method to identify systematic annotation errors in two named entity recognition corpora.
1 code implementation • 3 Dec 2022 • Diego Garcia-Olano, Yasumasa Onoe, Joydeep Ghosh, Byron C. Wallace
However, while fine-tuning sparse, interpretable representations improves accuracy on downstream tasks, it destroys the semantics of the dimensions which were enforced in pre-training.
no code implementations • 31 Jan 2023 • Jay DeYoung, Stephanie C. Martinez, Iain J. Marshall, Byron C. Wallace
In this paper we ask: To what extent do modern multi-document summarization models implicitly perform this type of synthesis?
1 code implementation • 4 Feb 2023 • Jinghan Yang, Sarthak Jain, Byron C. Wallace
We consider the problem of identifying a minimal subset of training data $\mathcal{S}_t$ such that if the instances comprising $\mathcal{S}_t$ had been removed prior to training, the categorization of a given test point $x_t$ would have been different.
1 code implementation • 11 Feb 2023 • Junru Lu, Jiazheng Li, Byron C. Wallace, Yulan He, Gabriele Pergola
In this work, we propose a summarize-then-simplify two-stage strategy, which we call NapSS, identifying the relevant content to simplify while ensuring that the original narrative flow is preserved.
no code implementations • 23 Feb 2023 • Denis Jered McInerney, Geoffrey Young, Jan-Willem van de Meent, Byron C. Wallace
We propose CHiLL (Crafting High-Level Latents), an approach for natural-language specification of features for linear models.
no code implementations • 7 Mar 2023 • Sanjana Ramprasad, Denis Jered McInerney, Iain J. Marshal, Byron C. Wallace
We present TrialsSummarizer, a system that aims to automatically summarize evidence presented in the set of randomized controlled trials most relevant to a given query.
no code implementations • 5 May 2023 • Somin Wadhwa, Jay DeYoung, Benjamin Nye, Silvio Amir, Byron C. Wallace
However, results from RCTs are presented in (often unstructured) natural language articles describing the design, execution, and outcomes of trials; clinicians must manually extract findings pertaining to interventions and outcomes of interest from such articles.
no code implementations • 8 May 2023 • Somin Wadhwa, Silvio Amir, Byron C. Wallace
Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text.
1 code implementation • 10 May 2023 • Chantal Shaib, Millicent L. Li, Sebastian Joseph, Iain J. Marshall, Junyi Jessy Li, Byron C. Wallace
Large language models, particularly GPT-3, are able to produce high quality summaries of general domain news articles in few- and zero-shot settings.
1 code implementation • 19 May 2023 • Hye Sun Yun, Iain J. Marshall, Thomas A. Trikalinos, Byron C. Wallace
We conducted 16 interviews with international systematic review experts to characterize the perceived utility and risks of LLMs in the specific context of medical evidence reviews.
1 code implementation • 21 May 2023 • Sebastian Joseph, Kathryn Kazanas, Keziah Reina, Vishnesh J. Ramanathan, Wei Xu, Byron C. Wallace, Junyi Jessy Li
This work addresses this limitation via multilingual simplification, i. e., directly simplifying complex texts into simplified texts in multiple languages.
1 code implementation • 23 May 2023 • Lucy Lu Wang, Yulia Otmakhova, Jay DeYoung, Thinh Hung Truong, Bailey E. Kuehl, Erin Bransom, Byron C. Wallace
We analyze how automated summarization evaluation metrics correlate with lexical features of generated summaries, to other automated metrics including several we propose in this work, and to aspects of human-assessed summary quality.
1 code implementation • 23 May 2023 • Kundan Krishna, Prakhar Gupta, Sanjana Ramprasad, Byron C. Wallace, Jeffrey P. Bigham, Zachary C. Lipton
While the NLP community has produced numerous summarization benchmarks, none provide the rich annotations required to simultaneously address many important problems related to control and reliability.
1 code implementation • 20 Jun 2023 • Jiuding Sun, Chantal Shaib, Byron C. Wallace
To answer the former, we collect a set of 319 instructions manually written by NLP practitioners for over 80 unique tasks included in widely used benchmarks, and we evaluate the variance and average performance of these instructions as compared to instruction phrasings observed during instruction fine-tuning.
no code implementations • 8 Sep 2023 • Hiba Ahsan, Denis Jered McInerney, Jisoo Kim, Christopher Potter, Geoffrey Young, Silvio Amir, Byron C. Wallace
Our method entails tasking an LLM to infer whether a patient has, or is at risk of, a particular condition on the basis of associated notes; if so, we ask the model to summarize the supporting evidence.
no code implementations • 23 Oct 2023 • Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, David Bau
Using causal mediation analysis on a diverse range of in-context-learning (ICL) tasks, we find that a small number attention heads transport a compact representation of the demonstrated task, which we call a function vector (FV).
no code implementations • 8 Nov 2023 • Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C. Wallace, David Bau
More concretely, in this paper we ask: Given a hidden (internal) representation of a single token at position $t$ in an input, can we reliably anticipate the tokens that will appear at positions $\geq t + 2$?
no code implementations • 19 Nov 2023 • Gongbo Zhang, Qiao Jin, Denis Jered McInerney, Yong Chen, Fei Wang, Curtis L. Cole, Qian Yang, Yanshan Wang, Bradley A. Malin, Mor Peleg, Byron C. Wallace, Zhiyong Lu, Chunhua Weng, Yifan Peng
Evidence-based medicine promises to improve the quality of healthcare by empowering medical decisions and practices with the best available evidence.
1 code implementation • 29 Jan 2024 • Jan Trienes, Sebastian Joseph, Jörg Schlötterer, Christin Seifert, Kyle Lo, Wei Xu, Byron C. Wallace, Junyi Jessy Li
Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness.
no code implementations • 15 Feb 2024 • Denis Jered McInerney, William Dickinson, Lucy C. Flynn, Andrea C. Young, Geoffrey S. Young, Jan-Willem van de Meent, Byron C. Wallace
In this work we propose a method to use LLMs to identify pieces of evidence in patient EHR data that indicate increased or decreased risk of specific diagnoses; our ultimate aim is to increase access to evidence and reduce diagnostic errors.
no code implementations • 19 Feb 2024 • Kundan Krishna, Sanjana Ramprasad, Prakhar Gupta, Byron C. Wallace, Zachary C. Lipton, Jeffrey P. Bigham
We present GenAudit -- a tool intended to assist fact-checking LLM responses for document-grounded tasks.
1 code implementation • 24 Feb 2024 • Zhaoyue Sun, Gabriele Pergola, Byron C. Wallace, Yulan He
With the advent of large language models (LLMs), there has been growing interest in exploring their potential for medical applications.
no code implementations • 28 Feb 2024 • Chantal Shaib, Joe Barrow, Alexa F. Siu, Byron C. Wallace, Ani Nenkova
Modern instruction-tuned models have become highly capable in text generation tasks such as summarization, and are expected to be released at a steady pace.
no code implementations • 1 Mar 2024 • Chantal Shaib, Joe Barrow, Jiuding Sun, Alexa F. Siu, Byron C. Wallace, Ani Nenkova
The applicability of scores extends beyond analysis of generative models; for example, we highlight applications on instruction-tuning datasets and human-produced texts.