no code implementations • EMNLP 2021 • Zhaohui Li, Yajur Tomar, Rebecca J. Passonneau
Automatic short answer grading (ASAG) is the task of assessing students’ short natural language responses to objective questions.
1 code implementation • NAACL (TextGraphs) 2021 • Yanjun Gao, Ting-Hao Huang, Rebecca J. Passonneau
We design a neural model to learn a semantic representation for clauses from graph convolution over latent representations of the subject and verb phrase.
no code implementations • NAACL (ACL) 2022 • Rui Zhang, Yangfeng Ji, Yue Zhang, Rebecca J. Passonneau
We then survey the benefits and the best practices of contrastive learning for various downstream NLP applications including Text Classification, Question Answering, Summarization, Text Generation, Interpretability and Explainability, Commonsense Knowledge and Reasoning, Vision-and-Language. This tutorial intends to help researchers in the NLP and computational linguistics community to understand this emerging topic and promote future research directions of using contrastive learning for NLP applications.
no code implementations • 7 Feb 2025 • Berk Atıl, Vipul Gupta, Sarkar Snigdha Sarathi Das, Rebecca J. Passonneau
Then, we evaluate three state-of-the-art large LLMs on their ability to annotate the harmfulness of these responses.
no code implementations • 31 Oct 2024 • Zhaohui Li, Rebecca J. Passonneau
Classifier models are prevalent in natural language processing (NLP), often with high accuracy.
1 code implementation • 26 Oct 2024 • Vipul Gupta, Candace Ross, David Pantoja, Rebecca J. Passonneau, Megan Ung, Adina Williams
To address these concerns, we propose Selection Methodology for Accurate, Reduced, and Targeted (SMART) filtering, a novel approach to select a high-quality subset of examples from existing benchmark datasets by systematically removing less informative and less challenging examples.
2 code implementations • 6 Aug 2024 • Berk Atıl, Sarp Aykent, Alexa Chittams, Lisheng Fu, Rebecca J. Passonneau, Evan Radcliffe, Guru Rajan Rajagopal, Adam Sloan, Tomasz Tudrej, Ferhan Ture, Zhe Wu, Lixinyu Xu, Breck Baldwin
We investigate non-determinism in five LLMs configured to be deterministic when applied to eight common tasks in across 10 runs, in both zero-shot and few-shot settings.
no code implementations • 17 Apr 2024 • Mahsa Sheikhi Karizaki, Dana Gnesdilow, Sadhana Puntambekar, Rebecca J. Passonneau
First, we find that the main ideas in the rubric differ with respect to how much freedom they afford in explanations of the idea, thus explanation of a natural law is relatively constrained.
1 code implementation • 7 Feb 2024 • Berk Atıl, Mahsa Sheikhi Karizaki, Rebecca J. Passonneau
With an increasing focus in STEM education on critical thinking skills, science writing plays an ever more important role in curricula that stress inquiry skills.
no code implementations • 18 Oct 2023 • Pranav Narayanan Venkit, Mukund Srinath, Sanjana Gautam, Saranya Venkatraman, Vipul Gupta, Rebecca J. Passonneau, Shomir Wilson
We conduct an inquiry into the sociotechnical aspects of sentiment analysis (SA) by critically examining 189 peer-reviewed papers on their applications, models, and datasets.
1 code implementation • 24 Aug 2023 • Vipul Gupta, Pranav Narayanan Venkit, Hugo Laurençon, Shomir Wilson, Rebecca J. Passonneau
We apply CALM to 20 large language models, and find that for 2 language model series, larger parameter models tend to be more biased than smaller ones.
no code implementations • 13 Jun 2023 • Vipul Gupta, Pranav Narayanan Venkit, Shomir Wilson, Rebecca J. Passonneau
Sociodemographic bias in language models (LMs) has the potential for harm when deployed in real-world settings.
1 code implementation • ACL 2022 • Sarkar Snigdha Sarathi Das, Arzoo Katiyar, Rebecca J. Passonneau, Rui Zhang
Named Entity Recognition (NER) in Few-Shot setting is imperative for entity tagging in low resource domains.
Ranked #9 on
Few-shot NER
on Few-NERD (INTRA)
(using extra training data)
2 code implementations • ACL 2021 • Yanjun Gao, Ting-Hao, Huang, Rebecca J. Passonneau
On DeSSE, which has a more even balance of complex sentence types, our model achieves higher accuracy on the number of atomic sentences than an encoder-decoder baseline.
1 code implementation • CONLL 2019 • Yanjun Gao, Chen Sun, Rebecca J. Passonneau
Pyramid evaluation was developed to assess the content of paragraph length summaries of source texts.
1 code implementation • WS 2019 • Yanjun Gao, Alex Driban, Brennan Xavier McManus, Elena Musi, Patricia Davies, Smar Muresan, a, Rebecca J. Passonneau
We present a unique dataset of student source-based argument essays to facilitate research on the relations between content, argumentation skills, and assessment.
no code implementations • WS 2018 • Yanjun Gao, Patricia M. Davies, Rebecca J. Passonneau
Technology is transforming Higher Education learning and teaching.
no code implementations • 10 Oct 2015 • Boyi Xie, Rebecca J. Passonneau
OmniGraph, a novel representation to support a range of NLP classification tasks, integrates lexical items, syntactic dependencies and frame semantic parses into graphs.
no code implementations • IJCNLP 2015 • Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, Rebecca J. Passonneau
We propose an abstraction-based multi-document summarization framework that can construct new sentences by exploring more fine-grained syntactic units than sentences, namely, noun/verb phrases.
no code implementations • LREC 2014 • Andrea Moro, Roberto Navigli, Francesco Maria Tucci, Rebecca J. Passonneau
Finally, we estimate the quality of our annotations using both manually-tagged named entities and word senses, obtaining an accuracy of roughly 70{\%} for both named entities and word sense annotations.
no code implementations • TACL 2014 • Rebecca J. Passonneau, Bob Carpenter
Standard agreement measures for interannotator reliability are neither necessary nor sufficient to ensure a high quality corpus.
no code implementations • LREC 2012 • Gerard de Melo, Collin F. Baker, Nancy Ide, Rebecca J. Passonneau, Christiane Fellbaum
We analyze how different conceptions of lexical semantics affect sense annotations and how multiple sense inventories can be compared empirically, based on annotated text.
no code implementations • LREC 2012 • Rebecca J. Passonneau, Collin F. Baker, Christiane Fellbaum, Nancy Ide
The MASC project has produced a multi-genre corpus with multiple layers of linguistic annotation, together with a sentence corpus containing WordNet 3. 1 sense tags for 1000 occurrences of each of 100 words produced by multiple annotators, accompanied by indepth inter-annotator agreement data.