no code implementations • EMNLP (WNUT) 2020 • Ming-Cheng Ma, John P. Lalor
Automated agents (“bots”) have emerged as an ubiquitous and influential presence on social media.
1 code implementation • EMNLP 2021 • Ahmed Abbasi, David Dobolyi, John P. Lalor, Richard G. Netemeyer, Kendall Smith, Yi Yang
We also discuss the important implications of our work and resulting testbed for future NLP research on psychometrics and fairness.
no code implementations • 20 Nov 2023 • Xiaojing Duan, John P. Lalor
With the rapid advancement of machine learning models for NLP tasks, collecting high-fidelity labels from AI models is a realistic possibility.
no code implementations • 17 Nov 2023 • Yi Yang, Hanyu Duan, Ahmed Abbasi, John P. Lalor, Kar Yan Tam
Although a burgeoning literature has emerged on stereotypical bias mitigation in PLMs, such as work on debiasing gender and racial stereotyping, how such biases manifest and behave internally within PLMs remains largely unknown.
no code implementations • 2 May 2023 • Wenchang Li, Yixing Chen, Shuang Zheng, Lei Wang, John P. Lalor
We also demonstrate the interpretability of DSPN's outputs on reviews to show the pyramid structure inherent in unified sentiment analysis.
Aspect Category Detection Aspect Category Sentiment Analysis +1
no code implementations • 20 May 2022 • John P. Lalor, Hong Guo
We illustrate the measurement framework through a toy example, describe the framework and its conceptual underpinnings, and demonstrate the benefits of the framework, in particular for managers considering tradeoffs when selecting algorithms.
1 code implementation • 2 Mar 2022 • John P. Lalor, Pedro Rodriguez
py-irt is a Python library for fitting Bayesian Item Response Theory (IRT) models.
1 code implementation • ACL 2021 • Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, Jordan Boyd-Graber
While leaderboards are a straightforward ranking of NLP models, this simplicity can mask nuances in evaluation items (examples) and subjects (NLP models).
no code implementations • Findings of the Association for Computational Linguistics 2020 • John P. Lalor, Hong Yu
Curriculum learning methods typically rely on heuristics to estimate the difficulty of training examples or the ability of the model.
no code implementations • 9 Oct 2019 • Eunah Cho, He Xie, John P. Lalor, Varun Kumar, William M. Campbell
In addition, methods optimizing diversity can reduce training data in many cases to 50% with little impact on performance.
Natural Language Understanding Task-Oriented Dialogue Systems
1 code implementation • IJCNLP 2019 • John P. Lalor, Hao Wu, Hong Yu
We demonstrate a use-case for latent difficulty item parameters, namely training set filtering, and show that using difficulty to sample training data outperforms baseline methods.
no code implementations • 27 Feb 2017 • John P. Lalor, Hao Wu, Hong Yu
Often when multiple labels are obtained for a training example it is assumed that there is an element of noise that must be accounted for.
no code implementations • EMNLP 2018 • John P. Lalor, Hao Wu, Tsendsuren Munkhdalai, Hong Yu
We examine the impact of a test set question's difficulty to determine if there is a relationship between difficulty and performance.
no code implementations • EMNLP 2016 • John P. Lalor, Hao Wu, Hong Yu
Evaluation of NLP methods requires testing against a previously vetted gold-standard test set and reporting standard metrics (accuracy/precision/recall/F1).