no code implementations • 16 Jun 2023 • Animesh Nighojkar, Antonio Laverghetta Jr., John Licato
Natural Language Inference (NLI) has been a cornerstone task in evaluating language models' inferential reasoning capabilities.
no code implementations • 19 Dec 2022 • John Licato, Logan Fields, Zaid Marji
Open-textured terms in written rules are typically settled through interpretive argumentation.
no code implementations • 20 Aug 2022 • Animesh Nighojkar, Anna Khlyzova, John Licato
We report preliminary evidence suggesting that, despite obvious implementational differences in how people and TLMs learn and use language, TLMs can be used to identify individual differences in human fluency task behaviors better than existing computational models, and may offer insights into human memory retrieval strategies -- cognitive process not typically considered to be the kinds of things TLMs can model.
no code implementations • 12 May 2022 • Antonio Laverghetta Jr., Animesh Nighojkar, Jamshidbek Mirzakhalov, John Licato
In other words, can LMs be of use in predicting the psychometric properties of test items, when those items are given to human participants?
1 code implementation • ACL 2022 • Antonio Laverghetta Jr., John Licato
Reasoning using negation is known to be difficult for transformer-based language models.
no code implementations • 26 Oct 2021 • John Licato
But here I refer to the kinds of rules expressed in human language that are the basis of laws, regulations, codes of conduct, ethical guidelines, and so on.
1 code implementation • WMT (EMNLP) 2021 • Jamshidbek Mirzakhalov, Anoop Babu, Aigiz Kunafin, Ahsan Wahab, Behzod Moydinboyev, Sardana Ivanova, Mokhiyakhon Uzokova, Shaxnoza Pulatova, Duygu Ataman, Julia Kreutzer, Francis Tyers, Orhan Firat, John Licato, Sriram Chellappan
Then, we train 26 bilingual baselines as well as a multi-way neural MT (MNMT) model using the corpus and perform an extensive analysis using automatic metrics as well as human evaluations.
1 code implementation • ACL 2021 • Animesh Nighojkar, John Licato
Can we teach them instead to identify paraphrases in a way that draws on the inferential properties of the sentences, and is not over-reliant on lexical and syntactic similarities of a sentence pair?
Ranked #1 on Paraphrase Identification on AP
1 code implementation • Joint Conference on Lexical and Computational Semantics 2021 • Antonio Laverghetta Jr., Animesh Nighojkar, Jamshidbek Mirzakhalov, John Licato
We then use the responses to calculate standard psychometric properties of the items in the diagnostic test, using the human responses and the LM responses separately.
1 code implementation • Asian Chapter of the Association for Computational Linguistics 2020 • Antonio Laverghetta Jr., Jamshidbek Mirzakhalov, John Licato
Curriculum learning, a training strategy where training data are ordered based on their difficulty, has been shown to improve performance and reduce training time on various NLP tasks.
no code implementations • 6 May 2020 • Elijah Malaby, Bradley Dragun, John Licato
There is an increasing interest in applying recent advances in AI to automated reasoning, as it may provide useful heuristics in reasoning over formalisms in first-order, second-order, or even meta-logics.
1 code implementation • 6 May 2020 • Zaid Marji, Animesh Nighojkar, John Licato
The Natural Language Inference (NLI) task is an important task in modern NLP, as it asks a broad question to which many other tasks may be reducible: Given a pair of sentences, does the first entail the second?
no code implementations • 5 Nov 2019 • John Licato, Zaid Marji, Sophia Abraham
Artificially intelligent systems, given a set of non-trivial ethical rules to follow, will inevitably be faced with scenarios which call into question the scope of those rules.