Search Results for author:
Found 15 papers, 11 papers with code
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses.
A Logical Fallacy-Informed Framework for Argument Generation
Despite the remarkable performance of Large Language Models (LLMs), they still struggle with generating logically sound arguments, resulting in potential risks such as spreading misinformation.
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning
In this paper, we perform a causal mediation analysis on twelve LLMs to examine how intermediate reasoning steps generated by the LLM influence the final outcome and find that LLMs do not reliably use their intermediate reasoning steps when generating an answer.
Exploring Defeasibility in Causal Reasoning
CESAR achieves a significant 69. 7% relative improvement over existing metrics, increasing from 47. 2% to 80. 1% in capturing the causal strength change brought by supporters and defeaters.
CRAB: Assessing the Strength of Causal Relationships Between Real-world Events
In this work, we present CRAB, a new Causal Reasoning Assessment Benchmark designed to evaluate causal understanding of events in real-world narratives.
CRoW: Benchmarking Commonsense Reasoning in Real-World Tasks
Recent efforts in natural language processing (NLP) commonsense reasoning research have yielded a considerable number of new datasets and benchmarks.
Flows: Building Blocks of Reasoning and Collaborating AI
To support rapid and rigorous research, we introduce the aiFlows library embodying Flows.
REFINER: Reasoning Feedback on Intermediate Representations
Language models (LMs) have recently shown remarkable performance on reasoning tasks by explicitly generating intermediate inferences, e. g., chain-of-thought prompting.
Language Model Decoding as Likelihood-Utility Alignment
Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.
Generating Hypothetical Events for Abductive Inference
This work offers the first study of how such knowledge impacts the Abductive NLI task -- which consists in choosing the more likely explanation for given observations.
COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion
Despite recent successes of large pre-trained language models in solving reasoning tasks, their inference capabilities remain opaque.
CO-NNECT: A Framework for Revealing Commonsense Knowledge Paths as Explicitations of Implicit Knowledge in Texts
We conduct evaluations on two argumentative datasets and show that a combination of the two model types generates meaningful, high-quality knowledge paths between sentences that reveal implicit knowledge conveyed in text.
Social Commonsense Reasoning with Multi-Head Knowledge Attention
Notably we are, to the best of our knowledge, the first to demonstrate that a model that learns to perform counterfactual reasoning helps predicting the best explanation in an abductive reasoning task.
Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs
To make machines better understand sentiments, research needs to move from polarity identification to understanding the reasons that underlie the expression of sentiment.
Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling
In our experiments on Chunking and NER, this approach performs more robustly than the baselines.
