Search Results for author:
Found 23 papers, 8 papers with code
MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning
While, TALMs have been successfully employed in different question-answering benchmarks, their efficacy on complex mathematical reasoning benchmarks, and the potential complementary benefits offered by tools for knowledge retrieval and mathematical equation solving are open research questions.
GRAFFORD: A Benchmark Dataset for Testing the Knowledge of Object Affordances of Language and Vision Models
We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs).
Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs
Reasoning is a fundamental component of language understanding.
Caught in the Quicksand of Reasoning, Far from AGI Summit: Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions
Recent advancements in Large Language Models (LLMs) have showcased striking results on existing logical reasoning benchmarks, with some models even surpassing human performance.
Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models
This work strives to understand the proficiency of LLMs in logical reasoning by offering a brief review of the latest progress in this area; with a focus on the logical reasoning datasets, tasks, and the methods adopted to utilize LLMs for reasoning.
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies.
ReMask: A Robust Information-Masking Approach for Domain Counterfactual Generation
Our experiments empirically show that the counterfactual samples sourced from our masked text lead to improved domain transfer on 10 out of 12 domain sentiment classification settings, with an average of 2% accuracy improvement over the state-of-the-art for unsupervised domain adaptation (UDA).
Generating Intermediate Steps for NLI with Next-Step Supervision
The Natural Language Inference (NLI) task often requires reasoning over multiple steps to reach the conclusion.
Multilingual CheckList: Generation and Evaluation
We compare the TEA CheckLists with CheckLists created with different levels of human intervention.
LoNLI: An Extensible Framework for Testing Diverse Logical Reasoning Capabilities for NLI
Natural Language Inference (NLI) is considered a representative task to test natural language understanding (NLU).
Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance
Multilingual language models achieve impressive zero-shot accuracies in many languages in complex tasks such as Natural Language Inference (NLI).
Trusting RoBERTa over BERT: Insights from CheckListing the Natural Language Inference Task
The recent state-of-the-art natural language understanding (NLU) systems often behave unpredictably, failing on simpler reasoning examples.
Analyzing the Nuances of Transformers' Polynomial Simplification Abilities
To understand Transformers' abilities in such tasks in a fine-grained manner, we deviate from traditional end-to-end settings, and explore a step-wise polynomial simplification task.
Do Transformers Understand Polynomial Simplification?
For a polynomial which is not necessarily in this normal form, a sequence of simplification steps is applied to reach the fully simplified (i. e., in the normal form) polynomial.
TaxiNLI: Taking a Ride up the NLU Hill
Pre-trained Transformer-based neural architectures have consistently achieved state-of-the-art performance in the Natural Language Inference (NLI) task.
Uncovering Relations for Marketing Knowledge Representation
Online behaviors of consumers and marketers generate massive marketing data, which ever more sophisticated models attempt to turn into insights and aid decisions by marketers.
Integrating Knowledge and Reasoning in Image Understanding
Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering.
Spatial Knowledge Distillation to aid Visual Reasoning
We propose a framework that combines recent advances in knowledge distillation (teacher-student framework), relational reasoning and probabilistic logical languages to incorporate such knowledge in existing neural networks for the task of Visual Question Answering.
Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering
Here we adopt Visual Question Answering (VQA) as an example task, where a system is expected to answer a question in natural language about an image.
Answering Image Riddles using Vision and Reasoning through Probabilistic Soft Logic
We compile a dataset of over 3k riddles where each riddle consists of 4 images and a groundtruth answer.
From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge
Specifically, commonsense reasoning is applied on (a) detections obtained from existing perception methods on given images, (b) a "commonsense" knowledge base constructed using natural language processing of image annotations and (c) lexical ontological knowledge from resources such as WordNet.
