Search Results for author:
Found 17 papers, 10 papers with code
Multi-Conditional Ranking with Large Language Models
Our analysis of LLMs using MCRank indicates a significant decrease in performance as the number and complexity of items and conditions grow.
Reasoning Capacity in Multi-Agent Systems: Limitations, Challenges and Human-Centered Solutions
We present a formal definition of reasoning capacity and illustrate its utility in identifying limitations within each component of the system.
Distilling Large Language Models using Skill-Occupation Graph Context for HR-Related Tasks
We meticulously craft this benchmark to cater to a wide array of HR tasks, including matching and explaining resumes to job descriptions, extracting skills and experiences from resumes, and editing resumes.
Less is More for Long Document Summary Evaluation by LLMs
Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational costs and the Lost-in-the-Middle problem where important information in the middle of long documents is often overlooked.
Rethinking Language Models as Symbolic Knowledge Graphs
Despite these advancements, there is a void in comprehensively evaluating whether LMs can encompass the intricate topological and semantic attributes of KGs, attributes crucial for reasoning processes.
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
Investigating the sensitivity of LLMs towards the order of options in multiple-choice questions, we demonstrate a considerable performance gap of approximately 13% to 75% in LLMs on different benchmarks, when answer options are reordered, even when using demonstrations in a few-shot setting.
Measuring and Modifying Factual Knowledge in Large Language Models
In this work, we employ information theory-based measurements to provide a framework estimating the factual knowledge contained within large language models.
Quantifying Social Biases Using Templates is Unreliable
Recently, there has been an increase in efforts to understand how large language models (LLMs) propagate and amplify social biases.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Combining Feature and Instance Attribution to Detect Artifacts
In this paper we evaluate use of different attribution methods for aiding identification of training data artifacts.
An Empirical Comparison of Instance Attribution Methods for NLP
Instance attribution methods constitute one means of accomplishing these goals by retrieving training instances that (may have) led to a particular prediction.
ParsiNLU: A Suite of Language Understanding Challenges for Persian
Despite the progress made in recent years in addressing natural language understanding (NLU) challenges, the majority of this progress remains to be concentrated on resource-rich languages like English.
Revisiting Evaluation of Knowledge Base Completion Models
To address these issues, we gather a semi-complete KG referred as YAGO3-TC, using a random subgraph from the test and validation data of YAGO3-10, which enables us to compute accurate triple classification accuracy on this data.
Generating User-friendly Explanations for Loan Denials using GANs
An important mechanism towards that end is to explain AI decisions to various stakeholders.
Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications
We use these techniques to evaluate the robustness of link prediction models (by measuring sensitivity to additional facts), study interpretability through the facts most responsible for predictions (by identifying the most influential neighbors), and detect incorrect facts in the knowledge base.
Embedding Multimodal Relational Data for Knowledge Base Completion
In this paper, we propose multimodal knowledge base embeddings (MKBE) that use different neural encoders for this variety of observed data, and combine them with existing relational models to learn embeddings of the entities and multimodal data.
Compact Factorization of Matrices Using Generalized Round-Rank
Matrix factorization is a well-studied task in machine learning for compactly representing large, noisy data.
