Search Results for author:
Found 27 papers, 11 papers with code
DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs
CONFLICTS is the first benchmark that enables tracking progress on how models address a wide range of knowledge conflicts.
ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatability
We introduce an evaluation framework for measuring concept explanations via automated simulatability: a simulator's ability to predict the explained model's outputs based on the provided explanations.
The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input
Models are evaluated using automated judge models in two phases: (1) responses are disqualified if they do not fulfill the user request; (2) they are judged as accurate if the response is fully grounded in the provided document.
CoverBench: A Challenging Benchmark for Complex Claim Verification
We introduce CoverBench, a challenging benchmark focused on verifying LM outputs in complex reasoning settings.
Data Contamination Report from the 2024 CONDA Shared Task
The workshop fostered a shared task to collect evidence on data contamination in current available datasets and models.
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
By using a descriptive vocabulary and discussing the relevant properties of difficulty in long-context, we can implement more informed research in this area.
Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations
Despite recent advancements in Large Language Models (LLMs), their performance on tasks involving long contexts remains sub-optimal.
TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools
We construct this dataset by leveraging an existing dataset of texts and their associated tables.
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
REVEAL includes comprehensive labels for the relevance, attribution to evidence passages, and logical correctness of each reasoning step in a language model's answer, across a variety of datasets and state-of-the-art language models.
A Comprehensive Evaluation of Tool-Assisted Generation Strategies
A growing area of research investigates augmenting language models with tools (e. g., search engines, calculators) to overcome their shortcomings (e. g., missing or incorrect knowledge, incorrect logical inferences).
Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review
We divided our investigation into the following areas: 1) terms used to describe HAII, 2) primary roles of AI-enabled systems, 3) factors that influence HAII, and 4) how HAII is measured.
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks
Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora.
Neighboring Words Affect Human Interpretation of Saliency Explanations
Word-level saliency explanations ("heat maps over words") are often used to communicate feature-attribution in text-based models.
Trends in Explainable AI (XAI) Literature
The XAI literature is decentralized, both in terminology and in publication venues, but recent years saw the community converge around keywords that make it possible to more reliably discover papers automatically.
Human Interpretation of Saliency-based Explanation Over Text
In this work, we focus on this question through a study of saliency-based explanations over textual data.
Diagnosing AI Explanation Methods with Folk Concepts of Behavior
We posit that folk concepts of behavior provide us with a "language" that humans understand behavior with.
Contrastive Explanations for Model Interpretability
Our method is based on projecting model representation to a latent space that captures only the features that are useful (to the model) to differentiate two potential decisions.
Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i. e., trust between people).
Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data
The process of collecting and annotating training data may introduce distribution artifacts which may limit the ability of models to learn correct generalization behavior.
Aligning Faithful Interpretations with their Social Attribution
We find that the requirement of model interpretations to be faithful is vague and incomplete.
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
In this work, we point out the inability to infer behavioral conclusions from probing results and offer an alternative method that focuses on how the information is being used, rather than on what information is encoded.
Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?
With the growing popularity of deep-learning based NLP models, comes a need for interpretable systems.
Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning
We consider the situation in which a user has collected a small set of documents on a cohesive topic, and they want to retrieve additional documents on this topic from a large collection.
Learning and Understanding Different Categories of Sexism Using Convolutional Neural Network's Filters
Sexism is very common in social media and makes the boundaries of free speech tighter for female users.
Neural network gradient-based learning of black-box function interfaces
We propose a method for end-to-end training of a base neural network that integrates calls to existing black-box functions.
Understanding Convolutional Neural Networks for Text Classification
We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text.
Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications
At inference time, we replace each estimator with its existing application counterpart and let the base network solve the task by interacting with the existing application.
