Search Results for author: Alon Jacovi

Found 27 papers, 11 papers with code

ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatability

2 code implementations10 Jan 2025 Antonin Poché, Alon Jacovi, Agustin Martin Picard, Victor Boutin, Fanny Jourdan

We introduce an evaluation framework for measuring concept explanations via automated simulatability: a simulator's ability to predict the explained model's outputs based on the provided explanations.

CoverBench: A Challenging Benchmark for Complex Claim Verification

no code implementations6 Aug 2024 Alon Jacovi, Moran Ambar, Eyal Ben-David, Uri Shaham, Amir Feder, Mor Geva, Dror Marcus, Avi Caciularu

We introduce CoverBench, a challenging benchmark focused on verifying LM outputs in complex reasoning settings.

Claim Verification

Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP

no code implementations29 Jun 2024 Omer Goldman, Alon Jacovi, Aviv Slobodkin, Aviya Maimon, Ido Dagan, Reut Tsarfaty

By using a descriptive vocabulary and discussing the relevant properties of difficulty in long-context, we can implement more informed research in this area.

All Book summarization +1

A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains

no code implementations1 Feb 2024 Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva

REVEAL includes comprehensive labels for the relevance, attribution to evidence passages, and logical correctness of each reasoning step in a language model's answer, across a variety of datasets and state-of-the-art language models.

Open-Domain Question Answering

A Comprehensive Evaluation of Tool-Assisted Generation Strategies

no code implementations16 Oct 2023 Alon Jacovi, Avi Caciularu, Jonathan Herzig, Roee Aharoni, Bernd Bohnet, Mor Geva

A growing area of research investigates augmenting language models with tools (e. g., search engines, calculators) to overcome their shortcomings (e. g., missing or incorrect knowledge, incorrect logical inferences).

Retrieval

Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review

no code implementations5 Oct 2023 Tita A. Bach, Jenny K. Kristiansen, Aleksandar Babic, Alon Jacovi

We divided our investigation into the following areas: 1) terms used to describe HAII, 2) primary roles of AI-enabled systems, 3) factors that influence HAII, and 4) how HAII is measured.

Articles Decision Making +1

Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks

1 code implementation17 May 2023 Alon Jacovi, Avi Caciularu, Omer Goldman, Yoav Goldberg

Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora.

Neighboring Words Affect Human Interpretation of Saliency Explanations

1 code implementation4 May 2023 Alon Jacovi, Hendrik Schuff, Heike Adel, Ngoc Thang Vu, Yoav Goldberg

Word-level saliency explanations ("heat maps over words") are often used to communicate feature-attribution in text-based models.

Trends in Explainable AI (XAI) Literature

1 code implementation13 Jan 2023 Alon Jacovi

The XAI literature is decentralized, both in terminology and in publication venues, but recent years saw the community converge around keywords that make it possible to more reliably discover papers automatically.

Explainable Artificial Intelligence (XAI)

Human Interpretation of Saliency-based Explanation Over Text

1 code implementation27 Jan 2022 Hendrik Schuff, Alon Jacovi, Heike Adel, Yoav Goldberg, Ngoc Thang Vu

In this work, we focus on this question through a study of saliency-based explanations over textual data.

Contrastive Explanations for Model Interpretability

1 code implementation EMNLP 2021 Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi, Yoav Goldberg

Our method is based on projecting model representation to a latent space that captures only the features that are useful (to the model) to differentiate two potential decisions.

model text-classification +1

Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data

1 code implementation EMNLP 2020 Shachar Rosenman, Alon Jacovi, Yoav Goldberg

The process of collecting and annotating training data may introduce distribution artifacts which may limit the ability of models to learn correct generalization behavior.

Attribute Question Answering +2

Aligning Faithful Interpretations with their Social Attribution

1 code implementation1 Jun 2020 Alon Jacovi, Yoav Goldberg

We find that the requirement of model interpretations to be faithful is vague and incomplete.

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

no code implementations1 Jun 2020 Yanai Elazar, Shauli Ravfogel, Alon Jacovi, Yoav Goldberg

In this work, we point out the inability to infer behavioral conclusions from probing results and offer an alternative method that focuses on how the information is being used, rather than on what information is encoded.

Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?

no code implementations ACL 2020 Alon Jacovi, Yoav Goldberg

With the growing popularity of deep-learning based NLP models, comes a need for interpretable systems.

Scalable Evaluation and Improvement of Document Set Expansion via Neural Positive-Unlabeled Learning

1 code implementation EACL 2021 Alon Jacovi, Gang Niu, Yoav Goldberg, Masashi Sugiyama

We consider the situation in which a user has collected a small set of documents on a cohesive topic, and they want to retrieve additional documents on this topic from a large collection.

Information Retrieval Retrieval

Neural network gradient-based learning of black-box function interfaces

no code implementations ICLR 2019 Alon Jacovi, Guy Hadash, Einat Kermany, Boaz Carmeli, Ofer Lavi, George Kour, Jonathan Berant

We propose a method for end-to-end training of a base neural network that integrates calls to existing black-box functions.

Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications

no code implementations24 Apr 2018 Guy Hadash, Einat Kermany, Boaz Carmeli, Ofer Lavi, George Kour, Alon Jacovi

At inference time, we replace each estimator with its existing application counterpart and let the base network solve the task by interacting with the existing application.

Cannot find the paper you are looking for? You can Submit a new open access paper.