Search Results for author: Zorik Gekhman

Found 8 papers, 5 papers with code

Can LLMs Learn Macroeconomic Narratives from Social Media?

no code implementations17 Jun 2024 Almog Gueta, Amir Feder, Zorik Gekhman, Ariel Goldstein, Roi Reichart

This study empirically tests the $\textit{Narrative Economics}$ hypothesis, which posits that narratives (ideas that are spread virally and affect public beliefs) can influence economic fluctuations.

Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?

no code implementations9 May 2024 Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, Jonathan Herzig

In this work, we study the impact of such exposure to new knowledge on the capability of the fine-tuned model to utilize its pre-existing knowledge.

Measuring the Robustness of NLP Models to Domain Shifts

2 code implementations31 May 2023 Nitay Calderon, Naveh Porat, Eyal Ben-David, Alexander Chapanin, Zorik Gekhman, Nadav Oved, Vitaly Shalumov, Roi Reichart

We then conducted a comprehensive large-scale DR study involving over 14, 000 domain shifts across 21 fine-tuned models and few-shot LLMs.

Few-Shot Learning In-Context Learning +2

TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models

1 code implementation18 May 2023 Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, Idan Szpektor

Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries.

Natural Language Inference Synthetic Data Generation

On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Method

1 code implementation29 Jun 2022 Zorik Gekhman, Nadav Oved, Orgad Keller, Idan Szpektor, Roi Reichart

We find that high benchmark scores do not necessarily translate to strong robustness, and that various methods can perform extremely differently under different settings.

Conversational Question Answering

RED-ACE: Robust Error Detection for ASR using Confidence Embeddings

1 code implementation14 Mar 2022 Zorik Gekhman, Dina Zverinski, Jonathan Mallinson, Genady Beryozkin

ASR Error Detection (AED) models aim to post-process the output of Automatic Speech Recognition (ASR) systems, in order to detect transcription errors.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

KoBE: Knowledge-Based Machine Translation Evaluation

1 code implementation Findings of the Association for Computational Linguistics 2020 Zorik Gekhman, Roee Aharoni, Genady Beryozkin, Markus Freitag, Wolfgang Macherey

Our approach achieves the highest correlation with human judgements on 9 out of the 18 language pairs from the WMT19 benchmark for evaluation without references, which is the largest number of wins for a single evaluation method on this task.

Machine Translation Sentence +1

Cannot find the paper you are looking for? You can Submit a new open access paper.