Search Results for author: James Thorne

Found 35 papers, 19 papers with code

An Extensible Framework for Verification of Numerical Claims

no code implementations EACL 2017 James Thorne, Andreas Vlachos

In this paper we present our automated fact checking system demonstration which we developed in order to participate in the Fast and Furious Fact Check challenge.

Fact Checking Rumour Detection +1

FEVER: a large-scale dataset for Fact Extraction and VERification

5 code implementations NAACL 2018 James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal

Thus we believe that FEVER is a challenging testbed that will help stimulate progress on claim verification against textual sources.

Claim Verification Sentence

Automated Fact Checking: Task formulations, methods and future directions

no code implementations COLING 2018 James Thorne, Andreas Vlachos

The recently increased focus on misinformation has stimulated research in fact checking, the task of assessing the truthfulness of a claim.

Fact Checking Misinformation

Adversarial attacks against Fact Extraction and VERification

no code implementations13 Mar 2019 James Thorne, Andreas Vlachos

This paper describes a baseline for the second iteration of the Fact Extraction and VERification shared task (FEVER2. 0) which explores the resilience of systems through adversarial evaluation.

Fact Checking Information Retrieval +2

Generating Token-Level Explanations for Natural Language Inference

no code implementations NAACL 2019 James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal

In this paper, we show that it is possible to generate token-level explanations for NLI without the need for training data explicitly annotated for this purpose.

Multiple Instance Learning Natural Language Inference +2

Elastic weight consolidation for better bias inoculation

1 code implementation EACL 2021 James Thorne, Andreas Vlachos

The biases present in training datasets have been shown to affect models for sentence pair classification tasks such as natural language inference (NLI) and fact verification.

Fact Verification General Classification +3

Neural Databases

no code implementations14 Oct 2020 James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy

We describe NeuralDB, a database system with no pre-defined schema, in which updates and queries are given in natural language.

Management

Evidence-based Factual Error Correction

3 code implementations31 Dec 2020 James Thorne, Andreas Vlachos

This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence.

Fact Checking Fact Verification +1

Database Reasoning Over Text

1 code implementation ACL 2021 James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy

Neural models have shown impressive performance gains in answering queries from natural language text.

Evidence-based Factual Error Correction

1 code implementation ACL 2021 James Thorne, Andreas Vlachos

This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence.

Fact Checking Fact Verification

FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information

1 code implementation10 Jun 2021 Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal

Fact verification has attracted a lot of attention in the machine learning and natural language processing communities, as it is one of the key methods for detecting misinformation.

Fact Verification Misinformation

Data-Efficient Autoregressive Document Retrieval for Fact Verification

no code implementations17 Nov 2022 James Thorne

Document retrieval is a core component of many knowledge-intensive natural language processing task formulations such as fact verification and question answering.

Fact Verification Question Answering +1

Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy

no code implementations5 Apr 2023 Jiwoo Hong, Yejin Cho, Jaemin Jung, Jiyoung Han, James Thorne

Our approach overcomes this limitation by considering both the sentence-level semantics and the document-level rhetorical structure, resulting in a more robust and style-agnostic approach to detecting political bias in news articles.

Bias Detection Document Classification +1

FactKG: Fact Verification via Reasoning on Knowledge Graphs

1 code implementation11 May 2023 Jiho Kim, Sungjin Park, Yeonsu Kwon, Yohan Jo, James Thorne, Edward Choi

KGs can be a valuable knowledge source in fact verification due to their reliability and broad applicability.

Fact Verification Knowledge Graphs +1

Can Large Language Models Capture Dissenting Human Voices?

1 code implementation23 May 2023 Noah Lee, Na Min An, James Thorne

Large language models (LLMs) have shown impressive achievements in solving a broad range of tasks.

Natural Language Inference Natural Language Understanding

FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets

1 code implementation20 Jul 2023 Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, Minjoon Seo

Evaluation of Large Language Models (LLMs) is challenging because instruction-following necessitates alignment with human values and the required set of skills varies depending on the instruction.

Instruction Following Language Modelling

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

1 code implementation12 Oct 2023 Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, Minjoon Seo

We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.

Language Modelling Large Language Model

Detrimental Contexts in Open-Domain Question Answering

1 code implementation27 Oct 2023 Philhoon Oh, James Thorne

However, counter-intuitively, too much context can have a negative impact on the model when evaluated on common question answering (QA) datasets.

Open-Domain Question Answering Retrieval

Knowledge Corpus Error in Question Answering

1 code implementation27 Oct 2023 Yejoon Lee, Philhoon Oh, James Thorne

This error arises when the knowledge corpus used for retrieval is only a subset of the entire string space, potentially excluding more helpful passages that exist outside the corpus.

Open-Domain Question Answering Retrieval

HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning

1 code implementation1 Nov 2023 Yongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho, James Thorne, Se-Young Yun

With the proliferation of social media, accurate detection of hate speech has become critical to ensure safety online.

Hate Speech Detection

Re3val: Reinforced and Reranked Generative Retrieval

no code implementations30 Jan 2024 EuiYul Song, Sangryul Kim, Haeju Lee, Joonkee Kim, James Thorne

Subsequently, we extract and rerank contexts from the KILT database using the rerank page titles.

Passage Retrieval Retrieval

eXplainable Bayesian Multi-Perspective Generative Retrieval

no code implementations4 Feb 2024 EuiYul Song, Philhoon Oh, Sangryul Kim, James Thorne

Modern deterministic retrieval pipelines prioritize achieving state-of-the-art performance but often lack interpretability in decision-making.

Decision Making Fact Checking +2

Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks

no code implementations21 Feb 2024 Minju Seo, Jinheon Baek, James Thorne, Sung Ju Hwang

Many existing works tackle this problem by generating synthetic data from the training data and then training models on them, recently using Large Language Models (LLMs).

Data Augmentation Retrieval

CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean

1 code implementation11 Mar 2024 Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, Alice Oh

Despite the rapid development of large language models (LLMs) for the Korean language, there remains an obvious lack of benchmark datasets that test the requisite Korean cultural and linguistic knowledge.

Hate Speech Detection

ORPO: Monolithic Preference Optimization without Reference Model

1 code implementation12 Mar 2024 Jiwoo Hong, Noah Lee, James Thorne

While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence.

BEnQA: A Question Answering and Reasoning Benchmark for Bengali and English

1 code implementation16 Mar 2024 Sheikh Shafayat, H M Quamran Hasan, Minhajur Rahman Chowdhury Mahim, Rifki Afina Putri, James Thorne, Alice Oh

In this study, we introduce BEnQA, a dataset comprising parallel Bengali and English exam questions for middle and high school levels in Bangladesh.

Question Answering

Epistemology of Language Models: Do Language Models Have Holistic Knowledge?

no code implementations19 Mar 2024 Minsu Kim, James Thorne

This paper investigates the inherent knowledge in language models from the perspective of epistemological holism.

The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task

no code implementations EMNLP (FEVER) 2021 Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal

The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) shared task, asks participating systems to determine whether human-authored claims are Supported or Refuted based on evidence retrieved from Wikipedia (or NotEnoughInfo if the claim cannot be verified).

Retrieval

Cannot find the paper you are looking for? You can Submit a new open access paper.