no code implementations • EACL 2017 • James Thorne, Andreas Vlachos
In this paper we present our automated fact checking system demonstration which we developed in order to participate in the Fast and Furious Fact Check challenge.
no code implementations • WS 2017 • James Thorne, Mingjie Chen, Giorgos Myrianthous, Jiashu Pu, Xiaoxuan Wang, Andreas Vlachos
Fake news has become a hotly debated topic in journalism.
5 code implementations • NAACL 2018 • James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal
Thus we believe that FEVER is a challenging testbed that will help stimulate progress on claim verification against textual sources.
no code implementations • COLING 2018 • James Thorne, Andreas Vlachos
The recently increased focus on misinformation has stimulated research in fact checking, the task of assessing the truthfulness of a claim.
no code implementations • WS 2018 • James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
We present the results of the first Fact Extraction and VERification (FEVER) Shared Task.
no code implementations • 13 Mar 2019 • James Thorne, Andreas Vlachos
This paper describes a baseline for the second iteration of the Fact Extraction and VERification shared task (FEVER2. 0) which explores the resilience of systems through adversarial evaluation.
no code implementations • NAACL 2019 • James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal
In this paper, we show that it is possible to generate token-level explanations for NLI without the need for training data explicitly annotated for this purpose.
no code implementations • WS 2019 • James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
We present the results of the second Fact Extraction and VERification (FEVER2. 0) Shared Task.
no code implementations • IJCNLP 2019 • James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal
Automated fact verification has been progressing owing to advancements in modeling and availability of large datasets.
1 code implementation • EACL 2021 • James Thorne, Andreas Vlachos
The biases present in training datasets have been shown to affect models for sentence pair classification tasks such as natural language inference (NLI) and fact verification.
3 code implementations • NAACL 2021 • Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance.
Ranked #3 on Entity Linking on KILT: WNED-CWEB
no code implementations • 14 Oct 2020 • James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy
We describe NeuralDB, a database system with no pre-defined schema, in which updates and queries are given in natural language.
3 code implementations • 31 Dec 2020 • James Thorne, Andreas Vlachos
This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence.
2 code implementations • 1 Apr 2021 • Max Glockner, Ieva Staliūnaitė, James Thorne, Gisela Vallejo, Andreas Vlachos, Iryna Gurevych
Automated fact-checking systems verify claims against evidence to predict their veracity.
1 code implementation • ACL 2021 • James Thorne, Majid Yazdani, Marzieh Saeidi, Fabrizio Silvestri, Sebastian Riedel, Alon Halevy
Neural models have shown impressive performance gains in answering queries from natural language text.
1 code implementation • ACL 2021 • James Thorne, Andreas Vlachos
This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence.
1 code implementation • 10 Jun 2021 • Rami Aly, Zhijiang Guo, Michael Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal
Fact verification has attracted a lot of attention in the machine learning and natural language processing communities, as it is one of the key methods for detecting misinformation.
no code implementations • 17 Nov 2022 • James Thorne
Document retrieval is a core component of many knowledge-intensive natural language processing task formulations such as fact verification and question answering.
no code implementations • 5 Apr 2023 • Jiwoo Hong, Yejin Cho, Jaemin Jung, Jiyoung Han, James Thorne
Our approach overcomes this limitation by considering both the sentence-level semantics and the document-level rhetorical structure, resulting in a more robust and style-agnostic approach to detecting political bias in news articles.
1 code implementation • 11 May 2023 • Jiho Kim, Sungjin Park, Yeonsu Kwon, Yohan Jo, James Thorne, Edward Choi
KGs can be a valuable knowledge source in fact verification due to their reliability and broad applicability.
1 code implementation • 23 May 2023 • Noah Lee, Na Min An, James Thorne
Large language models (LLMs) have shown impressive achievements in solving a broad range of tasks.
1 code implementation • 20 Jul 2023 • Seonghyeon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, Minjoon Seo
Evaluation of Large Language Models (LLMs) is challenging because instruction-following necessitates alignment with human values and the required set of skills varies depending on the instruction.
1 code implementation • 3 Aug 2023 • Jiyoung Lee, Seungho Kim, Seunghyun Won, Joonseok Lee, Marzyeh Ghassemi, James Thorne, Jaeseok Choi, O-Kil Kwon, Edward Choi
In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment.
1 code implementation • 12 Oct 2023 • Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, Minjoon Seo
We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4.
1 code implementation • 27 Oct 2023 • Philhoon Oh, James Thorne
However, counter-intuitively, too much context can have a negative impact on the model when evaluated on common question answering (QA) datasets.
1 code implementation • 27 Oct 2023 • Yejoon Lee, Philhoon Oh, James Thorne
This error arises when the knowledge corpus used for retrieval is only a subset of the entire string space, potentially excluding more helpful passages that exist outside the corpus.
1 code implementation • 1 Nov 2023 • Yongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho, James Thorne, Se-Young Yun
With the proliferation of social media, accurate detection of hate speech has become critical to ensure safety online.
no code implementations • 30 Jan 2024 • EuiYul Song, Sangryul Kim, Haeju Lee, Joonkee Kim, James Thorne
Subsequently, we extract and rerank contexts from the KILT database using the rerank page titles.
no code implementations • 4 Feb 2024 • EuiYul Song, Philhoon Oh, Sangryul Kim, James Thorne
Modern deterministic retrieval pipelines prioritize achieving state-of-the-art performance but often lack interpretability in decision-making.
no code implementations • 21 Feb 2024 • Minju Seo, Jinheon Baek, James Thorne, Sung Ju Hwang
Many existing works tackle this problem by generating synthetic data from the training data and then training models on them, recently using Large Language Models (LLMs).
1 code implementation • 11 Mar 2024 • Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, Alice Oh
Despite the rapid development of large language models (LLMs) for the Korean language, there remains an obvious lack of benchmark datasets that test the requisite Korean cultural and linguistic knowledge.
1 code implementation • 12 Mar 2024 • Jiwoo Hong, Noah Lee, James Thorne
While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence.
1 code implementation • 16 Mar 2024 • Sheikh Shafayat, H M Quamran Hasan, Minhajur Rahman Chowdhury Mahim, Rifki Afina Putri, James Thorne, Alice Oh
In this study, we introduce BEnQA, a dataset comprising parallel Bengali and English exam questions for middle and high school levels in Bangladesh.
no code implementations • 19 Mar 2024 • Minsu Kim, James Thorne
This paper investigates the inherent knowledge in language models from the perspective of epistemological holism.
no code implementations • EMNLP (FEVER) 2021 • Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, Arpit Mittal
The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) shared task, asks participating systems to determine whether human-authored claims are Supported or Refuted based on evidence retrieved from Wikipedia (or NotEnoughInfo if the claim cannot be verified).