Search Results for author: Chenyang Lyu

Found 29 papers, 10 papers with code

DCU-Lorcan at FinCausal 2022: Span-based Causality Extraction from Financial Documents using Pre-trained Language Models

no code implementations FNP (LREC) 2022 Chenyang Lyu, Tianbo Ji, Quanwei Sun, Liting Zhou

In this paper, we describe our DCU-Lorcan system for the FinCausal 2022 shared task: span-based cause and effect extraction from financial documents.

Findings of the WMT 2024 Shared Task on Discourse-Level Literary Translation

1 code implementation16 Dec 2024 Longyue Wang, Siyou Liu, Chenyang Lyu, Wenxiang Jiao, Xing Wang, Jiahao Xu, Zhaopeng Tu, Yan Gu, WeiYu Chen, Minghao Wu, Liting Zhou, Philipp Koehn, Andy Way, Yulin Yuan

Following last year, we have continued to host the WMT translation shared task this year, the second edition of the Discourse-Level Literary Translation.

Translation

Large Language Models as Code Executors: An Exploratory Study

no code implementations9 Oct 2024 Chenyang Lyu, Lecheng Yan, Rui Xing, Wenxi Li, Younes Samih, Tianbo Ji, Longyue Wang

The capabilities of Large Language Models (LLMs) have significantly evolved, extending from natural language processing to complex tasks like code understanding and generation.

Reference-free Hallucination Detection for Large Vision-Language Models

no code implementations11 Aug 2024 Qing Li, Jiahui Geng, Chenyang Lyu, Derui Zhu, Maxim Panov, Fakhri Karray

In particular, we conduct an extensive study on three kinds of techniques: uncertainty-based, consistency-based, and supervised uncertainty quantification methods on four representative LVLMs across two different tasks.

Hallucination Question Answering +1

CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark

no code implementations10 Jun 2024 David Romero, Chenyang Lyu, Haryo Akbarianto Wibowo, Teresa Lynn, Injy Hamed, Aditya Nanda Kishore, Aishik Mandal, Alina Dragonetti, Artem Abzaliev, Atnafu Lambebo Tonja, Bontu Fufa Balcha, Chenxi Whitehouse, Christian Salamea, Dan John Velasco, David Ifeoluwa Adelani, David Le Meur, Emilio Villa-Cueva, Fajri Koto, Fauzan Farooqui, Frederico Belcavello, Ganzorig Batnasan, Gisela Vallejo, Grainne Caulfield, Guido Ivetta, Haiyue Song, Henok Biadglign Ademtew, Hernán Maina, Holy Lovenia, Israel Abebe Azime, Jan Christian Blaise Cruz, Jay Gala, Jiahui Geng, Jesus-German Ortiz-Barajas, Jinheon Baek, Jocelyn Dunstan, Laura Alonso Alemany, Kumaranage Ravindu Yasas Nagasinghe, Luciana Benotti, Luis Fernando D'Haro, Marcelo Viridiano, Marcos Estecha-Garitagoitia, Maria Camila Buitrago Cabrera, Mario Rodríguez-Cantelar, Mélanie Jouitteau, Mihail Mihaylov, Mohamed Fazli Mohamed Imam, Muhammad Farid Adilazuarda, Munkhjargal Gochoo, Munkh-Erdene Otgonbold, Naome Etori, Olivier Niyomugisha, Paula Mónica Silva, Pranjal Chitale, Raj Dabre, Rendi Chevi, Ruochen Zhang, Ryandito Diandaru, Samuel Cahyawijaya, Santiago Góngora, Soyeong Jeong, Sukannya Purkayastha, Tatsuki Kuribayashi, Teresa Clifford, Thanmay Jayakumar, Tiago Timponi Torrent, Toqeer Ehsan, Vladimir Araujo, Yova Kementchedjhieva, Zara Burzo, Zheng Wei Lim, Zheng Xin Yong, Oana Ignat, Joan Nwatu, Rada Mihalcea, Thamar Solorio, Alham Fikri Aji

Visual Question Answering (VQA) is an important task in multimodal AI, and it is often used to test the ability of vision-language models to understand and reason on knowledge present in both visual and textual data.

Diversity Question Answering +1

Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models

no code implementations21 Feb 2024 Chenyang Lyu, Minghao Wu, Alham Fikri Aji

Large Language Models (LLMs) have demonstrated remarkable capabilities across various applications, fundamentally reshaping the landscape of natural language processing (NLP) research.

Multiple-choice

Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Models

no code implementations4 Dec 2023 Bingshuai Liu, Chenyang Lyu, Zijun Min, Zhanyu Wang, Jinsong Su, Longyue Wang

The advancement of Large Language Models (LLMs) has brought substantial attention to the Chain of Thought (CoT) approach, primarily due to its ability to enhance the capability of LLMs on complex reasoning tasks.

Question Answering Retrieval

GPT4Video: A Unified Multimodal Large Language Model for lnstruction-Followed Understanding and Safety-Aware Generation

no code implementations25 Nov 2023 Zhanyu Wang, Longyue Wang, Zhen Zhao, Minghao Wu, Chenyang Lyu, Huayang Li, Deng Cai, Luping Zhou, Shuming Shi, Zhaopeng Tu

While the recent advances in Multimodal Large Language Models (MLLMs) constitute a significant leap forward in the field, these models are predominantly confined to the realm of input-side multimodal comprehension, lacking the capacity for multimodal content generation.

Instruction Following Language Modeling +9

A Comprehensive Evaluation of GPT-4V on Knowledge-Intensive Visual Question Answering

1 code implementation13 Nov 2023 Yunxin Li, Longyue Wang, Baotian Hu, Xinyu Chen, Wanqi Zhong, Chenyang Lyu, Wei Wang, Min Zhang

The emergence of multimodal large models (MLMs) has significantly advanced the field of visual understanding, offering remarkable capabilities in the realm of visual question answering (VQA).

Decision Making Explanation Generation +4

On the Cultural Gap in Text-to-Image Generation

no code implementations6 Jul 2023 Bingshuai Liu, Longyue Wang, Chenyang Lyu, Yong Zhang, Jinsong Su, Shuming Shi, Zhaopeng Tu

Accordingly, we propose a novel multi-modal metric that considers object-text alignment to filter the fine-tuning data in the target culture, which is used to fine-tune a T2I model to improve cross-cultural generation.

Text-to-Image Generation

Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration

1 code implementation15 Jun 2023 Chenyang Lyu, Minghao Wu, Longyue Wang, Xinting Huang, Bingshuai Liu, Zefeng Du, Shuming Shi, Zhaopeng Tu

Although instruction-tuned large language models (LLMs) have exhibited remarkable capabilities across various NLP tasks, their effectiveness on other data modalities beyond text has not been fully studied.

Language Modeling Language Modelling

Out-of-Distribution Generalization in Text Classification: Past, Present, and Future

no code implementations23 May 2023 Linyi Yang, Yaoxiao Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang

Machine learning (ML) systems in natural language processing (NLP) face significant challenges in generalizing to out-of-distribution (OOD) data, where the test distribution differs from the training data distribution.

Out-of-Distribution Generalization text-classification +1

Is a Video worth $n\times n$ Images? A Highly Efficient Approach to Transformer-based Video Question Answering

no code implementations16 May 2023 Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster

We show that by integrating our approach into VideoQA systems we can achieve comparable, even superior, performance with a significant speed up for training and inference.

Question Answering Video Question Answering

Semantic-aware Dynamic Retrospective-Prospective Reasoning for Event-level Video Question Answering

no code implementations14 May 2023 Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster

Specifically, we explicitly use the Semantic Role Labeling (SRL) structure of the question in the dynamic reasoning process where we decide to move to the next frame based on which part of the SRL structure (agent, verb, patient, etc.)

Question Answering Semantic Role Labeling +1

A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models

no code implementations2 May 2023 Chenyang Lyu, Zefeng Du, Jitao Xu, Yitao Duan, Minghao Wu, Teresa Lynn, Alham Fikri Aji, Derek F. Wong, Siyou Liu, Longyue Wang

We conclude by emphasizing the critical role of LLMs in guiding the future evolution of MT and offer a roadmap for future exploration in the sector.

Document Translation Machine Translation +2

Document-Level Machine Translation with Large Language Models

1 code implementation5 Apr 2023 Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, Zhaopeng Tu

Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks.

Document Level Machine Translation Machine Translation +1

Dialogue-to-Video Retrieval

1 code implementation23 Mar 2023 Chenyang Lyu, Manh-Duy Nguyen, Van-Tu Ninh, Liting Zhou, Cathal Gurrin, Jennifer Foster

Recent years have witnessed an increasing amount of dialogue/conversation on the web especially on social media.

Recommendation Systems Retrieval +1

QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation

no code implementations9 Oct 2022 Tianbo Ji, Chenyang Lyu, Gareth Jones, Liting Zhou, Yvette Graham

Question Generation (QG) aims to automate the task of composing questions for a passage with a set of chosen answers found within the passage.

Language Modeling Language Modelling +2

Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains

1 code implementation insights (ACL) 2022 Chenyang Lyu, Jennifer Foster, Yvette Graham

Past works that investigate out-of-domain performance of QA systems have mainly focused on general domains (e. g. news domain, wikipedia domain), underestimating the importance of subdomains defined by the internal characteristics of QA datasets.

Position

Achieving Reliable Human Assessment of Open-Domain Dialogue Systems

1 code implementation ACL 2022 Tianbo Ji, Yvette Graham, Gareth J. F. Jones, Chenyang Lyu, Qun Liu

Answering the distress call of competitions that have emphasized the urgent need for better evaluation techniques in dialogue, we present the successful development of human evaluation that is highly reliable while still remaining feasible and low cost.

Dialogue Evaluation

Improving Unsupervised Question Answering via Summarization-Informed Question Generation

no code implementations EMNLP 2021 Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu

Template-based QG uses linguistically-informed heuristics to transform declarative sentences into interrogatives, whereas supervised QG uses existing Question Answering (QA) datasets to train a system to generate a question given a passage and an answer.

Dependency Parsing named-entity-recognition +8

Improving Document-Level Sentiment Analysis with User and Product Context

1 code implementation COLING 2020 Chenyang Lyu, Jennifer Foster, Yvette Graham

We achieve this by explicitly storing representations of reviews written by the same user and about the same product and force the model to memorize all reviews for one particular user and product.

Sentiment Analysis

UNIFUZZ: A Holistic and Pragmatic Metrics-Driven Platform for Evaluating Fuzzers

1 code implementation5 Oct 2020 Yuwei Li, Shouling Ji, Yuan Chen, Sizhuang Liang, Wei-Han Lee, Yueyao Chen, Chenyang Lyu, Chunming Wu, Raheem Beyah, Peng Cheng, Kangjie Lu, Ting Wang

We hope that our findings can shed light on reliable fuzzing evaluation, so that we can discover promising fuzzing primitives to effectively facilitate fuzzer designs in the future.

Cryptography and Security

SmartSeed: Smart Seed Generation for Efficient Fuzzing

no code implementations7 Jul 2018 Chenyang Lyu, Shouling Ji, Yuwei Li, Junfeng Zhou, Jian-hai Chen, Jing Chen

In total, our system discovers more than twice unique crashes and 5, 040 extra unique paths than the existing best seed selection strategy for the evaluated 12 applications.

Cryptography and Security

Cannot find the paper you are looking for? You can Submit a new open access paper.