Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automata

2 code implementations • 10 Apr 2024 • Jinghong Chen, Weizhe Lin, Jingbiao Mei, Bill Byrne

The Directed Acyclic Transformer is a fast non-autoregressive (NAR) model that performs well in Neural Machine Translation.

Machine Translation Text Generation +1

Few-Shot VQA with Frozen LLMs: A Tale of Two Approaches

no code implementations • 17 Mar 2024 • Igor Sterner, Weizhe Lin, Jinghong Chen, Bill Byrne

Two approaches have emerged to input images into large language models (LLMs).

Image Captioning Question Answering +1

PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers

1 code implementation • 13 Feb 2024 • Weizhe Lin, Jingbiao Mei, Jinghong Chen, Bill Byrne

Large Multimodal Models (LMMs) excel in natural language and visual understanding but are challenged by exacting tasks such as Knowledge-based Visual Question Answering (KB-VQA) which involve the retrieval of relevant information from document collections to use in shaping answers to questions.

Question Answering Retrieval +1

Improving hateful memes detection via learning hatefulness-aware embedding space through retrieval-guided contrastive learning

no code implementations • 14 Nov 2023 • Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, Marcus Tomalin

Finally, we demonstrate a retrieval-based hateful memes detection system, which is capable of making hatefulness classification based on data unseen in training from a database.

Contrastive Learning Meme Classification +1

Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

1 code implementation • 14 Nov 2023 • Guangyu Yang, Jinghong Chen, Weizhe Lin, Bill Byrne

Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs).

Machine Translation NMT +3

Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering

1 code implementation • NeurIPS 2023 • Weizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, Bill Byrne

FLMR addresses two major limitations in RA-VQA's retriever: (1) the image representations obtained via image-to-text transforms can be incomplete and inaccurate and (2) relevance scores between queries and documents are computed with one-dimensional embeddings, which can be insensitive to finer-grained relevance.

Passage Retrieval Question Answering +2

Grounding Description-Driven Dialogue State Trackers with Knowledge-Seeking Turns

no code implementations • 23 Sep 2023 • Alexandru Coca, Bo-Hsiang Tseng, Jinghong Chen, Weizhe Lin, Weixuan Zhang, Tisha Anders, Bill Byrne

Schema-guided dialogue state trackers can generalise to new domains without further training, yet they are sensitive to the writing style of the schemata.

Schema-Guided Semantic Accuracy: Faithfulness in Task-Oriented Dialogue Response Generation

1 code implementation • 29 Jan 2023 • Jinghong Chen, Weizhe Lin, Bill Byrne

We show that SGSAcc can be applied to evaluate utterances generated from a wide range of dialogue actions in the Schema Guided Dialogue (SGD) dataset with good agreement with human judgment.

Natural Language Inference Response Generation

LOCx2, a Low-latency, Low-overhead, 2 x 5.12-Gbps Transmitter ASIC for the ATLAS Liquid Argon Calorimeter Trigger Upgrade

no code implementations • 18 Sep 2020 • Le Xiao, Xiaoting Li, Datao Gong, Jinghong Chen, Di Guo, Huiqin He, Suen Hou, Guangming Huang, Chonghan Liu, Tiankuan Liu, Xiangming Sun, Ping-Kun Teng, Bozorgmehr Vosooghi, Annie C. Xiang, Jingbo Ye, Yang You, Zhiheng Zuo

In this paper, we present the design and test results of LOCx2, a transmitter ASIC for the ATLAS Liquid Argon Calorimeter trigger upgrade.

Instrumentation and Detectors