no code implementations • 20 Feb 2024 • Guan-Ting Lin, Cheng-Han Chiang, Hung-Yi Lee
When using text-only LLMs to model spoken dialogue, text-only LLMs cannot give different responses based on the speaking style of the current turn.
1 code implementation • 8 Feb 2024 • Cheng-Han Chiang, Hung-Yi Lee
We show that LLMs can generate paragraphs that contain verifiable facts, but the facts are combined to form a non-factual paragraph due to entity ambiguity.
no code implementations • 6 Feb 2024 • Liang-Hsuan Tseng, En-Pei Hu, Cheng-Han Chiang, Yuan Tseng, Hung-Yi Lee, Lin-shan Lee, Shao-Hua Sun
A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
1 code implementation • 21 Jan 2024 • Cheng-Han Chiang, Hung-Yi Lee
Large language models (LLMs) can solve problems step-by-step.
1 code implementation • 9 Oct 2023 • Cheng-Han Chiang, Hung-Yi Lee
In this paper, we analyze LLM evaluation (Chiang and Lee, 2023) and G-Eval (Liu et al., 2023), and we discuss how those details in the evaluation process change how well the ratings given by LLMs correlate with human ratings.
no code implementations • 8 Jun 2023 • Cheng-Han Chiang, Yung-Sung Chuang, James Glass, Hung-Yi Lee
We also show that even if two SEs have similar performance on STS benchmarks, they can have very different behavior on HEROS.
1 code implementation • 3 Jun 2023 • Cheng-Han Chiang, Wei-Ping Huang, Hung-Yi Lee
This paper emphasizes the importance of reporting experiment details in subjective evaluations and demonstrates how such details can significantly impact evaluation results in the field of speech synthesis.
no code implementations • 3 May 2023 • Cheng-Han Chiang, Hung-Yi Lee
We show that the result of LLM evaluation is consistent with the results obtained by expert human evaluation: the texts rated higher by human experts are also rated higher by the LLMs.
no code implementations • 6 Oct 2022 • Cheng-Han Chiang, Hung-Yi Lee
In this paper, we explore the following question: Are synonym substitution attacks really synonym substitution attacks (SSAs)?
no code implementations • 10 Apr 2022 • Cheng-Han Chiang, Hung-Yi Lee
Our results reveal that the annotation quality is highly subject to the workers' qualification, and workers can be guided to provide certain annotations by the instructions.
no code implementations • 9 Apr 2022 • Cheng-Han Chiang, Hung-Yi Lee
Based on our observations, we propose a simple method to separate ID, OOD, and Adv samples using the hidden representations and output probabilities of the model.
1 code implementation • 8 Sep 2021 • Cheng-Han Chiang, Hung-Yi Lee
In this work, we study what specific traits in the pre-training data, other than the semantics, make a pre-trained LM superior to their counterparts trained from scratch on downstream tasks.
no code implementations • 22 Dec 2020 • Cheng-Han Chiang, Hung-Yi Lee
In this paper, we study how the intrinsic nature of pre-training data contributes to the fine-tuned downstream performance.
1 code implementation • EMNLP 2020 • Cheng-Han Chiang, Sung-Feng Huang, Hung-Yi Lee
These findings suggest that knowledge of a pretrained model varies during pretraining, and having more pretrain steps does not necessarily provide a model with more comprehensive knowledge.