Search Results for author: Cheng-Han Chiang

Found 14 papers, 6 papers with code

Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations

no code implementations20 Feb 2024 Guan-Ting Lin, Cheng-Han Chiang, Hung-Yi Lee

When using text-only LLMs to model spoken dialogue, text-only LLMs cannot give different responses based on the speaking style of the current turn.

Sentence

Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations

1 code implementation8 Feb 2024 Cheng-Han Chiang, Hung-Yi Lee

We show that LLMs can generate paragraphs that contain verifiable facts, but the facts are combined to form a non-factual paragraph due to entity ambiguity.

REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR

no code implementations6 Feb 2024 Liang-Hsuan Tseng, En-Pei Hu, Cheng-Han Chiang, Yuan Tseng, Hung-Yi Lee, Lin-shan Lee, Shao-Hua Sun

A word/phoneme in the speech signal is represented by a segment of speech signal with variable length and unknown boundary, and this segmental structure makes learning the mapping between speech and text challenging, especially without paired data.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

A Closer Look into Automatic Evaluation Using Large Language Models

1 code implementation9 Oct 2023 Cheng-Han Chiang, Hung-Yi Lee

In this paper, we analyze LLM evaluation (Chiang and Lee, 2023) and G-Eval (Liu et al., 2023), and we discuss how those details in the evaluation process change how well the ratings given by LLMs correlate with human ratings.

Revealing the Blind Spot of Sentence Encoder Evaluation by HEROS

no code implementations8 Jun 2023 Cheng-Han Chiang, Yung-Sung Chuang, James Glass, Hung-Yi Lee

We also show that even if two SEs have similar performance on STS benchmarks, they can have very different behavior on HEROS.

Negation Sentence +1

Why We Should Report the Details in Subjective Evaluation of TTS More Rigorously

1 code implementation3 Jun 2023 Cheng-Han Chiang, Wei-Ping Huang, Hung-Yi Lee

This paper emphasizes the importance of reporting experiment details in subjective evaluations and demonstrates how such details can significantly impact evaluation results in the field of speech synthesis.

Speech Synthesis

Can Large Language Models Be an Alternative to Human Evaluations?

no code implementations3 May 2023 Cheng-Han Chiang, Hung-Yi Lee

We show that the result of LLM evaluation is consistent with the results obtained by expert human evaluation: the texts rated higher by human experts are also rated higher by the LLMs.

Story Generation

Are Synonym Substitution Attacks Really Synonym Substitution Attacks?

no code implementations6 Oct 2022 Cheng-Han Chiang, Hung-Yi Lee

In this paper, we explore the following question: Are synonym substitution attacks really synonym substitution attacks (SSAs)?

Sentence

Re-Examining Human Annotations for Interpretable NLP

no code implementations10 Apr 2022 Cheng-Han Chiang, Hung-Yi Lee

Our results reveal that the annotation quality is highly subject to the workers' qualification, and workers can be guided to provide certain annotations by the instructions.

Understanding, Detecting, and Separating Out-of-Distribution Samples and Adversarial Samples in Text Classification

no code implementations9 Apr 2022 Cheng-Han Chiang, Hung-Yi Lee

Based on our observations, we propose a simple method to separate ID, OOD, and Adv samples using the hidden representations and output probabilities of the model.

text-classification Text Classification

On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets

1 code implementation8 Sep 2021 Cheng-Han Chiang, Hung-Yi Lee

In this work, we study what specific traits in the pre-training data, other than the semantics, make a pre-trained LM superior to their counterparts trained from scratch on downstream tasks.

Pre-Training a Language Model Without Human Language

no code implementations22 Dec 2020 Cheng-Han Chiang, Hung-Yi Lee

In this paper, we study how the intrinsic nature of pre-training data contributes to the fine-tuned downstream performance.

Language Modelling

Pretrained Language Model Embryology: The Birth of ALBERT

1 code implementation EMNLP 2020 Cheng-Han Chiang, Sung-Feng Huang, Hung-Yi Lee

These findings suggest that knowledge of a pretrained model varies during pretraining, and having more pretrain steps does not necessarily provide a model with more comprehensive knowledge.

Language Modelling POS +1

Cannot find the paper you are looking for? You can Submit a new open access paper.