PrefScore: Pairwise Preference Learning for Reference-free Summarization Quality Assessment

1 code implementation • COLING 2022 • Ge Luo, Hebi Li, Youbiao He, Forrest Sheng Bao

Evaluating machine-generated summaries without a human-written reference summary has been a need for a long time.

DocAsRef: An Empirical Study on Repurposing Reference-Based Summary Quality Metrics Reference-Freely

1 code implementation • 20 Dec 2022 • Forrest Sheng Bao, Ruixuan Tu, Ge Luo, Yinfei Yang, Hebi Li, Minghui Qiu, Youbiao He, Cen Chen

Automated summary quality assessment falls into two categories: reference-based and reference-free.

Supervised Whole DAG Causal Discovery

1 code implementation • 8 Jun 2020 • Hebi Li, Qi Xiao, Jin Tian

We propose a novel approach of modeling the whole DAG structure discovery as a supervised learning.

Causal Discovery

SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling

1 code implementation • NAACL 2022 • Forrest Sheng Bao, Hebi Li, Ge Luo, Minghui Qiu, Yinfei Yang, Youbiao He, Cen Chen

Canonical automatic summary evaluation metrics, such as ROUGE, focus on lexical similarity which cannot well capture semantics nor linguistic quality and require a reference summary which is costly to obtain.

Abstractive Text Summarization Document Embedding +3

Purifying Adversarial Perturbation with Adversarially Trained Auto-encoders

no code implementations • 26 May 2019 • Hebi Li, Qi Xiao, Shixin Tian, Jin Tian

Machine learning models are vulnerable to adversarial examples.

BIG-bench Machine Learning