Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play

1 code implementation • 21 Oct 2022 • Qi Liu, Zihuiwen Ye, Tao Yu, Phil Blunsom, Linfeng Song

We first design a SQL-to-text model conditioned on a sampled goal query, which represents a user's intent, that then converses with a text-to-SQL semantic parser to generate new interactions.

Domain Generalization SQL-to-Text +1

ExplainaBoard: An Explainable Leaderboard for NLP

1 code implementation • ACL 2021 • PengFei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Zi-Yi Dou, Graham Neubig

In this paper, we present a new conceptualization and implementation of NLP evaluation: the ExplainaBoard, which in addition to inheriting the functionality of the standard leaderboard, also allows researchers to (i) diagnose strengths and weaknesses of a single system (e. g.~what is the best-performing system bad at?)

Machine Translation

Towards More Fine-grained and Reliable NLP Performance Prediction

1 code implementation • EACL 2021 • Zihuiwen Ye, PengFei Liu, Jinlan Fu, Graham Neubig

We perform an analysis of four types of NLP tasks, and both demonstrate the feasibility of fine-grained performance prediction and the necessity to perform reliability analysis for performance prediction methods in the future.