Training a Vision Transformer from scratch in less than 24 hours with 1 GPU

1 code implementation • 9 Nov 2022 • Saghar Irandoust, Thibaut Durand, Yunduz Rakhmangulova, Wenjie Zi, Hossein Hajimirsadeghi

We introduce some algorithmic improvements to enable training a ViT model from scratch with limited hardware (1 GPU) and time (24 hours) resources.

Turing: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface

no code implementations • ACL 2021 • Peng Xu, Wenjie Zi, Hamidreza Shahidi, Ákos Kádár, Keyi Tang, Wei Yang, Jawad Ateeq, Harsh Barot, Meidan Alon, Yanshuai Cao

A natural language database interface (NLDB) can democratize data-driven insights for non-technical users.

Text Generation Text-To-SQL +1

Optimizing Deeper Transformers on Small Datasets

1 code implementation • ACL 2021 • Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Simon J. D. Prince, Yanshuai Cao

This work shows that this does not always need to be the case: with proper initialization and optimization, the benefits of very deep transformers can carry over to challenging tasks with small datasets, including Text-to-SQL semantic parsing and logical reading comprehension.

Reading Comprehension Semantic Parsing +2

Classifying Out-of-vocabulary Terms in a Domain-Specific Social Media Corpus

no code implementations • LREC 2016 • SoHyun Park, Afsaneh Fazly, Annie Lee, Br Seibel, on, Wenjie Zi, Paul Cook

We then propose a supervised approach to classify out-of-vocabulary terms according to these categories, drawing on features based on word embeddings, and linguistic knowledge of common properties of out-of-vocabulary terms.

General Classification Word Embeddings