BERT, or Bidirectional Encoder Representations from Transformers, improves upon standard Transformers by removing the unidirectionality constraint by using a masked language model (MLM) pre-training objective. The masked language model randomly masks some of the tokens from the input, and the objective is to predict the original vocabulary id of the masked word based only on its context. Unlike left-to-right language model pre-training, the MLM objective enables the representation to fuse the left and the right context, which allows us to pre-train a deep bidirectional Transformer. In addition to the masked language model, BERT uses a next sentence prediction task that jointly pre-trains text-pair representations.

There are two steps in BERT: pre-training and fine-tuning. During pre-training, the model is trained on unlabeled data over different pre-training tasks. For fine-tuning, the BERT model is first initialized with the pre-trained parameters, and all of the parameters are fine-tuned using labeled data from the downstream tasks. Each downstream task has separate fine-tuned models, even though they are initialized with the same pre-trained parameters.

Source: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Latest Papers

PAPER DATE
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Wei ZhuXiaoling WangXipeng QiuYuan NiGuotong Xie
2020-09-22
GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis
| Huaishao LuoLei JiTianrui LiNan DuanDaxin Jiang
2020-09-22
Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application
Chris J. KennedyGeoff BaconAlexander SahnClaudia von Vacano
2020-09-22
"When they say weed causes depression, but it's your fav antidepressant": Knowledge-aware Attention Framework for Relationship Extraction
Shweta YadavUsha LokalaRaminta DaniulaityteKrishnaprasad ThirunarayanFrancois LamyAmit Sheth
2020-09-21
Profile Consistency Identification for Open-domain Dialogue Agents
Haoyu SongYan WangWei-Nan ZhangZhengyu ZhaoTing LiuXiaojiang Liu
2020-09-21
Latin BERT: A Contextual Language Model for Classical Philology
David BammanPatrick J. Burns
2020-09-21
Dual-path CNN with Max Gated block for Text-Based Person Re-identification
Tinghuai MaMingming YangHuan RongYurong QianYurong QianYuan TianNajlaAl-Nabhan
2020-09-20
Longformer for MS MARCO Document Re-ranking Task
| Ivan SekulićAmir SoleimaniMohammad AliannejadiFabio Crestani
2020-09-20
Persian Ezafe Recognition Using Transformers and Its Role in Part-Of-Speech Tagging
| Ehsan DoostmohammadiMinoo NassajianAdel Rahimi
2020-09-20
VirtualFlow: Decoupling Deep Learning Model Execution from Underlying Hardware
Andrew OrHaoyu ZhangMichael J. Freedman
2020-09-20
Prior Art Search and Reranking for Generated Patent Text
Jieh-Sheng LeeJieh Hsiang
2020-09-19
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data
Jonathan PilaultAmine ElhattamiChristopher Pal
2020-09-19
Nominal Compound Chain Extraction: A New Task for Semantic-enriched Lexical Chain
Bobo LiHao FeiYafeng RenDonghong Ji
2020-09-19
Will it Unblend?
Yuval PinterCassandra L. JacobsJacob Eisenstein
2020-09-18
FarsTail: A Persian Natural Language Inference Dataset
| Hossein AmirkhaniMohammad Azari JafariAzadeh AmirakZohreh PourjafariSoroush Faridan JahromiZeinab Kouhkan
2020-09-18
The birth of Romanian BERT
Stefan Daniel DumitrescuAndrei-Marius AvramSampo Pyysalo
2020-09-18
fastHan: A BERT-based Joint Many-Task Toolkit for Chinese NLP
Zhichao GengHang YanXipeng QiuXuanjing Huang
2020-09-18
NEU at WNUT-2020 Task 2: Data Augmentation To Tell BERT That Death Is Not Necessarily Informative
Kumud Chauhan
2020-09-18
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
Bingbing LiZhenglun KongTianyun ZhangJi LiZhengang LiHang LiuCaiwen Ding
2020-09-17
Multi^2OIE: Multilingual Open Information Extraction based on Multi-Head Attention with BERT
Youngbin RoYukyung LeePilsung Kang
2020-09-17
DSC IIT-ISM at SemEval-2020 Task 6: Boosting BERT with Dependencies for Definition Extraction
| Aadarsh SinghPriyanshu KumarAman Sinha
2020-09-17
Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA
Ieva StaliūnaitėIgnacio Iacobacci
2020-09-17
A Multimodal Memes Classification: A Survey and Open Research Issues
Tariq Habib AfridiAftab AlamMuhammad Numan KhanJawad KhanYoung-Koo Lee
2020-09-17
Solomon at SemEval-2020 Task 11: Ensemble Architecture for Fine-Tuned Propaganda Detection in News Articles
Mayank RajAjay JaiswalRohit R. RAnkita GuptaSudeep Kumar SahooVertika SrivastavaYeon Hyang Kim
2020-09-16
Simplified TinyBERT: Knowledge Distillation for Document Retrieval
Xuanang ChenBen HeKai HuiLe SunYingfei Sun
2020-09-16
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation
| Jian GuanMinlie Huang
2020-09-16
Deep Learning Approaches for Extracting Adverse Events and Indications of Dietary Supplements from Clinical Text
Yadan FanSicheng ZhouYifan LiRui Zhang
2020-09-16
Achieving Real-Time Execution of Transformer-based Large-scale Models on Mobile with Compiler-aware Neural Architecture Optimization
Wei NiuZhenglun KongGeng YuanWeiwen JiangJiexiong GuanCaiwen DingPu ZhaoSijia LiuBin RenYanzhi Wang
2020-09-15
MLMLM: Link Prediction with Mean Likelihood Masked Language Model
Louis ClouatrePhilippe TrempeAmal ZouaqSarath Chandar
2020-09-15
Event Presence Prediction Helps Trigger Detection Across Languages
Parul AwasthyTahira NaseemJian NiTaesun MoonRadu Florian
2020-09-15
Lessons Learned from Applying off-the-shelf BERT: There is no SilverBullet
Victor MakarenkovLior Rokach
2020-09-15
BERT-QE: Contextualized Query Expansion for Document Re-ranking
Zhi ZhengKai HuiBen HeXianpei HanLe SunAndrew Yates
2020-09-15
Can Fine-tuning Pre-trained Models Lead to Perfect NLP? A Study of the Generalizability of Relation Extraction
Ningyu ZhangLuoqiu LiShumin DengHaiyang YuXu ChengWei ZhangHuajun Chen
2020-09-14
Beyond Accuracy: ROI-driven Data Analytics of Empirical Data
Gouri DeshpandeGuenther Ruhe
2020-09-14
BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks
Tongwen HuangQingyun SheJunlin Zhang
2020-09-13
CIA_NITT at WNUT-2020 Task 2: Classification of COVID-19 Tweets Using Pre-trained Language Models
Yandrapati Prakash BabuRajagopal Eswari
2020-09-12
Country Image in COVID-19 Pandemic: A Case Study of China
Huimin ChenZeyu ZhuFanchao QiYining YeZhiyuan LiuMaosong SunJianbin Jin
2020-09-12
Fine-tuning Pre-trained Contextual Embeddings for Citation Content Analysis in Scholarly Publication
Haihua ChenHuyen Nguyen
2020-09-12
Compressed Deep Networks: Goodbye SVD, Hello Robust Low-Rank Approximation
Murad TukanAlaa MaaloufMatan WekslerDan Feldman
2020-09-11
UPB at SemEval-2020 Task 6: Pretrained Language Models for DefinitionExtraction
Andrei-Marius AvramDumitru-Clementin CercelCostin-Gabriel Chiru
2020-09-11
UPB at SemEval-2020 Task 11: Propaganda Detection with Domain-Specific Trained BERT
Andrei ParaschivDumitru-Clementin CercelMihai Dascalu
2020-09-11
A Comparison of LSTM and BERT for Small Corpus
Aysu Ezen-Can
2020-09-11
Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection
| Taesun WhangDongyub LeeDongsuk OhChanhee LeeKijong HanDong-hun LeeSaebyeok Lee
2020-09-10
Modern Methods for Text Generation
| Dimas Munoz Montesinos
2020-09-10
Investigating Gender Bias in BERT
Rishabh BhardwajNavonil MajumderSoujanya Poria
2020-09-10
Pay Attention when Required
Swetha MandavaSzymon MigaczAlex Fit Florea
2020-09-09
Comparative Study of Language Models on Cross-Domain Data with Model Agnostic Explainability
Mayank ChhipaHrushikesh Mahesh VazurkarAbhijeet KumarMridul Mishra
2020-09-09
ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model
Zhengjie HuangShikun FengWeiyue SuXuyi ChenShuohuan WangJiaxiang LiuXuan OuyangYu Sun
2020-09-08
E-BERT: A Phrase and Product Knowledge Enhanced Language Model for E-commerce
Denghui ZhangZixuan YuanYanchi LiuFuzhen ZhuangHui Xiong
2020-09-07
EdinburghNLP at WNUT-2020 Task 2: Leveraging Transformers with Generalized Augmentation for Identifying Informativeness in COVID-19 Tweets
Nickil Maveli
2020-09-06
QiaoNing at SemEval-2020 Task 4: Commonsense Validation and Explanation system based on ensemble of language model
Pai Liu
2020-09-06
Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models
Evan WilliamsPaul RodriguesValerie Novak
2020-09-05
Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading
Sasi Kiran GaddipatiDeebul NairPaul G. Plöger
2020-09-02
Automatic Assignment of Radiology Examination Protocols Using Pre-trained Language Models with Knowledge Distillation
Wilson LauLaura AaltonenMartin GunnMeliha Yetisgen
2020-09-01
Sentimental LIAR: Extended Corpus and Deep Learning Models for Fake Claim Classification
Bibek UpadhayayVahid Behzadan
2020-09-01
BiTT: Bidirectional Tree Tagging for Joint Extraction of Overlapping Entities and Relations
Xukun LuoWeijie LiuMeng MaPing Wang
2020-08-31
SocCogCom at SemEval-2020 Task 11: Characterizing and Detecting Propaganda using Sentence-Level Emotional Salience Features
Gangeshwar KrishnamurthyRaj Kumar GuptaYinping Yang
2020-08-29
Knowledge Efficient Deep Learning for Natural Language Processing
Hai Wang
2020-08-28
Rethinking the objectives of extractive question answering
Martin FajcikJosef JonSantosh KesirajuPavel Smrz
2020-08-28
Entity and Evidence Guided Relation Extraction for DocRED
Kevin HuangGuangtao WangTengyu MaJing Huang
2020-08-27
GREEK-BERT: The Greeks visiting Sesame Street
John KoutsikakisIlias ChalkidisProdromos MalakasiotisIon Androutsopoulos
2020-08-27
Query Focused Multi-document Summarisation of Biomedical Texts
Diego MollaChristopher JonesVincent Nguyen
2020-08-27
MultiGBS: A multi-layer graph approach to biomedical summarization
Ensieh DavoodijamNasser GhadiriMaryam Lotfi ShahrezaFabio Rinaldi
2020-08-27
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization
Xinsong ZhangHang Li
2020-08-27
Language Models and Word Sense Disambiguation: An Overview and Analysis
| Daniel LoureiroKiamehr RezaeeMohammad Taher PilehvarJose Camacho-Collados
2020-08-26
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm
Hanlin TangShaoduo GanSamyam RajbhandariXiangru LianJi LiuYuxiong HeCe Zhang
2020-08-26
Conceptualized Representation Learning for Chinese Biomedical Text Mining
| Ningyu ZhangQianghuai JiaKangping YinLiang DongFeng GaoNengwei Hua
2020-08-25
YNU-HPCC at SemEval-2020 Task 11: LSTM Network for Detection of Propaganda Techniques in News Articles
Jiaxu DaoJin WangXuejie Zhang
2020-08-24
Prediction of ICD Codes with Clinical BERT Embeddings and Text Augmentation with Label Balancing using MIMIC-III
Brent BisedaGaurav DesaiHaifeng LinAnish Philip
2020-08-24
Two Stages Approach for Tweet Engagement Prediction
Amine DadounIsmail HarrandoPasquale LisenaAlison ReboudRaphael Troncy
2020-08-24
Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources
Taolin ZhangChengyu WangMinghui QiuBite YangXiaofeng HeJun Huang
2020-08-24
syrapropa at SemEval-2020 Task 11: BERT-based Models Design For Propagandistic Technique and Span Detection
Jinfen LiLu Xiao
2020-08-24
DUTH at SemEval-2020 Task 11: BERT with Entity Mapping for Propaganda Classification
Anastasios BairaktarisSymeon SymeonidisAvi Arampatzis
2020-08-22
CyberWallE at SemEval-2020 Task 11: An Analysis of Feature Engineering for Ensemble Models for Propaganda Detection
| Verena BlaschkeMaxim KorniyenkoSam Tureski
2020-08-22
HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection
Meghana BhangeNirant Kasliwal
2020-08-22
Applications of BERT Based Sequence Tagging Models on Chinese Medical Text Attributes Extraction
Gang ZhaoTeng ZhangChenxiao WangPing LvJi Wu
2020-08-22
FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
| Omar MossadAmgad AhmedAnandharaju RajuHari KarthikeyanZayed Ahmed
2020-08-22
Abstractive Summarization of Spoken andWritten Instructions with BERT
| Alexandra SavelievaBryan Au-YeungVasanth Ramani
2020-08-21
Adapting Event Extractors to Medical Data: Bridging the Covariate Shift
Aakanksha NaikJill LehmanCarolyn Rose
2020-08-21
An Experimental Study of Deep Neural Network Models for Vietnamese Multiple-Choice Reading Comprehension
Son T. LuuKiet Van NguyenAnh Gia-Tuan NguyenNgan Luu-Thuy Nguyen
2020-08-20
UoB at SemEval-2020 Task 12: Boosting BERT with Corpus Level Information
Wah Meng LimHarish Tayyar Madabushi
2020-08-19
Ranking Clarification Questions via Natural Language Inference
Vaibhav KumarVikas RaunakJamie Callan
2020-08-18
Stock Index Prediction with Multi-task Learning and Word Polarity Over Time
Yue ZhouKerstin Voigt
2020-08-17
DeVLBert: Learning Deconfounded Visio-Linguistic Representations
| Shengyu ZhangTan JiangTan WangKun KuangZhou ZhaoJianke ZhuJin YuHongxia YangFei Wu
2020-08-16
Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition
Henry TsaiJayden OoiChun-Sung FerngHyung Won ChungJason Riesa
2020-08-15
Jointly Fine-Tuning "BERT-like" Self Supervised Models to Improve Multimodal Speech Emotion Recognition
| Shamane SiriwardhanaAndrew ReisRivindu WeerasekeraSuranga Nanayakkara
2020-08-15
Jointly Fine-Tuning “BERT-like” Self Supervised Models to Improve Multimodal Speech Emotion Recognition
| Shamane SiriwardhanaAndrew ReisRivindu WeerasekeraSuranga Nanayakkara
2020-08-15
Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT model
Marzieh MozafariReza FarahbakhshNoel Crespi
2020-08-14
MICE: Mining Idioms with Contextual Embeddings
| Tadej ŠkvorcPolona GantarMarko Robnik-Šikonja
2020-08-13
ANDES at SemEval-2020 Task 12: A jointly-trained BERT multilingual model for offensive language detection
| Juan Manuel PérezAymé ArangoFranco Luque
2020-08-13
Variance-reduced Language Pretraining via a Mask Proposal Network
Liang Chen
2020-08-12
FireBERT: Hardening BERT-based classifiers against adversarial attack
Gunnar MeinKevin HartmanAndrew Morris
2020-08-10
KR-BERT: A Small-Scale Korean-Specific Language Model
| Sangah LeeHansol JangYunmee BaikSuzi ParkHyopil Shin
2020-08-10
Does BERT Solve Commonsense Task via Commonsense Knowledge?
Leyang CuiSijie ChengYu WuYue Zhang
2020-08-10
Beyond Lexical: A Semantic Retrieval Framework for Textual SearchEngine
Kuan FangLong ZhaoZhan ShenRuiXing WangRiKang ZhourLiWen Fan
2020-08-10
GANBERT: Generative Adversarial Networks with Bidirectional Encoder Representations from Transformers for MRI to PET synthesis
Hoo-Chang ShinAlvin IhsaniSwetha MandavaSharath Turuvekere SreenivasChristopher ForsterJiook ChaAlzheimer's Disease Neuroimaging Initiative
2020-08-10
Distilling the Knowledge of BERT for Sequence-to-Sequence ASR
| Hayato FutamiHirofumi InagumaSei UenoMasato MimuraShinsuke SakaiTatsuya Kawahara
2020-08-09
Fast and Accurate Neural CRF Constituency Parsing
| Yu ZhangHouquan ZhouZhenghua Li
2020-08-09
SemEval-2020 Task 10: Emphasis Selection for Written Text in Visual Media
Amirreza ShiraniFranck DernoncourtNedim LipkaPaul AsenteJose EchevarriaThamar Solorio
2020-08-07
ConvBERT: Improving BERT with Span-based Dynamic Convolution
Zihang JiangWeihao YuDaquan ZhouYunpeng ChenJiashi FengShuicheng Yan
2020-08-06
DeText: A Deep Text Ranking Framework with BERT
| Weiwei GuoXiaowei LiuSida WangHuiji GaoAnanth SankarZimeng YangQi GuoLiang ZhangBo LongBee-Chung ChenDeepak Agarwal
2020-08-06
aschern at SemEval-2020 Task 11: It Takes Three to Tango: RoBERTa, CRF, and Transfer Learning
| Anton ChernyavskiyDmitry IlvovskyPreslav Nakov
2020-08-06
Taking Notes on the Fly Helps BERT Pre-training
Qiyu WuChen XingYatao LiGuolin KeDi HeTie-Yan Liu
2020-08-04
NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer
Hwijeen AhnJimin SunChan Young ParkJungyun Seo
2020-08-04
Improving One-stage Visual Grounding by Recursive Sub-query Construction
| Zhengyuan YangTianlang ChenLiwei WangJiebo Luo
2020-08-03
[email protected] at SemEval-2020 Task 12: Multilingual or language-specific BERT?
Marc PàmiesEmily ÖhmanKaisla KajavaJörg Tiedemann
2020-08-03
Multi-node Bert-pretraining: Cost-efficient Approach
Jiahuang LinXin LiGennady Pekhimenko
2020-08-01
On Learning Universal Representations Across Languages
Xiangpeng WeiYue HuRongxiang WengLuxi XingHeng YuWeihua Luo
2020-07-31
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Yu GuRobert TinnHao ChengMichael LucasNaoto UsuyamaXiaodong LiuTristan NaumannJianfeng GaoHoifung Poon
2020-07-31
Model Reduction of Shallow CNN Model for Reliable Deployment of Information Extraction from Medical Reports
Abhishek K DubeyAlina PelusoJacob HinkleDevanshu AgarawalZilong Tan
2020-07-31
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
| Gustavo PenhaClaudia Hauff
2020-07-30
Depressive, Drug Abusive, or Informative: Knowledge-aware Study of News Exposure during COVID-19 Outbreak
Amanuel AlamboManas GaurKrishnaprasad Thirunarayan
2020-07-30
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
| Shayne LongpreYi LuJoachim Daiber
2020-07-30
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
TJ TsaiKevin Ji
2020-07-29
Improving Results on Russian Sentiment Datasets
| Anton GolubevNatalia Loukachevitch
2020-07-28
BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Martin FajcikJosef JonMartin DocekalPavel Smrz
2020-07-28
Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge
Hoang Manh HungHyung-Jeong YangSoo-Hyung KimGuee-Sang Lee
2020-07-28
GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection
Sajad SotudehTong XiangHao-Ren YaoSean MacAvaneyEugene YangNazli GoharianOphir Frieder
2020-07-28
Deep Learning Brasil -- NLP at SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets
Manoel Veríssimo dos Santos NetoAyrton Denner da Silva AmaralNádia Félix Felipe da SilvaAnderson da Silva Soares
2020-07-28
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media
| Ali SafayaMoutasem AbdullatifDeniz Yuret
2020-07-26
Reed at SemEval-2020 Task 9: Fine-Tuning and Bag-of-Words Approaches to Code-Mixed Sentiment Analysis
Vinay GopalanMark Hopkins
2020-07-26
To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection
Aparna BalagopalanBenjamin EyreFrank RudziczJekaterina Novikova
2020-07-26
MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning
Aina Garí SolerMarianna Apidianaki
2020-07-24
Product Title Generation for Conversational Systems using BERT
Mansi Ranjit ManeShashank KediaAditya ManthaStephen GuoKannan Achan
2020-07-23
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
| Tianlong ChenJonathan FrankleShiyu ChangSijia LiuYang ZhangZhangyang WangMichael Carbin
2020-07-23
IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Vishal KeswaniSakshi SinghAshutosh Modi
2020-07-22
Multi-task learning for natural language processing in the 2020s: where are we going?
Joseph WorshamJugal Kalita
2020-07-22
problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches
Karishma LaudJagriti SinghRandeep Kumar SahuAshutosh Modi
2020-07-21
newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification
| Paramansh SinghSiraj SandhuSubham KumarAshutosh Modi
2020-07-21
Word Representation for Rhythms
Tongyu LuLyucheng YanGus Xia
2020-07-21
Understanding BERT Rankers Under Distillation
Luyu GaoZhuyun DaiJamie Callan
2020-07-21
A Comparison of Supervised Learning to Match Methods for Product Search
| Fatemeh SarviNikos VoskaridesLois MooimanSebastian SchelterMaarten de Rijke
2020-07-20
Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
Diego de Vargas FeijoViviane Pereira Moreira
2020-07-19
Multi-Perspective Semantic Information Retrieval in the Biomedical Domain
Samarth Rawal
2020-07-17
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-16
Translate Reverberated Speech to Anechoic Ones: Speech Dereverberation with BERT
Yang Jiao
2020-07-16
Hopfield Networks is All You Need
| Hubert RamsauerBernhard SchäflJohannes LehnerPhilipp SeidlMichael WidrichLukas GruberMarkus HolzleitnerMilena PavlovićGeir Kjetil SandveVictor GreiffDavid KreilMichael KoppGünter KlambauerJohannes BrandstetterSepp Hochreiter
2020-07-16
AdapterHub: A Framework for Adapting Transformers
| Jonas PfeifferAndreas RückléClifton PothAishwarya KamathIvan VulićSebastian RuderKyunghyun ChoIryna Gurevych
2020-07-15
Multimodal Word Sense Disambiguation in Creative Practice
Manuel Ladron de GuevaraChristopher GeorgeAkshat GuptaDaragh ByrneRamesh Krishnamurti
2020-07-15
Logic Constrained Pointer Networks for Interpretable Textual Similarity
| Subhadeep MajiRohan KumarManish BansalKalyani RoyPawan Goyal
2020-07-15
Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks
Pavel BlinovManvel AvetisianVladimir KokhDmitry UmerenkovAlexander Tuzhilin
2020-07-15
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
| Alberto Barron-CedenoTamer ElsayedPreslav NakovGiovanni Da San MartinoMaram HasanainReem SuwailehFatima HaouariNikolay BabulkovBayan HamdanAlex NikolovShaden ShaarZien Sheikh Ali
2020-07-15
Deep Reinforced Query Reformulation for Information Retrieval
Xiao WangCraig MacdonaldIadh Ounis
2020-07-15
Fast and Accurate Neural CRF Constituency Parsing
| Yu ZhangHouquan ZhouZhenghua Li
2020-07-14
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-14
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Lifu TuGarima LalwaniSpandana GellaHe He
2020-07-14
Can neural networks acquire a structural bias from raw linguistic data?
Alex WarstadtSamuel R. Bowman
2020-07-14
Emoji Prediction: Extensions and Benchmarking
Weicheng MaRuibo LiuLili WangSoroush Vosoughi
2020-07-14
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
Shauharda KhadkaEstelle AflaloMattias MarderAvrech Ben-DavidSantiago MiretHanlin TangShie MannorTamir HazanSomdeb Majumdar
2020-07-14
Add a SideNet to your MainNet
Adrien Morisot
2020-07-14
An Enhanced Text Classification to Explore Health based Indian Government Policy Tweets
Aarzoo DhimanDurga Toshniwal
2020-07-13
Generative Graph Perturbations for Scene Graph Prediction
Boris KnyazevHarm de VriesCătălina CangeaGraham W. TaylorAaron CourvilleEugene Belilovsky
2020-07-11
BERT Learns (and Teaches) Chemistry
Josh PayneMario SroujiDian Ang YapVineet Kosaraju
2020-07-11
To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection
Kristian MiokBlaz SkrljDaniela ZaharieMarko Robnik-Sikonja
2020-07-10
BISON:BM25-weighted Self-Attention Framework for Multi-Fields Document Search
| Xuan ShanChuanjie LiuYiqian XiaQi ChenYusi ZhangAngen LuoYuxiang Luo
2020-07-10
Multi-Dialect Arabic BERT for Country-Level Dialect Identification
| Bashar TalafhaMohammad AliMuhy Eddin Za'terHaitham SeelawiIbraheem TuffahaMostafa SamirWael FarhanHussein T. Al-Natsheh
2020-07-10
Contrastive Code Representation Learning
| Paras JainAjay JainTianjun ZhangPieter AbbeelJoseph E. GonzalezIon Stoica
2020-07-09
Fast Transformers with Clustered Attention
| Apoorv VyasAngelos KatharopoulosFrançois Fleuret
2020-07-09
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
Jong Won Park
2020-07-07
Exploring Heterogeneous Information Networks via Pre-Training
Yang FangXiang ZhaoWeidong Xiao
2020-07-07
Deep Contextual Embeddings for Address Classification in E-commerce
Shreyas MangalgiLakshya KumarRavindra Babu Tallamraju
2020-07-06
Text Data Augmentation: Towards better detection of spear-phishing emails
Mehdi ReginaMaxime MeyerSébastien Goutal
2020-07-04
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-04
Language-agnostic BERT Sentence Embedding
| Fangxiaoyu FengYinfei YangDaniel CerNaveen ArivazhaganWei Wang
2020-07-03
Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning
Pavel DenisovNgoc Thang Vu
2020-07-03
Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer
Kateřina MackováMilan Straka
2020-07-03
Playing with Words at the National Library of Sweden -- Making a Swedish BERT
| Martin MalmstenLove BörjesonChris Haffenden
2020-07-03
MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
Yusi ZhangChuanjie LiuAngen LuoHui XueXuan ShanYuxiang LuoYiqian XiaYuanchi YanHaidong Wang
2020-07-03
Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey
Shivaji AlaparthiManit Mishra
2020-07-02
The Impact of Explanations on AI Competency Prediction in VQA
Kamran AlipourArijit RayXiao LinJurgen P. SchulzeYi YaoGiedrius T. Burachas
2020-07-02
Improving Event Detection using Contextual Word and Sentence Embeddings
Mariano MaisonnaveFernando DelbiancoFernando TohméAna MaguitmanEvangelos Milios
2020-07-02
Unsupervised FAQ Retrieval with Question Generation and BERT
Yosi MassBoaz CarmeliHaggai RoitmanDavid Konopnicki
2020-07-01
GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
| Danilo CroceGiuseppe CastellucciRoberto Basili
2020-07-01
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur RahmanMd Kamrul HasanSangwu LeeAmirAli Bagher ZadehChengfeng MaoLouis-Philippe MorencyEhsan Hoque
2020-07-01
Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis
Minh Hieu PhanPhilip O. Ogunbona
2020-07-01
Roles and Utilization of Attention Heads in Transformer-based Neural Language Models
Jae-young JoSung-Hyon Myaeng
2020-07-01
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis
Chunning DuHaifeng SunJingyu WangQi QiJianxin Liao
2020-07-01
How does BERT's attention change when you fine-tune? An analysis methodology and a case study in negation scope
Yiyun ZhaoSteven Bethard
2020-07-01
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-07-01
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences
Yi TayDonovan OngJie FuAlvin ChanNancy ChenAnh Tuan LuuChris Pal
2020-07-01
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-01
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study
Xinyu XingXiaosheng FanXiaojun Wan
2020-07-01
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fern{\'a}ndez-Gonz{\'a}lezCarlos G{\'o}mez-Rodr{\'\i}guez
2020-07-01
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection
Nicole PeineltDong NguyenMaria Liakata
2020-07-01
Understanding Advertisements with BERT
Kanika KalraBhargav KurmaSilpa Vadakkeeveetil SreelathaManasi PatwardhanKarShirish e
2020-07-01
Feature Projection for Improved Text Classification
Qi QinWenpeng HuBing Liu
2020-07-01
A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization
Dongfang XuZeyu ZhangSteven Bethard
2020-07-01
Revisiting Higher-Order Dependency Parsers
Erick FonsecaAndr{\'e} F. T. Martins
2020-07-01
SUPP.AI: finding evidence for supplement-drug interactions
Lucy WangOyvind TafjordArman CohanSarthak JainSam SkjonsbergCarissa SchoenickNick BotnerWaleed Ammar
2020-07-01
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models
Pia Sommerauer
2020-07-01
A Simple and Effective Dependency Parser for Telugu
Sneha NallaniManish ShrivastavaDipti Sharma
2020-07-01
Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup
Jishnu Ray ChowdhuryCornelia CarageaDoina Caragea
2020-07-01
Should You Fine-Tune BERT for Automated Essay Scoring?
Elijah MayfieldAlan W Black
2020-07-01
A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction
Chen LinTimothy MillerDmitriy DligachFarig SadequeSteven BethardGuergana Savova
2020-07-01
Evaluating the Utility of Model Configurations and Data Augmentation on Clinical Semantic Textual Similarity
Yuxia WangFei LiuKarin VerspoorTimothy Baldwin
2020-07-01
Item-based Collaborative Filtering with BERT
Tian WangYuyangzi Fu
2020-07-01
Sarcasm Identification and Detection in Conversion Context using BERT
Kalaivani A.Thenmozhi D.
2020-07-01
Neural Sarcasm Detection using Conversation Context
Nikhil Jaiswal
2020-07-01
Context-Aware Sarcasm Detection Using BERT
Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-07-01
Character aware models with similarity learning for metaphor detection
Tarun KumarYashvardhan Sharma
2020-07-01
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Hongyu GongKshitij GuptaAkriti JainSuma Bhat
2020-07-01
Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task
Xianyang ChenChee Wee (Ben) LeongMichael FlorBeata Beigman Klebanov
2020-07-01
Metaphor Detection Using Contextual Word Embeddings From Transformers
Jerry LiuNathan O{'}HaraAlex RubinerRachel DraelosCynthia Rudin
2020-07-01
A Transformer Approach to Contextual Sarcasm Detection in Twitter
Hunter GregorySteven LiPouya MohammadiNatalie TarnRachel DraelosCynthia Rudin
2020-07-01
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
Jenna KanervaFilip GinterSampo Pyysalo
2020-07-01
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-07-01
RobertNLP at the IWPT 2020 Shared Task: Surprisingly Simple Enhanced UD Parsing for English
Stefan Gr{\"u}newaldAnnemarie Friedrich
2020-07-01
The HW-TSC Video Speech Translation System at IWSLT 2020
Minghan WangHao YangYao DengYing QinLizhi LeiDaimeng WeiHengchao ShangNing XieXiaochun LiJiaxian Guo
2020-07-01
CopyBERT: A Unified Approach to Question Generation with Self-Attention
Stalin VaranasiSaadullah AminGuenter Neumann
2020-07-01
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-01
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Ashutosh AdhikariAchyudh RamRaphael TangWilliam L. HamiltonJimmy Lin
2020-07-01
Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference
Cemil CengizDeniz Yuret
2020-07-01
A Metric Learning Approach to Misogyny Categorization
Juan Manuel CoriaSahar GhannaySophie RossetHerv{\'e} Bredin
2020-07-01
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
Alessio MiaschiFelice Dell{'}Orletta
2020-07-01
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-01
Getting the \#\#life out of living: How Adequate Are Word-Pieces for Modelling Complex Morphology?
Stav KleinReut Tsarfaty
2020-07-01
SentiTel: TABSA for Twitter reviews on Uganda Telecoms
David KabiitoJoyce Nakatumba Nabende
2020-07-01
Adversarial Evaluation of BERT for Biomedical Named Entity Recognition
Vladimir AraujoAndr{\'e}s CarvalloDenis Parra
2020-07-01
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer
Jianfei YuJing JiangLi YangRui Xia
2020-07-01
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions
Hannah CraigheadAndrew CainesPaula ButteryHelen Yannakoudakis
2020-07-01
DocVQA: A Dataset for VQA on Document Images
| Minesh MathewDimosthenis KaratzasR. ManmathaC. V. Jawahar
2020-07-01
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
Denny ZhouMao YeChen ChenTianjian MengMingxing TanXiaodan SongQuoc LeQiang LiuDale Schuurmans
2020-07-01
SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models
Eliane M. De Bortoli FáveroDalcimar CasanovaAndrey Ricardo Pimentel
2020-06-30
Data Movement Is All You Need: A Case Study on Optimizing Transformers
| Andrei IvanovNikoli DrydenTal Ben-NunShigang LiTorsten Hoefler
2020-06-30
Segmentation Approach for Coreference Resolution Task
Aref JafariAli Ghodsi
2020-06-30
Want to Identify, Extract and Normalize Adverse Drug Reactions in Tweets? Use RoBERTa
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-29
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models
Viet Bui TheOanh Tran ThiPhuong Le-Hong
2020-06-29
Interpreting Hierarchical Linguistic Interactions in DNNs
Die ZhangHuilin ZhouXiaoyi BaoDa HuoRuizhao ChenXu ChengHao ZhangMengyue WuQuanshi Zhang
2020-06-29
Rethinking Positional Encoding in Language Pre-training
| Guolin KeDi HeTie-Yan Liu
2020-06-28
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
| Chen LiangYue YuHaoming JiangSiawpeng ErRuijia WangTuo ZhaoChao Zhang
2020-06-28
FastSpec: Scalable Generation and Detection of Spectre Gadgets Using Neural Embeddings
| M. Caner TolKoray YurtsevenBerk GulmezogluBerk Sunar
2020-06-25
Normalizing Text using Language Modelling based on Phonetics and String Similarity
Fenil DoshiJimit GandhiDeep GosaliaSudhir Bagul
2020-06-25
LSBert: A Simple Framework for Lexical Simplification
| Jipeng QiangYun LiYi ZhuYunhao YuanXindong Wu
2020-06-25
Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
Shuai ZhengHaibin LinSheng ZhaMu Li
2020-06-24
Efficient Constituency Parsing by Pointing
Thanh-Tung NguyenXuan-Phi NguyenShafiq JotyXiaoli Li
2020-06-24
ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
| BingningWangTing YaoQi ZhangJingfang XuXiaochuan Wang
2020-06-22
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Shijing SiRui WangJedrek WosikHao ZhangDavid DovGuoyin WangRicardo HenaoLawrence Carin
2020-06-22
Sarcasm Detection in Tweets with BERT and GloVe Embeddings
Akshay KhatriPranav PDr. Anand Kumar M
2020-06-20
New Vietnamese Corpus for Machine ReadingComprehension of Health News Articles
Kiet Van NguyenDuc-Vu NguyenAnh Gia-Tuan NguyenNgan Luu-Thuy Nguyen
2020-06-19
A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19
| David OnianiYanshan Wang
2020-06-19
Exploring the BERT Cross-Lingual Transferability: a Case Study in Reading Comprehension
Konovalov V. P.Gulyaev P. A.Sorokin A. A.Kuratov Y. M.Burtsev M. S.
2020-06-17
Tagging and parsing of multidomain collections
| Alexey SorokinIvan SmurovDenis Kirianov
2020-06-17
Improving accuracy and speeding up Document Image Classification through parallel systems
| Javier FerrandoJuan Luis DominguezJordi TorresRaul GarciaDavid GarciaDaniel GarridoJordi CortadaMateo Valero
2020-06-16
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
| Eyal Ben-DavidCarmel RabinovitzRoi Reichart
2020-06-16
The SPPD System for Schema Guided Dialogue State Tracking Challenge
Miao LiHaoqi XiongYunbo Cao
2020-06-16
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation
Kellie WebsterEmily Pitler
2020-06-16
End-to-End Code Switching Language Models for Automatic Speech Recognition
Ahan M. R.Shreyas Sunil Kulkarni
2020-06-16
FinBERT: A Pretrained Language Model for Financial Communications
| Yi YangMark Christopher Siy UYAllen Huang
2020-06-15
Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code)
Subramaniam KazhuparambilAbhishek Kaushik
2020-06-15
FinEst BERT and CroSloEngual BERT: less is more in multilingual models
Matej UlčarMarko Robnik-Šikonja
2020-06-14
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
Abrhalei TelaAbraham WoubieVille Hautamaki
2020-06-13
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
Pedro Javier Ortiz SuárezLaurent RomaryBenoît Sagot
2020-06-11
Revisiting Few-sample BERT Fine-tuning
| Tianyi ZhangFelix WuArzoo KatiyarKilian Q. WeinbergerYoav Artzi
2020-06-10
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
| Marius MosbachMaksym AndriushchenkoDietrich Klakow
2020-06-08
Pre-training Polish Transformer-based Language Models at Scale
| Sławomir DadasMichał PerełkiewiczRafał Poświata
2020-06-07
Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-07
Accelerating Natural Language Understanding in Task-Oriented Dialog
Ojas AhujaShrey Desai
2020-06-05
UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings
Milan StrakaJana Straková
2020-06-05
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
| Pengcheng HeXiaodong LiuJianfeng GaoWeizhu Chen
2020-06-05
The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain
Annemarie FriedrichHeike AdelFederico TomazicJohannes HingerlRenou BenteauAnika MaruscykLukas Lange
2020-06-04
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
| Virapat KieuvongngamBowen TanYiming Niu
2020-06-03
WikiBERT models: deep transfer learning for many languages
Sampo PyysaloJenna KanervaAntti VirtanenFilip Ginter
2020-06-02
Question Answering on Scholarly Knowledge Graphs
Mohamad Yaser JaradehMarkus StockerSören Auer
2020-06-02
A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension
Jie CaiZhengzhou ZhuPing NieQian Liu
2020-06-02
BERT Based Multilingual Machine Comprehension in English and Hindi
| Somil GuptaNilesh Khade
2020-06-02
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT
Jouni LuomaSampo Pyysalo
2020-06-02
Position Masking for Language Models
Andy WagnerTiyasa MitraMrinal IyerGodfrey Da CostaMarc Tremblay
2020-06-02
Emergence of Separable Manifolds in Deep Language Representations
Jonathan MamouHang LeMiguel Del RioCory StephensonHanlin TangYoon KimSueYeon Chung
2020-06-01
Conversational Machine Comprehension: a Literature Review
Somil GuptaBhanu Pratap Singh Rawat
2020-06-01
When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions
Yanai ElazarShauli RavfogelAlon JacoviYoav Goldberg
2020-06-01
An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
Shi-Yan WengTien-Hong LoBerlin Chen
2020-06-01
BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text
Tanvi DaduKartikey PantRadhika Mamidi
2020-06-01
Introduction d'informations s\'emantiques dans un syst\`eme de reconnaissance de la parole (Despite spectacular advances in recent years, the Automatic Speech Recognition (ASR) systems still make mistakes, especially in noisy environments)
St{\'e}phane LevelIrina IllinaDominique Fohr
2020-06-01
Les mod\`eles de langue contextuels Camembert pour le fran\ccais : impact de la taille et de l'h\'et\'erog\'en\'eit\'e des donn\'ees d'entrainement (C AMEM BERT Contextual Language Models for French: Impact of Training Data Size and Heterogeneity )
Louis MartinBenjamin MullerPedro Javier Ortiz Su{\'a}rezYoann DupontLaurent Romary{\'E}ric Villemonte de la ClergerieBeno{\^\i}t SagotDjam{\'e} Seddah
2020-06-01
Qu'apporte BERT \`a l'analyse syntaxique en constituants discontinus ? Une suite de tests pour \'evaluer les pr\'edictions de structures syntaxiques discontinues en anglais (What does BERT contribute to discontinuous constituency parsing ? A test suite to evaluate discontinuous constituency structure predictions in English)
Maximin Coavoux
2020-06-01
\'Etude des variations s\'emantiques \`a travers plusieurs dimensions (Studying semantic variations through several dimensions )
Syrielle MontariolAlex Allauzenre
2020-06-01
R\'e-entra\^\iner ou entra\^\iner soi-m\^eme ? Strat\'egies de pr\'e-entra\^\inement de BERT en domaine m\'edical (Re-train or train from scratch ? Pre-training strategies for BERT in the medical domain )
Hicham El Boukkouri
2020-06-01
Neural Entity Linking: A Survey of Models based on Deep Learning
| Ozge SevgiliArtem ShelmanovMikhail ArkhipovAlexander PanchenkoChris Biemann
2020-05-31
"Judge me by my size (noun), do you?'' YodaLib: A Demographic-Aware Humor Generation Framework
Aparna GarimellaCarmen BaneaNabil HossainRada Mihalcea
2020-05-31
BPGC at SemEval-2020 Task 11: Propaganda Detection in News Articles with Multi-Granularity Knowledge Sharing and Linguistic Features based Ensemble Learning
Rajaswa PatilSomesh SinghSwati Agarwal
2020-05-31
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
Siddhant MahurkarRajaswa Patil
2020-05-31
Detecting Problem Statements in Peer Assessments
Yunkai XiaoGabriel ZingleQinjin JiaHarsh R. ShahYi ZhangTianyi LiMohsin KarovaliyaWeixiang ZhaoYang SongJie JiAshwin BalasubramaniamHarshit PatelPriyankha BhalasubbramanianVikram PatelEdward F. Gehringer
2020-05-30
Using Large Pretrained Language Models for Answering User Queries from Product Specifications
Kalyani RoySmit ShahNithish PaiJaidam RamtejPrajit Prashant NadkarnJyotirmoy BanerjeePawan GoyalSurender Kumar
2020-05-29
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
Mao YeChengyue GongQiang Liu
2020-05-29
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
Nikolay ArefyevBoris SheludkoAlexander PodolskiyAlexander Panchenko
2020-05-29
Stance Prediction for Contemporary Issues: Data and Experiments
| Marjan HosseiniaEduard DragutArjun Mukherjee
2020-05-29
On Incorporating Structural Information to improve Dialogue Response Generation
| Nikita MoghePriyesh VijayanBalaraman RavindranMitesh M. Khapra
2020-05-28
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna KuncoroLingpeng KongDaniel FriedDani YogatamaLaura RimellChris DyerPhil Blunsom
2020-05-27
CausaLM: Causal Model Explanation Through Counterfactual Language Models
Amir FederNadav OvedUri ShalitRoi Reichart
2020-05-27
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fernández-GonzálezCarlos Gómez-Rodríguez
2020-05-27
Language Representation Models for Fine-Grained Sentiment Classification
Brian CheangBailey WeiDavid KoganHowey QiuMasud Ahmed
2020-05-27
Network Fusion for Content Creation with Conditional INNs
Robin RombachPatrick EsserBjörn Ommer
2020-05-27
A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction
Saadullah AminKatherine Ann DunfieldAnna VechkaevaGünter Neumann
2020-05-26
What Are People Asking About COVID-19? A Question Classification Dataset
| Jerry WeiChengyu HuangSoroush VosoughiJason Wei
2020-05-26
ParsBERT: Transformer-based Model for Persian Language Understanding
| Mehrdad FarahaniMohammad GharachorlooMarzieh FarahaniMohammad Manthouri
2020-05-26
BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection
| Jihyung MoonWon Ik ChoJunbum Lee
2020-05-26
Comparing BERT against traditional machine learning text classification
Santiago González-CarvajalEduardo C. Garrido-Merchán
2020-05-26
BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Zachariah ZhangJingshu LiuNarges Razavian
2020-05-26
An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
Chia-Chih KuoShang-Bao LuoKuan-Yu Chen
2020-05-25
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
| Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-05-25
Pointwise Paraphrase Appraisal is Potentially Problematic
Hannah ChenYangfeng JiDavid Evans
2020-05-25
Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
| Chen LiuSu ZhuZijian ZhaoRuisheng CaoLu ChenKai Yu
2020-05-24
Comparative Study of Machine Learning Models and BERT on SQuAD
Devshree PatelParam RavalRatnam ParikhYesha Shastri
2020-05-22
L2R2: Leveraging Ranking for Abductive Reasoning
| Yunchang ZhuLiang PangYanyan LanXueqi Cheng
2020-05-22
Living Machines: A study of atypical animacy
Mariona Coll ArdanuyFederico NanniKaspar BeelenKasra HosseiniRuth AhnertJon LawrenceKatherine McDonoughGiorgia TolfoDaniel CS WilsonBarbara McGillivray
2020-05-22
Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
Mengxi WeiYifan HeQiong Zhang
2020-05-22
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
Laila RasmyYang XiangZiqian XieCui TaoDegui Zhi
2020-05-22
BERTweet: A pre-trained language model for English Tweets
| Dat Quoc NguyenThanh VuAnh Tuan Nguyen
2020-05-20
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
| Dehong GaoLinbo JinBen ChenMinghui QiuPeng LiYi WeiYi HuHao Wang
2020-05-20
Cross-lingual Transfer Learning for Dialogue Act Recognition
Jiří MartínekChristophe CerisaraPavel KrálLadislav Lenc
2020-05-19
Table Search Using a Deep Contextualized Language Model
| Zhiyu ChenMohamed TrabelsiJeff HeflinYinan XuBrian D. Davison
2020-05-19
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
Hangyu LinYanwei FuYu-Gang JiangXiangyang Xue
2020-05-19
Are All Languages Created Equal in Multilingual BERT?
Shijie WuMark Dredze
2020-05-18
Context-Based Quotation Recommendation
Ansel MacLaughlinTao ChenBurcu Karagol AyanDan Roth
2020-05-17
Support-BERT: Predicting Quality of Question-Answer Pairs in MSDN using Deep Bidirectional Transformer
Bhaskar SenNikhil GopalXinwei Xue
2020-05-17
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben EyalMichael Elhadad
2020-05-17
Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce
| Juntao LiChang LiuJian WangLidong BingHongsong LiXiaozhong LiuDongyan ZhaoRui Yan
2020-05-17
Adversarial Training for Commonsense Inference
Lis PereiraXiaodong LiuFei ChengMasayuki AsaharaIchiro Kobayashi
2020-05-17
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
| Pengcheng YinGraham NeubigWen-tau YihSebastian Riedel
2020-05-17
CERT: Contrastive Self-supervised Learning for Language Understanding
Hongchao FangSicheng WangMeng ZhouJiayuan DingPengtao Xie
2020-05-16
Leveraging Affective Bidirectional Transformers for Offensive Language Detection
AbdelRahim ElmadanyChiyu ZhangMuhammad Abdul-MageedAzadeh Hashemi
2020-05-16
Spelling Error Correction with Soft-Masked BERT
| Shaohua ZhangHaoran HuangJicong LiuHang Li
2020-05-15
Neural Entity Linking on Technical Service Tickets
Nadja KurzFelix HamannAdrian Ulges
2020-05-15
Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline
David HelbigEnrica TroianoRoman Klinger
2020-05-15
[email protected] at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT
Saja Khaled TawalbehMahmoud HammadMohammad AL-Smadi
2020-05-15
NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor
Steve Durairaj SwamyShubham LaddhaBasil AbdussalamDebayan DattaAnupam Jamatia
2020-05-14
A pre-training technique to localize medical BERT and enhance BioBERT
| Shoya WadaToshihiro TakedaShiro ManabeShozo KonishiJun KamoharaYasushi Matsumura
2020-05-14
Parallel Corpus Filtering via Pre-trained Language Models
Boliang ZhangAjay NageshKevin Knight
2020-05-13
Entity-Enriched Neural Models for Clinical Question Answering
| Bhanu Pratap Singh RawatWei-Hung WengPreethi RaghavanPeter Szolovits
2020-05-13
On the Robustness of Language Encoders against Grammatical Errors
Fan YinQuanyu LongTao MengKai-Wei Chang
2020-05-12
Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Amy BredenLee Moore
2020-05-11
How Context Affects Language Models' Factual Predictions
Fabio PetroniPatrick LewisAleksandra PiktusTim RocktäschelYuxiang WuAlexander H. MillerSebastian Riedel
2020-05-10
Transformer Based Language Models for Similar Text Retrieval and Ranking
Javed Qadrud-DinAshraf Bah RabiouRyan WalkerRavi SoniMartin GajekGabriel PackAkhil Rangaraj
2020-05-10
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A. ChiJohn HewittChristopher D. Manning
2020-05-09
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
| Samson TanShafiq JotyMin-Yen KanRichard Socher
2020-05-09
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation
Gustavo AguilarSudipta KarThamar Solorio
2020-05-09
schuBERT: Optimizing Elements of BERT
Ashish KhetanZohar Karnin
2020-05-09
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
| Da YinTao MengKai-Wei Chang
2020-05-08
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
Xing WuYibing LiuXiangyang ZhouDianhai Yu
2020-05-08
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
Ali Hadi ZadehAndreas Moshovos
2020-05-08
Temporal Common Sense Acquisition with Minimal Supervision
Ben ZhouQiang NingDaniel KhashabiDan Roth
2020-05-08
Comparative Analysis of Text Classification Approaches in Electronic Health Records
Aurelie MascioZeljko KraljevicDaniel BeanRichard DobsonRobert StewartRebecca BendayanAngus Roberts
2020-05-08
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
| Marco Tulio RibeiroTongshuang WuCarlos GuestrinSameer Singh
2020-05-08
LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification
Erfan GhaderyMarie-Francine Moens
2020-05-07
Harvesting and Refining Question-Answer Pairs for Unsupervised QA
| Zhongli LiWenhui WangLi DongFuru WeiKe Xu
2020-05-06
An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Yifan PengQingyu ChenZhiyong Lu
2020-05-06
Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics
Guy Emerson
2020-05-06
Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality
Lachlan McPheatMehrnoosh SadrzadehHadi WazniGijs Wijnholds
2020-05-06
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
| Mandy GuoYinfei YangDaniel CerQinlan ShenNoah Constant
2020-05-05
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan KennedyXisen JinAida Mostafazadeh DavaniMorteza DehghaniXiang Ren
2020-05-05
Establishing Baselines for Text Classification in Low-Resource Languages
| Jan Christian Blaise CruzCharibeth Cheng
2020-05-05
ExpBERT: Representation Engineering with Natural Language Explanations
| Shikhar MurtyPang Wei KohPercy Liang
2020-05-05
ImpactCite: An XLNet-based method for Citation Impact Analysis
Dominique MercierSyed Tahseen Raza RizviVikas RajashekarAndreas DengelSheraz Ahmed
2020-05-05
Robust Encodings: A Framework for Combating Adversarial Typos
Erik JonesRobin JiaAditi RaghunathanPercy Liang
2020-05-04
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering
| Vikas YadavSteven BethardMihai Surdeanu
2020-05-04
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
Josef KlafkaAllyson Ettinger
2020-05-04
Code and Named Entity Recognition in StackOverflow
| Jeniya TabassumMounica MaddelaWei XuAlan Ritter
2020-05-04
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
| Masahiro KanekoMasato MitaShun KiyonoJun SuzukiKentaro Inui
2020-05-03
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora KassnerHinrich Schütze
2020-05-02
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
| Qingqing CaoHarsh TrivediAruna BalasubramanianNiranjan Balasubramanian
2020-05-02
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
Bill Yuchen LinSeyeon LeeRahul KhannaXiang Ren
2020-05-02
Generating Derivational Morphology with BERT
Valentin HofmannJanet B. PierrehumbertHinrich Schütze
2020-05-02
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
| Wenxuan ZhouBill Yuchen LinXiang Ren
2020-05-02
Contrastive Self-Supervised Learning for Commonsense Reasoning
| Tassilo KleinMoin Nabi
2020-05-02
HipoRank: Incorporating Hierarchical and Positional Information into Graph-based Unsupervised Long Document Extractive Summarization
Yue DongAndrei RomascanuJackie C. K. Cheung
2020-05-01
Identifying Necessary Elements for BERT's Multilinguality
| Philipp DufterHinrich Schütze
2020-05-01
Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing
Manikandan RavikiranAmin Ekant MuljibhaiToshinori MiyoshiHiroaki OzakiYuta KoreedaSakata Masayuki
2020-05-01
Cross-Linguistic Syntactic Evaluation of Word Prediction Models
| Aaron MuellerGarrett NicolaiPanayiota Petrou-ZeniouNatalia TalminaTal Linzen
2020-05-01
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-05-01
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Xiang YueBernal Jimenez GutierrezHuan Sun
2020-05-01
When BERT Plays the Lottery, All Tickets Are Winning
Sai PrasannaAnna RogersAnna Rumshisky
2020-05-01
POINTER: Constrained Text Generation via Insertion-based Generative Pre-training
| Yizhe ZhangGuoyin WangChunyuan LiZhe GanChris BrockettBill Dolan
2020-05-01
Probing Text Models for Common Ground with Visual Representations
Gabriel IlharcoRowan ZellersAli FarhadiHannaneh Hajishirzi
2020-05-01
Text Categorization for Conflict Event Annotation
Fredrik OlssonMagnus SahlgrenFehmi ben AbdesslemAriel EkgrenKristine Eck
2020-05-01
TF-IDF Character N-grams versus Word Embedding-based Models for Fine-grained Event Classification: A Preliminary Study
Jakub PiskorskiGuillaume Jacquet
2020-05-01
TermEval 2020: TALN-LS2N System for Automatic Term Extraction
Amir HazemBouhM{\'e}rieme iFlorian BoudinBeatrice Daille
2020-05-01
FrameNet Annotations Alignment using Attention-based Machine Translation
Gabriel Marzinotto
2020-05-01
Implementation of Supervised Training Approaches for Monolingual Word Sense Alignment: ACDH-CH System Description for the MWSA Shared Task at GlobaLex 2020
Lenka BajceticSeung-bin Yim
2020-05-01
Transfer learning applied to text classification in Spanish radiological reports
Pilar L{\'o}pez {\'U}bedaManuel Carlos D{\'\i}az-GalianoL. Alfonso Urena LopezMaite MartinTeodoro Mart{\'\i}n-NoguerolAntonio Luna
2020-05-01
Aggression Identification in Social Media: a Transfer Learning Based Approach
RamiFaneva risoaJosiane Mothe
2020-05-01
IRIT at TRAC 2020
RamiFaneva risoaJosiane Mothe
2020-05-01
Bagging BERT Models for Robust Aggression Identification
Julian RischRalf Krestel
2020-05-01
Scmhl5 at TRAC-2 Shared Task on Aggression Identification: Bert Based Ensemble Learning Approach
Han LiuPete BurnapWafa AlorainyMatthew Williams
2020-05-01
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM
| Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-05-01
Aggression and Misogyny Detection using BERT: A Multi-Task Approach
| Niloofar Safi SamghabadiParth PatwaSrinivas PYKLPrerana MukherjeeAmitava DasThamar Solorio
2020-05-01
From Web Crawl to Clean Register-Annotated Corpora
Veronika LaippalaSamuel R{\"o}nnqvistSaara Hellstr{\"o}mJuhani LuotolahtiLiina RepoAnna SalmelaValtteri SkantsiSampo Pyysalo
2020-05-01
Cross-lingual Zero Pronoun Resolution
Abdulrahman AlorainiMassimo Poesio
2020-05-01
Understanding User Utterances in a Dialog System for Caregiving
Yoshihiko AsaoJulien KloetzerJunta MizunoDai SaikiKazuma KadowakiKentaro Torisawa
2020-05-01
Joint Learning of Syntactic Features Helps Discourse Segmentation
Takshak DesaiParag Pravin DakleDan Moldovan
2020-05-01
Adapting BERT to Implicit Discourse Relation Classification with a Focus on Discourse Connectives
Yudai KishimotoYugo MurawakiSadao Kurohashi
2020-05-01
Automated Essay Scoring System for Nonnative Japanese Learners
Reo HiraoMio AraiHiroki ShimanakaSatoru KatsumataMamoru Komachi
2020-05-01
Development and Validation of a Corpus for Machine Humor Comprehension
Yuen-Hsien TsengWun-Syuan WuChia-Yueh ChangHsueh-Chih ChenWei-Lun Hsu
2020-05-01
Abusive language in Spanish children and young teenager's conversations: data preparation and short text classification with contextual word embeddings
Marta R. Costa-juss{\`a}Esther Gonz{\'a}lezAsuncion MorenoEudald Cumalat
2020-05-01
An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly Papers
Kenichi IwatsukiFlorian BoudinAkiko Aizawa
2020-05-01
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
Jiahao ChenChenjie CaoXiuyan Jiang
2020-05-01
Adaptation of Deep Bidirectional Transformers for Afrikaans Language
Sello Ralethe
2020-05-01
Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yor\`ub\'a and Twi
Jesujoba AlabiKwabena Amponsah-KaakyireDavid AdelaniCristina Espa{\~n}a-Bonet
2020-05-01
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque
Maddalen L{\'o}pez de LacalleXabier SaralegiI{\~n}aki San Vicente
2020-05-01
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts
Oanh TranTu PhamVu DangBang Nguyen
2020-05-01
DaNE: A Named Entity Resource for Danish
Rasmus HvingelbyAmalie Brogaard PauliMaria BarrettChristina RostedLasse Malm LidegaardAnders S{\o}gaard
2020-05-01
Is Language Modeling Enough? Evaluating Effective Embedding Combinations
Rudolf SchneiderTom OberhauserPaul GrundmannFelix Alex GerserAlex LoesererSteffen Staab
2020-05-01
Parsing as Tagging
Robert VacareanuGeorge Caique Gouveia BarbosaMarco A. Valenzuela-Esc{\'a}rcegaMihai Surdeanu
2020-05-01
AIA-BDE: A Corpus of FAQs in Portuguese and their Variations
Hugo Gon{\c{c}}alo OliveiraJo{\~a}o FerreiraJos{\'e} SantosPedro FialhoRicardo RodriguesLuisa CoheurAna Alves
2020-05-01
Cross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora
Delphine CharletGeraldine DamnatiFrederic Bechetgabriel marzinottoJohannes Heinecke
2020-05-01
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task
| Md Tahmid Rahman LaskarJimmy Xiangji HuangEnamul Hoque
2020-05-01
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
Prafulla Kumar ChoubeyRuihong Huang
2020-05-01
A Summarization Dataset of Slovak News Articles
| Marek SuppaJergus Adamec
2020-05-01
KLEJ: Comprehensive Benchmark for Polish Language Understanding
| Piotr RybakRobert MroczkowskiJanusz TraczIreneusz Gawlik
2020-05-01
Analyzing ELMo and DistilBERT on Socio-political News Classification
Berfu B{\"u}y{\"u}k{\"o}zAli H{\"u}rriyeto{\u{g}}luArzucan {\"O}zg{\"u}r
2020-05-01
SciREX: A Challenge Dataset for Document-Level Information Extraction
| Sarthak JainMadeleine van ZuylenHannaneh HajishirziIz Beltagy
2020-05-01
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Xinyun ChenChen LiangAdams Wei YuDenny ZhouDawn SongQuoc V. Le
2020-05-01
Improving Neural Language Generation with Spectrum Control
Lingxiao WangJing HuangKevin HuangZiniu HuGuangtao WangQuanquan Gu
2020-05-01
GigaBERT: A Bilingual BERT for English and Arabic
| Wuwei LanYang ChenWei XuAlan Ritter
2020-04-30
WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context
Anna BreitArtem RevenkoKiamehr RezaeeMohammad Taher PilehvarJose Camacho-Collados
2020-04-30
On the Evaluation of Contextual Embeddings for Zero-Shot Cross-Lingual Transfer Learning
Phillip KeungYichao LuJulian SalazarVikas Bhardwaj
2020-04-30
A Matter of Framing: The Impact of Linguistic Formalism on Probing Results
Ilia KuznetsovIryna Gurevych
2020-04-30
SegaBERT: Pre-training of Segment-aware BERT for Language Understanding
He BaiPeng ShiJimmy LinLuchen TanKun XiongWen GaoMing Li
2020-04-30
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
| Nicola De CaoMichael SchlichtkrullWilker AzizIvan Titov
2020-04-30
Investigating Transferability in Pretrained Language Models
Alex TamkinTrisha SinghDavide GiovanardiNoah Goodman
2020-04-30
Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection
Momchil HardalovIvan KoychevPreslav Nakov
2020-04-30
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong WuYun ChenBen KaoQun Liu
2020-04-30
Robust Question Answering Through Sub-part Alignment
Jifan ChenGreg Durrett
2020-04-30
Modular Representation Underlies Systematic Generalization in Neural Natural Language Inference Models
Atticus GeigerKyle RichardsonChristopher Potts
2020-04-30
Universal Dependencies according to BERT: both more specific and more general
| Tomasz LimisiewiczRudolf RosaDavid Mareček
2020-04-30
Look at the First Sentence: Position Bias in Question Answering
Miyoung KoJinhyuk LeeHyunjae KimGangwoo KimJaewoo Kang
2020-04-30
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing
Hayley RossJonathan CaiBonan Min
2020-04-30
Interpretable Entity Representations through Large-Scale Typing
Yasumasa OnoeGreg Durrett
2020-04-30
MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer
Jonas PfeifferIvan VulićIryna GurevychSebastian Ruder
2020-04-30
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Weijia XuBatool HaiderSaab Mansour
2020-04-29
Detecting Perceived Emotions in Hurricane Disasters
Shrey DesaiCornelia CarageaJunyi Jessy Li
2020-04-29
Training Curricula for Open Domain Answer Re-Ranking
| Sean MacAvaneyFranco Maria NardiniRaffaele PeregoNicola TonellottoNazli GoharianOphir Frieder
2020-04-29
Analysing Lexical Semantic Change with Contextualised Word Representations
Mario GiulianelliMarco Del TrediciRaquel Fernández
2020-04-29
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur KulmizevVinit RavishankarMostafa AbdouJoakim Nivre
2020-04-29
BURT: BERT-inspired Universal Representation from Twin Structure
Yian LiHai Zhao
2020-04-29
Bilingual Text Extraction as Reading Comprehension
Katsuki ChousaMasaaki NagataMasaaki Nishino
2020-04-29
What Happens To BERT Embeddings During Fine-tuning?
Amil MerchantElahe RahimtoroghiEllie PavlickIan Tenney
2020-04-29
Distantly-Supervised Neural Relation Extraction with Side Information using BERT
| Johny MoreiraChaina OliveiraDavid MacêdoCleber ZanchettinLuciano Barbosa
2020-04-29
Revisiting Pre-Trained Models for Chinese Natural Language Processing
| Yiming CuiWanxiang CheTing LiuBing QinShijin WangGuoping Hu
2020-04-29
A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
Masaaki NagataChousa KatsukiMasaaki Nishino
2020-04-29
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Julian MichaelJan A. BothaIan Tenney
2020-04-29
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
| John X. MorrisEli LiflandJin Yong YooJake GrigsbyDi JinYanjun Qi
2020-04-29
Extending Multilingual BERT to Low-Resource Languages
Zihan WangKarthikeyan KStephen MayhewDan Roth
2020-04-28
Joint Keyphrase Chunking and Salience Ranking with BERT
| Si SunChenyan XiongZhenghao LiuZhiyuan LiuJie Bao
2020-04-28
EARL: Speedup Transformer-based Rankers with Pre-computed Representation
Luyu GaoZhuyun DaiJamie Callan
2020-04-28
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Yue WangShafiq JotyMichael R. LyuIrwin KingCaiming XiongSteven C. H. Hoi
2020-04-28
DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
Hu XuBing LiuLei ShuPhilip S. Yu
2020-04-28
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection
| Wenliang DaiTiezheng YuZihan LiuPascale Fung
2020-04-28
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
| Ji XinRaphael TangJaejun LeeYaoliang YuJimmy Lin
2020-04-27
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Omar KhattabMatei Zaharia
2020-04-27
ColBERT: Using BERT Sentence Embedding for Humor Detection
| Issa Annamoradnejad
2020-04-27
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
| Xin LiuJiefu OuYangqiu SongXin Jiang
2020-04-27
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao SongHao SunXu TanTao QinJianfeng LuHongzhi LiuTie-Yan Liu
2020-04-27
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie ZhaoTao LinMartin JaggiHinrich Schütze
2020-04-26
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching
Liu YangMingyang ZhangCheng LiMichael BenderskyMarc Najork
2020-04-26
Classification of Cuisines from Sequentially Structured Recipes
Tript SharmaUtkarsh UpadhyayGanesh Bagler
2020-04-26
Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
Xinyue ZhengPeng WangQigang WangZhongchao Shi
2020-04-26
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
| Xingyi ChengWeidi XuKunlong ChenShaohua JiangFeng WangTaifeng WangWei ChuYuan Qi
2020-04-26
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Mengjie ZhaoPhilipp DufterYadollah YaghoobzadehHinrich Schütze
2020-04-25
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi LiaoXin JiangQun Liu
2020-04-24
Syntactic Data Augmentation Increases Robustness to Inference Heuristics
Junghyun MinR. Thomas McCoyDipanjan DasEmily PitlerTal Linzen
2020-04-24
The Inception Team at NSURL-2019 Task 8: Semantic Question Similarity in Arabic
Hana Al-TheiabatAisha Al-Sadi
2020-04-24
Cross-lingual Information Retrieval with BERT
Zhuolin JiangAmro El-JaroudiWilliam HartmannDamianos KarakosLingjun Zhao
2020-04-24
A Tailored Pre-Training Model for Task-Oriented Dialog Generation
Jing GuQingyang WuChongruo WuWeiyan ShiZhou Yu
2020-04-24
Data Annealing for Informal Language Understanding Tasks
Jing GuZhou Yu
2020-04-24
Contextualized Representations Using Textual Encyclopedic Knowledge
Mandar JoshiKenton LeeYi LuanKristina Toutanova
2020-04-24
Collecting Entailment Data for Pretraining: New Protocols and Negative Results
| Samuel R. BowmanJennimaria PalomakiLivio Baldini SoaresEmily Pitler
2020-04-24
On Adversarial Examples for Biomedical NLP Tasks
Vladimir AraujoAndres CarvalloCarlos AspillagaDenis Parra
2020-04-23
Same Side Stance Classification Task: Facilitating Argument Stance Classification by Fine-tuning a BERT Model
Stefan OllingerLorik DumaniPremtim SahitajRalph BergmannRalf Schenkel
2020-04-23
Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Yaru HaoLi DongFuru WeiKe Xu
2020-04-23
UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection
Gregor WiedemannSeid Muhie YimamChris Biemann
2020-04-23
Keyphrase Prediction With Pre-trained Language Model
Rui LiuZheng LinWeiping Wang
2020-04-22
Learning to Classify Intents and Slot Labels Given a Handful of Examples
Jason KroneYi ZhangMona Diab
2020-04-22
Residual Energy-Based Models for Text Generation
Yuntian DengAnton BakhtinMyle OttArthur SzlamMarc'Aurelio Ranzato
2020-04-22
Attention Module is Not Only a Weight: Analyzing Transformers with Vector Norms
Goro KobayashiTatsuki KuribayashiSho YokoiKentaro Inui
2020-04-21
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Linyang LiRuotian MaQipeng GuoXiangyang XueXipeng Qiu
2020-04-21
DIET: Lightweight Language Understanding for Dialogue Systems
| Tanja BunkDaksh VarshneyaVladimir VlasovAlan Nichol
2020-04-21
Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media
Payam KarisaniJoyce C. HoEugene Agichtein
2020-04-21
Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasets
Jinghui LuBrian MacNamee
2020-04-21
MPNet: Masked and Permuted Pre-training for Language Understanding
| Kaitao SongXu TanTao QinJianfeng LuTie-Yan Liu
2020-04-20
A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
Chi-Liang LiuTsung-Yuan HsuYung-Sung ChuangHung-Yi Lee
2020-04-20
CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
| Akshay SmitSaahil JainPranav RajpurkarAnuj PareekAndrew Y. NgMatthew P. Lungren
2020-04-20
Adversarial Training for Large Neural Language Models
| Xiaodong LiuHao ChengPengcheng HeWeizhu ChenYu WangHoifung PoonJianfeng Gao
2020-04-20
StereoSet: Measuring stereotypical bias in pretrained language models
| Moin NadeemAnna BethkeSiva Reddy
2020-04-20
Enhancing Pharmacovigilance with Drug Reviews and Social Media
| Brent BisedaKatie Mo
2020-04-18
Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness
Yavuz Selim KartalBusra GuvenenMucahid Kutlu
2020-04-17
Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning
| Joongbo ShinYoonhyung LeeSeunghyun YoonKyomin Jung
2020-04-17
Learning-to-Rank with BERT in TF-Ranking
Shuguang HanXuanhui WangMike BenderskyMarc Najork
2020-04-17
The Right Tool for the Job: Matching Model and Instance Complexities
| Roy SchwartzGabriel StanovskySwabha SwayamdiptaJesse DodgeNoah A. Smith
2020-04-16
SPECTER: Document-level Representation Learning using Citation-informed Transformers
| Arman CohanSergey FeldmanIz BeltagyDoug DowneyDaniel S. Weld
2020-04-15
lamBERT: Language and Action Learning Using Multimodal BERT
Kazuki MiyazawaTatsuya AokiTakato HoriiTakayuki Nagai
2020-04-15
ToD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogues
| Chien-Sheng WuSteven HoiRichard SocherCaiming Xiong
2020-04-15
Coreferential Reasoning Learning for Language Representation
| Deming YeYankai LinJiaju DuZhenghao LiuMaosong SunZhiyuan Liu
2020-04-15
Training with Quantization Noise for Extreme Model Compression
| Angela FanPierre StockBenjamin GrahamEdouard GraveRemi GribonvalHerve JegouArmand Joulin
2020-04-15
Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models
Siqi Liu
2020-04-15
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models
| Wietse de VriesAndreas van CranenburghMalvina Nissim
2020-04-14
Deep Learning Models for Multilingual Hate Speech Detection
| Sai Saketh AluruBinny MathewPunyajoy SahaAnimesh Mukherjee
2020-04-14
Standardizing and Benchmarking Crisis-related Social Media Datasets for Humanitarian Information Processing
Firoj AlamHassan SajjadMuhammad ImranFerda Ofli
2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
Dirk GroeneveldTushar KhotMausamAshish Sabharwal
2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
Bin BiChenliang LiChen WuMing YanWei Wang
2020-04-14
Pretrained Transformers Improve Out-of-Distribution Robustness
Dan HendrycksXiaoyuan LiuEric WallaceAdam DziedzicRishabh KrishnanDawn Song
2020-04-13
Unified Multi-Criteria Chinese Word Segmentation with BERT
Zhen KeLiang ShiErli MengBin WangXipeng QiuXuanjing Huang
2020-04-13
ProFormer: Towards On-Device LSH Projection Based Transformers
Chinnadhurai SankarSujith RaviZornitsa Kozareva
2020-04-13
Cascade Neural Ensemble for Identifying Scientifically Sound Articles
Ashwin Karthik AmbalavananMurthy Devarakonda
2020-04-13
Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction
| Hong GuanJianfu LiHua XuMurthy Devarakonda
2020-04-13
Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification
Ming JiangJennifer D'SouzaSören AuerJ. Stephen Downie
2020-04-13
VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
| Zhibin LuPan DuJian-Yun Nie
2020-04-12
Pre-training Text Representations as Meta Learning
Shangwen LvYuechen WangDaya GuoDuyu TangNan DuanFuqing ZhuMing GongLinjun ShouRyan MaDaxin JiangGuihong CaoMing ZhouSonglin Hu
2020-04-12
AMR Parsing via Graph-Sequence Iterative Inference
Deng CaiWai Lam
2020-04-12
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma RoyNoah ConstantRami Al-RfouAditya BaruaAaron PhillipsYinfei Yang
2020-04-11
End to End Chinese Lexical Fusion Recognition with Sememe Knowledge
Yijiang LiuMeishan ZhangDonghong Ji
2020-04-11
SimpleTran: Transferring Pre-Trained Sentence Embeddings for Low Resource Text Classification
Siddhant GargRohit Kumar SharmaYingyu Liang
2020-04-10
An In-depth Walkthrough on Evolution of Neural Machine Translation
Rohan JagtapDr. Sudhir N. Dhage
2020-04-10
Telling BERT's full story: from Local Attention to Global Aggregation
Damian PascualGino BrunnerRoger Wattenhofer
2020-04-10
BLEURT: Learning Robust Metrics for Text Generation
Thibault SellamDipanjan DasAnkur P. Parikh
2020-04-09
On the Language Neutrality of Pre-trained Multilingual Representations
Jindřich LibovickýRudolf RosaAlexander Fraser
2020-04-09
Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve
Oshin AgarwalYinfei YangByron C. WallaceAni Nenkova
2020-04-09
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
Yihuan MaoYujing WangChufan WuChen ZhangYang WangYaming YangQuanlu ZhangYunhai TongJing Bai
2020-04-08
DynaBERT: Dynamic BERT with Adaptive Width and Depth
Lu HouLifeng ShangXin JiangQun Liu
2020-04-08
Exploiting Redundancy in Pre-trained Language Models for Efficient Transfer Learning
Fahim DalviHassan SajjadNadir DurraniYonatan Belinkov
2020-04-08
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence
| Federico BianchiSilvia TerragniDirk Hovy
2020-04-08
Poor Man's BERT: Smaller and Faster Transformer Models
| Hassan SajjadFahim DalviNadir DurraniPreslav Nakov
2020-04-08
Improving BERT with Self-Supervised Attention
Xiaoyu KouYaming YangYujing WangCe ZhangYiren ChenYunhai TongYan ZhangJing Bai
2020-04-08
DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
Tianda LiJia-Chen GuXiaodan ZhuQuan LiuZhen-Hua LingZhiming SuSi Wei
2020-04-08
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events
Miguel BallesterosRishita AnubhaiShuai WangNima PourdamghaniYogarshi VyasJie MaParminder BhatiaKathleen McKeownYaser Al-Onaizan
2020-04-08
Error-correction and extraction in request dialogs
Stefan ConstantinAlex Waibel
2020-04-08
SciWING -- A Software Toolkit for Scientific Document Processing
| Abhinav Ramesh KashyapMin-Yen Kan
2020-04-08
Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
| Changmao LiJinho D. Choi
2020-04-07
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
Bowen WuHuan ZhangMengyuan LiZongsheng WangQihang FengJunhong HuangBaoxun Wang
2020-04-07
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Paloma JereticAlex WarstadtSuvrat BhooshanAdina Williams
2020-04-07
Information-Theoretic Probing for Linguistic Structure
| Tiago PimentelJosef ValvodaRowan Hall MaudslayRan ZmigrodAdina WilliamsRyan Cotterell
2020-04-07
Towards Evaluating the Robustness of Chinese BERT Classifiers
Boxin WangBoyuan PanXin LiBo Li
2020-04-07
The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews
| Elena TutubalinaIlseyar AlimovaZulfat MiftahutdinovAndrey SakhovskiyValentin MalykhSergey Nikolenko
2020-04-07
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
| Jia-Chen GuTianda LiQuan LiuZhen-Hua LingZhiming SuSi WeiXiaodan Zhu
2020-04-07
TextGAIL: Generative Adversarial Imitation Learning for Text Generation
Qingyang WuLei LiZhou Yu
2020-04-07
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
DongHyun ChoiMyeong Cheol ShinEungGyun KimDong Ryeol Shin
2020-04-07
Leveraging the Inherent Hierarchy of Vacancy Titles for Automated Job Ontology Expansion
Jeroen Van HautteVincent SchelstraeteMikaël Wornoo
2020-04-06
Enhancing Review Comprehension with Domain-Specific Commonsense
Aaron TraylorChen ChenBehzad GolshanXiaolan WangYuliang LiYoshihiko SuharaJinfeng LiCagatay DemiralpWang-Chiew Tan
2020-04-06
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Zhiqing SunHongkun YuXiaodan SongRenjie LiuYiming YangDenny Zhou
2020-04-06
Bootstrapping a Crosslingual Semantic Parser
Tom SherborneYumo XuMirella Lapata
2020-04-06
FastBERT: a Self-distilling BERT with Adaptive Inference Time
| Weijie LiuPeng ZhouZhe ZhaoZhiruo WangHaotang DengQi Ju
2020-04-05
Improved Pretraining for Domain-specific Contextual Embedding Models
Subendhu RongaliAbhyuday JagannathaBhanu Pratap Singh RawatHong Yu
2020-04-05
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
| Chunyuan LiXiang GaoYuan LiXiujun LiBaolin PengYizhe ZhangJianfeng Gao
2020-04-05
A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis
| Yunlong LiangFandong MengJinchao ZhangJinan XuYufeng ChenJie Zhou
2020-04-04
CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
Congying XiaChenwei ZhangHoang NguyenJiawei ZhangPhilip Yu
2020-04-04
Finding Black Cat in a Coal Cellar -- Keyphrase Extraction & Keyphrase-Rubric Relationship Classification from Complex Assignments
| Manikandan Ravikiran
2020-04-03
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
| Yaobo LiangNan DuanYeyun GongNing WuFenfei GuoWeizhen QiMing GongLinjun ShouDaxin JiangGuihong CaoXiaodong FanRuofei ZhangRahul AgrawalEdward CuiSining WeiTaroon BhartiYing QiaoJiun-Hung ChenWinnie WuShuguang LiuFan YangDaniel CamposRangan MajumderMing Zhou
2020-04-03
Testing pre-trained Transformer models for Lithuanian news clustering
Lukas StankevičiusMantas Lukoševičius
2020-04-03
Gestalt: a Stacking Ensemble for SQuAD2.0
Mohamed El-Geish
2020-04-02
Deep Entity Matching with Pre-Trained Language Models
Yuliang LiJinfeng LiYoshihiko SuharaAnHai DoanWang-Chiew Tan
2020-04-01
Towards Productionizing Subjective Search Systems
Aaron FengShuwei ChenYuliang LiHiroshi MatsudaHidekazu TamakiWang-Chiew Tan
2020-03-31
Unification-based Reconstruction of Explanations for Science Questions
| Marco ValentinoMokanarangan ThayaparanAndré Freitas
2020-03-31
Give your Text Representation Models some Love: the Case for Basque
Rodrigo AgerriIñaki San VicenteJon Ander CamposAnder BarrenaXabier SaralegiAitor SoroaEneko Agirre
2020-03-31
InterBERT: An Effective Multi-Modal Pretraining Approach via Vision-and-Language Interaction
Junyang LinAn YangYichang ZhangJie LiuJingren ZhouHongxia Yang
2020-03-30
NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain
Ayush JainDr. N. M. MeenachiDr. B. Venkatraman
2020-03-30
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
Alireza MohammadshahiJames Henderson
2020-03-29
Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling
| Dmitrii AksenovJulián Moreno-SchneiderPeter BourgonjeRobert SchwarzenbergLeonhard HennigGeorg Rehm
2020-03-29
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining
Chengyu WangMinghui QiuJun HuangXiaofeng He
2020-03-29
User Generated Data: Achilles' Heel of BERT
Ankit KumarPiyush MakhijaAnuj Gupta
2020-03-29
BERT Fine-tuning For Arabic Text Summarization
| Khalid N. ElmadaniMukhtar ElgezouliAnas Showk
2020-03-29
HIN: Hierarchical Inference Network for Document-Level Relation Extraction
Hengzhu TangYanan CaoZhenyu ZhangJiangxia CaoFang FangShi WangPengfei Yin
2020-03-28
Cycle Text-To-Image GAN with BERT
| Trevor TsueSamir SenJason Li
2020-03-26
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
| Kevin ClarkMinh-Thang LuongQuoc V. LeChristopher D. Manning
2020-03-23
Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles
| Malte OstendorffTerry RuasMoritz SchubotzGeorg RehmBela Gipp
2020-03-22
Beheshti-NER: Persian Named Entity Recognition Using BERT
| Ehsan TaherSeyed Abbas HoseiniMehrnoush Shamsfard
2020-03-19
Temporal Embeddings and Transformer Models for Narrative Text Understanding
| Vani KSimone MellaceAlessandro Antonucci
2020-03-19
The value of text for small business default prediction: A deep learning approach
Matthew StevensonChristophe MuesCristián Bravo
2020-03-19
Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections
Yi-An LaiXuan ZhuYi ZhangMona Diab
2020-03-19
X-Stance: A Multilingual Multi-Target Dataset for Stance Detection
| Jannis VamvasRico Sennrich
2020-03-18
Calibration of Pre-trained Transformers
| Shrey DesaiGreg Durrett
2020-03-17
Author2Vec: A Framework for Generating User Embedding
Xiaodong WuWeizhe LinZhilin WangElena Rastorgueva
2020-03-17
PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
| Thomas HaiderSteffen EgerEvgeny KimRoman KlingerWinfried Menninghaus
2020-03-17
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
Zhiheng HuangPeng XuDavis LiangAjay MishraBing Xiang
2020-03-16
Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data
Harish Tayyar MadabushiElena KochkinaMichael Castelle
2020-03-16
A Survey on Contextual Embeddings
| Qi LiuMatt J. KusnerPhil Blunsom
2020-03-16
Finnish Language Modeling with Deep Transformer Models
Abhilash JainAku RuoheStig-Arne GrönroosMikko Kurimo
2020-03-14
Document Ranking with a Pretrained Sequence-to-Sequence Model
Rodrigo NogueiraZhiying JiangJimmy Lin
2020-03-14
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
| Samuel Broscheit
2020-03-11
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
| Haoran ZhangAmy X. LuMohamed AbdallaMatthew McDermottMarzyeh Ghassemi
2020-03-11
Keyword-Attentive Deep Semantic Matching
| Changyu MiaoZhen CaoYik-Cheung Tam
2020-03-11
Efficient Intent Detection with Dual Sentence Encoders
| Iñigo CasanuevaTadas TemčinasDaniela GerzMatthew HendersonIvan Vulić
2020-03-10
Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT
Aitor García-PablosNaiara PerezMontse Cuadros
2020-03-06
Transfer Learning for Information Extraction with Limited Data
Minh-Tien NguyenViet-Anh PhanLe Thai LinhNguyen Hong SonLe Tien DungMiku HiranoHajime Hotta
2020-03-06
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Florian SchmidtThomas Hofmann
2020-03-05
What the [MASK]? Making Sense of Language-Specific BERT Models
Debora NozzaFederico BianchiDirk Hovy
2020-03-05
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
Tianyu LiuXin ZhengBaobao ChangZhifang Sui
2020-03-05
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
| Yada PruksachatkunPhil YeresHaokun LiuJason PhangPhu Mon HtutAlex WangIan TenneySamuel R. Bowman
2020-03-04
Data Augmentation using Pre-trained Transformer Models
| Varun KumarAshutosh ChoudharyEunah Cho
2020-03-04
Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout
Filip GralińskiTomasz StanisławekAnna WróblewskaDawid LipińskiAgnieszka KaliskaPaulina RosalskaBartosz TopolskiPrzemysław Biecek
2020-03-04
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection
Daniele BonadimanAlessandro Moschitti
2020-03-04
CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model
| Liang XuXuanwei ZhangQianqian Dong
2020-03-03
Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion
Jingyuan YangGuang LiuYuzhao MaoZhiwei ZhaoWeiguo GaoXuan LiHaiqin YangJianping Shen
2020-03-03
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing
| Ziqing YangYiming CuiZhipeng ChenWanxiang CheTing LiuShijin WangGuoping Hu
2020-02-28
DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
Yuyu ZhangPing NieXiubo GengArun RamamurthyLe SongDaxin Jiang
2020-02-28
AraBERT: Transformer-based Model for Arabic Language Understanding
| Wissam AntounFady BalyHazem Hajj
2020-02-28
A Primer in BERTology: What we know about how BERT works
Anna RogersOlga KovalevaAnna Rumshisky
2020-02-27
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT
Prakhar GaneshYao ChenXin LouMohammad Ali KhanYin YangDeming ChenMarianne WinslettHassan SajjadPreslav Nakov
2020-02-27
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
Lichao SunKazuma HashimotoWenpeng YinAkari AsaiJia LiPhilip YuCaiming Xiong
2020-02-27
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
| Wenhui WangFuru WeiLi DongHangbo BaoNan YangMing Zhou
2020-02-25
BERT Can See Out of the Box: On the Cross-modal Transferability of Text Representations
Thomas ScialomPatrick BordesPaul-Alexis DrayJacopo StaianoPatrick Gallinari
2020-02-25
Exploring BERT Parameter Efficiency on the Stanford Question Answering Dataset v2.0
Eric Hulburd
2020-02-25
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
| Yige XuXipeng QiuLigao ZhouXuanjing Huang
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
Dianbo LiuTim Miller
2020-02-20
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning
Mitchell A. GordonKevin DuhNicholas Andrews
2020-02-19
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
| Xiaodong LiuYu WangJianshu JiHao ChengXueyun ZhuEmmanuel AwaPengcheng HeWeizhu ChenHoifung PoonGuihong CaoJianfeng Gao
2020-02-19
From English To Foreign Languages: Transferring Pre-trained Language Models
Ke Tran
2020-02-18
Incorporating BERT into Neural Machine Translation
| Jinhua ZhuYingce XiaLijun WuDi HeTao QinWengang ZhouHouqiang LiTie-Yan Liu
2020-02-17
A Financial Service Chatbot based on Deep Bidirectional Transformers
Shi YuYuxin ChenHussain Zaidi
2020-02-17