Search Results for author:
Found 30 papers, 21 papers with code
Towards Adaptive Residual Network Training: A Neural-ODE Perspective
Serving as a crucial factor, the depth of residual networks balances model capacity, performance, and training efficiency.
SoTeacher: A Student-oriented Teacher Network Training Framework for Knowledge Distillation
To fill this gap, we propose a novel student-oriented teacher network training framework SoTeacher, inspired by recent findings that student performance hinges on teacher's capability to approximate the true label distribution of training samples.
P4E: Few-Shot Event Detection as Prompt-Guided Identification and Localization
Our framework decomposes event detection into an identification task and a localization task.
Label Noise in Adversarial Training: A Novel Perspective to Study Robust Overfitting
We show that label noise exists in adversarial training.
Perturbation Deterioration: The Other Side of Catastrophic Overfitting
While this phenomenon is commonly explained as overfitting, we observe that it is a twin process: not only does the model catastrophic overfits to one type of perturbation, but also the perturbation deteriorates into random noise.
Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training
In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels.
Multi-head or Single-head? An Empirical Comparison for Transformer Training
Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications.
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging
Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.
Ranked #1 on
Phrase Tagging
on KPTimes
Data Quality Matters For Adversarial Training: An Empirical Study
Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness.
On the Transformer Growth for Progressive BERT Training
Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity.
Overfitting or Underfitting? Understand Robustness Drop in Adversarial Training
Our goal is to understand why the robustness drops after conducting adversarial training for too long.
Very Deep Transformers for Neural Machine Translation
We explore the application of very deep Transformer models for Neural Machine Translation (NMT).
Ranked #1 on
Machine Translation
on WMT2014 English-French
(using extra training data)
Partially-Typed NER Datasets Integration: Connecting Practice to Theory
While typical named entity recognition (NER) models require the training set to be annotated with all target types, each available datasets may only cover a part of them.
Understanding the Difficulty of Training Transformers
Transformers have proved effective in many NLP tasks.
Ranked #5 on
Machine Translation
on WMT2014 English-French
TBC-Net: A real-time detector for infrared small target detection using semantic constraint
Meanwhile, we propose a joint loss function and a training method.
Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling
Its performance is largely influenced by the annotation quality and quantity in supervised learning scenarios, and obtaining ground truth labels is often costly.
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Therefore, we manually correct these label mistakes and form a cleaner test set.
Ranked #3 on
Named Entity Recognition (NER)
on CoNLL++
(using extra training data)
Facet-Aware Evaluation for Extractive Summarization
In this paper, we present a facet-aware evaluation setup for better assessment of the information coverage in extracted summaries.
Raw-to-End Name Entity Recognition in Social Media
Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries.
On the Variance of the Adaptive Learning Rate and Beyond
The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.
Arabic Named Entity Recognition: What Works and What's Next
This paper presents the winning solution to the Arabic Named Entity Recognition challenge run by Topcoder. com.
Reliability-aware Dynamic Feature Composition for Name Tagging
We design a set of word frequency-based reliability signals to indicate the quality of each word embedding.
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution.
Cross-relation Cross-bag Attention for Distantly-supervised Relation Extraction
Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations.
Learning Named Entity Tagger using Domain-Specific Dictionary
Recent advances in deep neural models allow us to build reliable named entity recognition (NER) systems without handcrafting features.
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Many efforts have been made to facilitate natural language processing tasks with pre-trained language models (LMs), and brought significant improvements to various applications.
Ranked #45 on
Named Entity Recognition (NER)
on CoNLL 2003 (English)
Expert Finding in Heterogeneous Bibliographic Networks with Locally-trained Embeddings
We study the task of expert finding in heterogeneous bibliographical networks based on two aspects: textual content analysis and authority ranking.
Graph Clustering with Dynamic Embedding
Unlike most existing embedding methods that are task-agnostic, we simultaneously solve for the underlying node representations and the optimal clustering assignments in an end-to-end manner.
Social and Information Networks Physics and Society
Empower Sequence Labeling with Task-Aware Neural Language Model
In this study, we develop a novel neural framework to extract abundant knowledge hidden in raw texts to empower the sequence labeling task.
Ranked #13 on
Part-Of-Speech Tagging
on Penn Treebank
Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach
These annotations, referred as heterogeneous supervision, often conflict with each other, which brings a new challenge to the original relation extraction task: how to infer the true label from noisy labels for a given instance.
