Search Results for author:
Found 25 papers, 16 papers with code
Findings of the WMT 2020 Shared Task on Machine Translation Robustness
We report the findings of the second edition of the shared task on improving robustness in Machine Translation (MT).
Curriculum Learning with Adam: The Devil Is in the Wrong Details
We present a number of different case studies with different common hand-crafted and automated CL approaches to illustrate this phenomenon, and we find that none of them outperforms optimisation with only Adam with well-chosen hyperparameters.
Emergent Communication: Generalization and Overfitting in Lewis Games
Lewis signaling games are a class of simple communication games for simulating the emergence of language.
AANG: Automating Auxiliary Learning
Auxiliary objectives, supplementary learning signals that are introduced to help aid learning on data-starved or highly complex end-tasks, are commonplace in machine learning.
Distributionally Robust Models with Parametric Likelihood Ratios
As machine learning models are deployed ever more broadly, it becomes increasingly important that they are not only able to perform well on their training distribution, but also yield accurate predictions when confronted with distribution shift.
Balancing Average and Worst-case Accuracy in Multitask Learning
When training and evaluating machine learning models on a large number of tasks, it is important to not only look at average task accuracy -- which may be biased by easy or redundant tasks -- but also worst-case accuracy (i. e. the performance on the task with the lowest accuracy).
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative
In most settings of practical concern, machine learning practitioners know in advance what end-task they wish to boost with auxiliary tasks.
Learning Neural Models for Natural Language Processing in the Face of Distributional Shift
that the data is sampled from a fixed distribution both at training and test time.
Examining and Combating Spurious Features under Distribution Shift
Group distributionally robust optimization (DRO) provides an effective tool to alleviate covariate shift by minimizing the worst-case training loss over a set of pre-defined groups.
Modeling the Second Player in Distributionally Robust Optimization
Distributionally robust optimization (DRO) provides a framework for training machine learning models that are able to perform well on a collection of related data distributions (the "uncertainty set").
Weight Poisoning Attacks on Pretrained Models
Recently, NLP has seen a surge in the usage of large pre-trained models.
Weight Poisoning Attacks on Pre-trained Models
We show that by applying a regularization method, which we call RIPPLe, and an initialization procedure, which we call Embedding Surgery, such attacks are possible even with limited knowledge of the dataset and fine-tuning procedure.
Optimizing Data Usage via Differentiable Rewards
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems.
Regularizing Trajectories to Mitigate Catastrophic Forgetting
Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective.
Findings of the First Shared Task on Machine Translation Robustness
We share the findings of the first shared task on improving robustness of Machine Translation (MT).
Are Sixteen Heads Really Better than One?
Attention is a powerful and ubiquitous mechanism for allowing neural models to focus on particular salient pieces of information by taking their weighted average when making predictions.
On Meaning-Preserving Adversarial Perturbations for Sequence-to-Sequence Models
Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance.
compare-mt: A Tool for Holistic Comparison of Language Generation Systems
In this paper, we describe compare-mt, a tool for holistic analysis and comparison of the results of systems for language generation tasks such as machine translation.
On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models
Adversarial examples --- perturbations to the input of a model that elicit large changes in the output --- have been shown to be an effective way of assessing the robustness of sequence-to-sequence (seq2seq) models.
MTNT: A Testbed for Machine Translation of Noisy Text
In this paper, we propose a benchmark dataset for Machine Translation of Noisy Text (MTNT), consisting of noisy comments on Reddit (www. reddit. com) and professionally sourced translations.
Extreme Adaptation for Personalized Neural Machine Translation
Every person speaks or writes their own flavor of their native language, influenced by a number of factors: the content they tend to talk about, their gender, their social status, or their geographical origin.
Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology-Based Representations
We investigate the pertinence of methods from algebraic topology for text data analysis.
Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representations
We investigate the pertinence of methods from algebraic topology for text data analysis.
DyNet: The Dynamic Neural Network Toolkit
In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives.
Blind phoneme segmentation with temporal prediction errors
Phonemic segmentation of speech is a critical step of speech recognition systems.
