Search Results for author:
Found 13 papers, 1 papers with code
Limitations of Knowledge Distillation for Zero-shot Transfer Learning
We demonstrate that in contradiction to the previous observation in the case of monolingual distillation, in multilingual settings, distillation during pretraining is more effective than distillation during fine-tuning for zero-shot transfer learning.
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models
We examine two such promising techniques, prefix tuning and bias-term tuning, specifically on semantic parsing.
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Specifically, training a bidirectional model like BERT on a discriminative objective such as minimum WER (MWER) has not been explored.
Automatic Speech Recognition
Natural Language Understanding
+1
Output Randomization: A Novel Defense for both White-box and Black-box Adversarial Models
Adversarial examples pose a threat to deep neural network models in a variety of scenarios, from settings where the adversary has complete knowledge of the model in a "white box" setting and to the opposite in a "black box" setting.
Using multiple ASR hypotheses to boost i18n NLU performance
For IC and NER multi-task experiments, when evaluating on the mismatched test set, we see improvements across all domains in German and in 17 out of 19 domains in Portuguese (improvements based on change in SeMER scores).
Abstractive Text Summarization
Automatic Speech Recognition
+5
Compressing Transformer-Based Semantic Parsing Models using Compositional Code Embeddings
The current state-of-the-art task-oriented semantic parsing models use BERT or RoBERTa as pretrained encoders; these models have huge memory footprints.
Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding
For complex parsing tasks, the state-of-the-art method is based on autoregressive sequence to sequence models to generate the parse directly.
Optimal Mini-Batch Size Selection for Fast Gradient Descent
This paper presents a methodology for selecting the mini-batch size that minimizes Stochastic Gradient Descent (SGD) learning time for single and multiple learner problems.
Thwarting finite difference adversarial attacks with output randomization
Adversarial examples pose a threat to deep neural network models in a variety of scenarios, from settings where the adversary has complete knowledge of the model and to the opposite "black box" setting.
Deep density ratio estimation for change point detection
In this work, we propose new objective functions to train deep neural network based density ratio estimators and apply it to a change point detection problem.
Generation & Evaluation of Adversarial Examples for Malware Obfuscation
There has been an increased interest in the application of convolutional neural networks for image based malware classification, but the susceptibility of neural networks to adversarial examples allows malicious actors to evade classifiers.
Learning filter widths of spectral decompositions with wavelets
Our results show that the WD layer can improve neural network based time series classifiers both in accuracy and interpretability by learning directly from the input signal.
Focal onset seizure prediction using convolutional networks
Significance: We demonstrate that a robust set of features can be learned from scalp EEG that characterize the preictal state of focal seizures.
