Search Results for author:
Found 30 papers, 6 papers with code
Limitations of Knowledge Distillation for Zero-shot Transfer Learning
We demonstrate that in contradiction to the previous observation in the case of monolingual distillation, in multilingual settings, distillation during pretraining is more effective than distillation during fine-tuning for zero-shot transfer learning.
Controlled Data Generation via Insertion Operations for NLU
Use of synthetic data is rapidly emerging as a realistic alternative to manually annotating live traffic for industry-scale model building.
Attention Fusion: a light yet efficient late fusion mechanism for task adaptation in NLU
However, given the trend of larger pre-trained models, fine-tuning these models for each downstream task is parameter-inefficient and computationally-expensive deeming this approach sub-optimal for adoption by NLU systems.
Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with transcribed speech instead of conventional written text.
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models
(2) Conversely, using an encoder to warm-start seq2seq training, we show that by unfreezing the encoder partway through training, we can match task performance of a from-scratch seq2seq model.
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data
Currently popular video-text data mining approach via automatic speech recognition (ASR) used in HowTo100M provides low-quality captions that often do not refer to the video content.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Low-Resource Compositional Semantic Parsing with Concept Pretraining
In this work, we present an architecture to perform such domain adaptation automatically, with only a small amount of metadata about the new domain and without any new training data (zero-shot) or with very few examples (few-shot).
CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsing
A bottleneck to developing Semantic Parsing (SP) models is the need for a large volume of human-labeled training data.
LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging
We present LINGUIST, a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilingual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt.
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model
In this work, we demonstrate that multilingual large-scale sequence-to-sequence (seq2seq) models, pre-trained on a mixture of denoising and Causal Language Modeling (CLM) tasks, are more efficient few-shot learners than decoder-only models on various tasks.
Ranked #14 on
Natural Language Inference
on CommitmentBank
Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9. 3B, their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
Cross-Lingual Natural Language Inference
intent-classification
+5
Training Naturalized Semantic Parsers with Very Little Data
Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant.
Instilling Type Knowledge in Language Models via Multi-Task QA
Understanding human language often necessitates understanding entities and their place in a taxonomy of knowledge -- their types.
Contextual Domain Classification with Temporal Representations
In commercial dialogue systems, the Spoken Language Understanding (SLU) component tends to have numerous domains thus context is needed to help resolve ambiguities.
Zero-shot Generalization in Dialog State Tracking through Generative Question Answering
Dialog State Tracking (DST), an integral part of modern dialog systems, aims to track user preferences and constraints (slots) in task-oriented dialogs.
Exploring Transfer Learning For End-to-End Spoken Language Understanding
Since our model can process both speech and text input sequences and learn to predict a target sequence, it also allows us to do zero-shot E2E SLU by training on only text-hypothesis data (without any speech) from a new domain.
Ranked #3 on
Spoken Language Understanding
on Snips-SmartLights
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Delexicalized Paraphrase Generation
We present a neural model for paraphrasing and train it to generate delexicalized sentences.
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention
The comparison shows that our model could recover the transcription by integrating the fragmented information among hypotheses and identifying the frequent error patterns of the ASR module, and even rewrite the query for a better understanding, which reveals the characteristic of multi-task learning of broadcasting knowledge.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Style Attuned Pre-training and Parameter Efficient Fine-tuning for Spoken Language Understanding
The light encoder architecture separates the shared pre-trained networks from the mappings of generally encoded knowledge to specific domains of SLU, allowing for the domain adaptation to be performed solely at the light encoder and thus increasing efficiency.
Don't Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding
For complex parsing tasks, the state-of-the-art method is based on autoregressive sequence to sequence models to generate the parse directly.
Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
Virtual assistants such as Amazon Alexa, Apple Siri, and Google Assistant often rely on a semantic parsing component to understand which action(s) to execute for an utterance spoken by its users.
Improving Spoken Language Understanding By Exploiting ASR N-best Hypotheses
The NLU module usually uses the first best interpretation of a given speech in downstream tasks such as domain and intent classification.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Neural Cross-Lingual Coreference Resolution and its Application to Entity Linking
We propose an entity-centric neural cross-lingual coreference model that builds on multi-lingual embeddings and language-independent features.
Leveraging Context Information for Natural Question Generation
The task of natural question generation is to generate a corresponding question given the input passage (fact) and answer.
Ranked #11 on
Question Generation
on SQuAD1.1
Neural Cross-Lingual Entity Linking
A major challenge in Entity Linking (EL) is making effective use of contextual information to disambiguate mentions to Wikipedia that might refer to different entities in different contexts.
Ranked #3 on
Entity Disambiguation
on TAC2010
A Unified Query-based Generative Model for Question Generation and Question Answering
In the QG task, a question is generated from the system given the passage and the target answer, whereas in the QA task, the answer is generated given the question and the passage.
$k$-Nearest Neighbor Augmented Neural Networks for Text Classification
However, it lacks the capacity of utilizing instance-level information from individual instances in the training set.
Reinforcement Learning for Transition-Based Mention Detection
This paper describes an application of reinforcement learning to the mention detection task.
Bilateral Multi-Perspective Matching for Natural Language Sentences
Natural language sentence matching is a fundamental technology for a variety of tasks.
Ranked #17 on
Paraphrase Identification
on Quora Question Pairs
(Accuracy metric)
Multi-Perspective Context Matching for Machine Comprehension
Based on this dataset, we propose a Multi-Perspective Context Matching (MPCM) model, which is an end-to-end system that directly predicts the answer beginning and ending points in a passage.
Ranked #3 on
Open-Domain Question Answering
on SQuAD1.1
