Search Results for author:
Found 12 papers, 6 papers with code
A Comparative Analysis of Task-Agnostic Distillation Methods for Compressing Transformer Language Models
Our target of study includes Output Distribution (OD) transfer, Hidden State (HS) transfer with various layer mapping strategies, and Multi-Head Attention (MHA) transfer based on MiniLMv2.
Multiple Representation Transfer from Large Language Models to End-to-End ASR Systems
However, existing works only transfer a single representation of LLM (e. g. the last layer of pretrained BERT), while the representation of a text is inherently non-unique and can be obtained variously from different layers, contexts and models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models
In each episode of the search process, a NAS controller predicts a reward based on the distillation loss and latency of inference.
Sentence Identification with BOS and EOS Label Combinations
To tackle this issue, we formulate a novel task of sentence identification, where the goal is to identify SUs while excluding NSUs in a given text.
Policy-Adaptive Estimator Selection for Off-Policy Evaluation
Although many estimators have been developed, there is no single estimator that dominates the others, because the estimators' accuracy can vary greatly depending on a given OPE task such as the evaluation policy, number of actions, and noise level.
Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems
Large-scale language models (LLMs) such as GPT-2, BERT and RoBERTa have been successfully applied to ASR N-best rescoring.
Data-Driven Off-Policy Estimator Selection: An Application in User Marketing on An Online Content Delivery Service
As proof of concept, we use our procedure to select the best estimator to evaluate coupon treatment policies on a real-world online content delivery service.
Evaluating the Robustness of Off-Policy Evaluation
Unfortunately, identifying a reliable estimator from results reported in research papers is often difficult because the current experimental procedure evaluates and compares the estimators' performance on a narrow set of hyperparameters and evaluation policies.
Maintaining Common Ground in Dynamic Environments
Common grounding is the process of creating and maintaining mutual understandings, which is a critical aspect of sophisticated human communication.
End-To-End Dialogue Modelling
Goal-Oriented Dialogue Systems
+1
A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions
Recent models achieve promising results in visually grounded dialogues.
An Annotated Corpus of Reference Resolution for Interpreting Common Grounding
Common grounding is the process of creating, repairing and updating mutual understandings, which is a fundamental aspect of natural language conversation.
A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context
Finally, we evaluate and analyze baseline neural models on a simple subtask that requires recognition of the created common ground.
