Search Results for author:
Found 48 papers, 24 papers with code
Energy-Based Models with Applications to Speech and Language Processing
Therefore, the purpose of this monograph is to present a systematic introduction to energy-based models, including both algorithmic progress and applications in speech and language processing.
Prompt Pool based Class-Incremental Continual Learning for Dialog State Tracking
Inspired by the recently emerging prompt tuning method that performs well on dialog systems, we propose to use the prompt pool method, where we maintain a pool of key-value paired prompts and select prompts from the pool according to the distance between the dialog history and the prompt keys.
UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt
Using the high-quality prompts generated, we scale the corpus of the pre-trained conversation model to 122 datasets from 15 dialog-related tasks, resulting in Universal Pre-trained Conversation Model (UniPCM), a powerful foundation model for various conversational tasks and different dialog systems.
Exploring Energy-based Language Models with Different Architectures and Training Methods for Speech Recognition
Energy-based language models (ELMs) parameterize an unnormalized distribution for natural sentences and are radically different from popular autoregressive language models (ALMs).
Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervision
Most existing task-oriented dialog (TOD) systems track dialog states in terms of slots and values and use them to query a database to get relevant knowledge to generate responses.
Persistently Trained, Diffusion-assisted Energy-based Models
Maximum likelihood (ML) learning for energy-based models (EBMs) is challenging, partly due to non-convergence of Markov chain Monte Carlo. Several variations of ML learning have been proposed, but existing methods all fail to achieve both post-training image generation and proper density estimation.
A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems
Second, an important ingredient in a US is that the user goal can be effectively incorporated and tracked; but how to flexibly integrate goal state tracking and develop an end-to-end trainable US for multi-domains has remained to be a challenge.
Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture
Recently, there has been progress in supervised funetuning pretrained GPT-2 to build end-to-end task-oriented dialog (TOD) systems.
Information Extraction and Human-Robot Dialogue towards Real-life Tasks: A Baseline Study with the MobileCS Dataset
Recently, there have merged a class of task-oriented dialogue (TOD) datasets collected through Wizard-of-Oz simulated games.
Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models
In this paper, we propose to apply JSA to semi-supervised learning of the latent state TOD models, which is referred to as JSA-TOD.
A Challenge on Semi-Supervised and Reinforced Task-Oriented Dialog Systems
A challenge on Semi-Supervised and Reinforced Task-Oriented Dialog Systems, Co-located with EMNLP2022 SereTOD Workshop.
Building Markovian Generative Architectures over Pretrained LM Backbones for Efficient Task-Oriented Dialog Systems
Recently, Transformer based pretrained language models (PLMs), such as GPT2 and T5, have been leveraged to build generative task-oriented dialog (TOD) systems.
CUSIDE: Chunking, Simulating Future Context and Decoding for Streaming ASR
The simulation module is jointly trained with the ASR model using a self-supervised loss; the ASR model is optimized with the usual ASR loss, e. g., CTC-CRF as used in our experiments.
Exploiting Single-Channel Speech for Multi-Channel End-to-End Speech Recognition: A Comparative Study
In this paper, we systematically compare the performance of three schemes to exploit external single-channel data for multi-channel end-to-end ASR, namely back-end pre-training, data scheduling, and data simulation, under different settings such as the sizes of the single-channel data and the choices of the front-end.
An Empirical Study of Language Model Integration for Transducer based Speech Recognition
Based on the DR method, we propose a low-order density ratio method (LODR) by replacing the estimation with a low-order weak language model.
Callee: Recovering Call Graphs for Binaries with Transfer and Contrastive Learning
Recovering binary programs' call graphs is crucial for inter-procedural analysis tasks and applications based on them. transfer One of the core challenges is recognizing targets of indirect calls (i. e., indirect callees).
Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems
In this paper, we propose Variational Latent-State GPT model (VLS-GPT), which is the first to combine the strengths of the two approaches.
Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings
The use of phonological features (PFs) potentially allows language-specific phones to remain linked in training, which is highly desirable for information sharing for multilingual and crosslingual speech recognition methods for low-resourced languages.
Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers
Automatic speech recognition systems have been largely improved in the past few decades and current systems are mainly hybrid-based and end-to-end-based.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition
Recently, the end-to-end training approach for neural beamformer-supported multi-channel ASR has shown its effectiveness in multi-channel speech recognition.
Deformable TDNN with adaptive receptive fields for speech recognition
Time Delay Neural Networks (TDNNs) are widely used in both DNN-HMM based hybrid speech recognition systems and recent end-to-end systems.
The SLT 2021 children speech recognition challenge: Open datasets, rules and baselines
Automatic speech recognition (ASR) has been significantly advanced with the use of deep learning and big data.
Sound Audio and Speech Processing
Efficient Neural Architecture Search for End-to-end Speech Recognition via Straight-Through Gradients
Using ST gradients to support sub-graph sampling is a core element to achieve efficient NAS beyond DARTS and SNAS.
Ranked #1 on
Speech Recognition
on WSJ dev93
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
An empirical study of domain-agnostic semi-supervised learning via energy-based models: joint-training and pre-training
In contrast, generative SSL methods involve unsupervised learning based on generative models by either joint-training or pre-training, and are more appealing from the perspective of being domain-agnostic, since they do not inherently require data augmentations.
A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning
In this paper we aim at alleviating the reliance on belief state labels in building end-to-end dialog systems, by leveraging unlabeled dialog data towards semi-supervised learning.
Ranked #2 on
End-To-End Dialogue Modelling
on MULTIWOZ 2.1
Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models
Although with progress in introducing auxiliary amortized inference models, learning discrete latent variable models is still challenging.
CAT: A CTC-CRF based ASR Toolkit Bridging the Hybrid and the End-to-end Approaches towards Data Efficiency and Low Latency
In this paper, we present a new open source toolkit for speech recognition, named CAT (CTC-CRF based ASR Toolkit).
Ranked #1 on
Speech Recognition
on Hub5'00 FISHER-SWBD
Paraphrase Augmented Task-Oriented Dialog Generation
Neural generative models have achieved promising performance on dialog generation tasks if given a huge data set.
Integrating Discrete and Neural Features via Mixed-feature Trans-dimensional Random Field Language Models
There has been a long recognition that discrete features (n-gram features) and neural network based features have complementary strengths for language models (LMs).
Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context
Conversations have an intrinsic one-to-many property, which means that multiple responses can be appropriate for the same dialog context.
Ranked #6 on
End-To-End Dialogue Modelling
on MULTIWOZ 2.0
CAT: CRF-based ASR Toolkit
In this paper, we present a new open source toolkit for automatic speech recognition (ASR), named CAT (CRF-based ASR Toolkit).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
CRF-based Single-stage Acoustic Modeling with CTC Topology
CTC-CRF is conceptually simple, which basically implements a CRF layer on top of features generated by the bottom neural network with the special state topology.
Ranked #2 on
Speech Recognition
on WSJ eval93
Elastic CRFs for Open-ontology Slot Filling
Second, the one-hot encoding of slot labels ignores the semantic meanings and relations for slots, which are implicit in their natural language descriptions.
Learning Neural Random Fields with Inclusive Auxiliary Generators
Neural random fields (NRFs), which are defined by using neural networks to implement potential functions in undirected models, provide an interesting family of model spaces for machine learning.
A Review of Learning with Deep Generative Models from Perspective of Graphical Modeling
This document aims to provide a review on learning with deep generative models (DGMs), which is an highly-active area in machine learning and more generally, artificial intelligence.
Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units
In this paper, we present an end-to-end automatic speech recognition system, which successfully employs subword units in a hybrid CTC-Attention based system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
First, a dynamic noise distribution is introduced and trained simultaneously to converge to the data distribution.
Generative Modeling by Inclusive Neural Random Fields with Applications in Image Generation and Anomaly Detection
With these contributions and results, this paper significantly advances the learning and applications of NRFs to a new level, both theoretically and empirically, which have never been obtained before.
Learning Sparse Structured Ensembles with SG-MCMC and Network Pruning
An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing.
Tracking of enriched dialog states for flexible conversational information access
The above observations motivate us to enrich current representation of dialog states and collect a brand new dialog dataset about movies, based upon which we build a new DST, called enriched DST (EDST), for flexible accessing movie information.
Learning neural trans-dimensional random field language models with noise-contrastive estimation
However, the training efficiency of neural TRF LMs is not satisfactory, which limits the scalability of TRF LMs on large training corpus.
Language modeling with Neural trans-dimensional random fields
The idea is to use nonlinear potentials with continuous features, implemented by neural networks (NNs), in the TRF framework.
Joint Bayesian Gaussian discriminant analysis for speaker verification
State-of-the-art i-vector based speaker verification relies on variants of Probabilistic Linear Discriminant Analysis (PLDA) for discriminant analysis.
Model Interpolation with Trans-dimensional Random Field Language Models for Speech Recognition
The dominant language models (LMs) such as n-gram and neural network (NN) models represent sentence probabilities in terms of conditionals.
Joint Stochastic Approximation learning of Helmholtz Machines
Though with progress, model learning and performing posterior inference still remains a common challenge for using deep generative models, especially for handling discrete hidden variables.
Block-Wise MAP Inference for Determinantal Point Processes with Application to Change-Point Detection
Existing MAP inference algorithms for determinantal point processes (DPPs) need to calculate determinants or conduct eigenvalue decomposition generally at the scale of the full kernel, which presents a great challenge for real-world applications.
