Search Results for author:
Found 17 papers, 3 papers with code
Expand-and-Quantize: Unsupervised Semantic Segmentation Using High-Dimensional Space and Product Quantization
In this paper, we propose a novel USS framework called Expand-and-Quantize Unsupervised Semantic Segmentation (EQUSS), which combines the benefits of high-dimensional spaces for better clustering and product quantization for effective information compression.
Ranked #3 on
Unsupervised Semantic Segmentation
on Potsdam-3
Knowledge Distillation from Non-streaming to Streaming ASR Encoder using Auxiliary Non-streaming Layer
Streaming automatic speech recognition (ASR) models are restricted from accessing future context, which results in worse performance compared to the non-streaming models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improving Small Footprint Few-shot Keyword Spotting with Supervision on Auxiliary Data
Few-shot keyword spotting (FS-KWS) models usually require large-scale annotated datasets to generalize to unseen target keywords.
Depth-Relative Self Attention for Monocular Depth Estimation
Monocular depth estimation is very challenging because clues to the exact depth are incomplete in a single RGB image.
Semantic-Preserving Augmentation for Robust Image-Text Retrieval
Image text retrieval is a task to search for the proper textual descriptions of the visual world and vice versa.
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers
Pre-trained Transformer models such as BERT have shown great success in a wide range of applications, but at the cost of substantial increases in model complexity.
Vision Transformer-based Feature Extraction for Generalized Zero-Shot Learning
In AAM, the correlation between each patch feature and the synthetic image attribute is used as the importance weight for each patch.
Exploring Attention Map Reuse for Efficient Transformer Neural Networks
In this paper, we provide a comprehensive study on attention map reuse focusing on its ability to accelerate inference.
A Comparison of Transformer, Convolutional, and Recurrent Neural Networks on Phoneme Recognition
Our analyses show that Transformer and Conformer models benefit from the long-range accessibility of self-attention through input frames.
Towards Intelligent Millimeter and Terahertz Communication for 6G: Computer Vision-aided Beamforming
Beamforming technique realized by the multiple-input-multiple-output (MIMO) antenna arrays has been widely used to compensate for the severe path loss in the millimeter wave (mmWave) bands.
Similarity and Content-based Phonetic Self Attention for Speech Recognition
Especially, SA heads in lower layers capture various phonetic characteristics by the query-key dot product, which is designed to compute the pairwise relationship between frames.
Korean Tokenization for Beam Search Rescoring in Speech Recognition
Although the common approach is to use the same tokenization method for external LM as the ASR model, we show that it may not be the best choice for Korean.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Semantic Feature Extraction for Generalized Zero-shot Learning
Key idea of the proposed approach, henceforth referred to as semantic feature extraction-based GZSL (SE-GZSL), is to use the semantic feature containing only attribute-related information in learning the relationship between the image and the attribute.
TernGEMM: GEneral Matrix Multiply Library with Ternary Weights for Fast DNN Inference
We propose TernGEMM, a special GEMM library using SIMD instructions for Deep Neural Network (DNN) inference with ternary weights and activations under 8-bit.
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
While Transformer-based models have shown impressive language modeling performance, the large computation cost is often prohibitive for practical use.
Understanding the Role of Self Attention for Efficient Speech Recognition
Self-attention (SA) is a critical component of Transformer neural networks that have succeeded in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SVD-Softmax: Fast Softmax Approximation on Large Vocabulary Neural Networks
The approximate probability of each word can be estimated with only a small part of the weight matrix by using a few large singular values and the corresponding elements for most of the words.
