Search Results for author:
Found 19 papers, 7 papers with code
ParZC: Parametric Zero-Cost Proxies for Efficient NAS
Recent advancements in Zero-shot Neural Architecture Search (NAS) highlight the efficacy of zero-cost proxies in various NAS benchmarks.
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery
Based on the discovered zero-cost proxy, we conduct a ViT architecture search in a training-free manner.
TVT: Training-Free Vision Transformer Search on Tiny Datasets
In this paper, for the first time, we investigate how to search in a training-free manner with the help of teacher models and devise an effective Training-free ViT (TVT) search framework.
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization
In particular, we devise an elaborate search space involving the existing proxies and perform an evolution search to discover the best correlated MQ proxy.
NORM: Knowledge Distillation via N-to-One Representation Matching
By sequentially splitting the expanded student representation into N non-overlapping feature segments having the same number of feature channels as the teacher's, they can be readily forced to approximate the intact teacher representation simultaneously, formulating a novel many-to-one representation matching mechanism conditioned on a single teacher-student layer pair.
Catch-Up Distillation: You Only Need to Train Once for Accelerating Sampling
On CIFAR-10, we obtain a FID of 2. 80 by sampling in 15 steps under one-session training and the new state-of-the-art FID of 3. 37 by sampling in one step with additional training.
DisWOT: Student Architecture Search for Distillation WithOut Training
In this way, our student architecture search for Distillation WithOut Training (DisWOT) significantly improves the performance of the model in the distillation stage with at least 180$\times$ training acceleration.
Progressive Meta-Pooling Learning for Lightweight Image Classification Model
Practical networks for edge devices adopt shallow depth and small convolutional kernels to save memory and computational cost, which leads to a restricted receptive field.
RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies
In this paper, we propose Ranking Distillation one-shot NAS (RD-NAS) to enhance ranking consistency, which utilizes zero-cost proxies as the cheap teacher and adopts the margin ranking loss to distill the ranking knowledge.
GP-NAS-ensemble: a model for NAS Performance Prediction
It is of great significance to estimate the performance of a given model architecture without training in the application of Neural Architecture Search (NAS) as it may take a lot of time to evaluate the performance of an architecture.
Automated Knowledge Distillation via Monte Carlo Tree Search
In this paper, we present Auto-KD, the first automated search framework for optimal knowledge distillation design.
DMFormer: Closing the Gap Between CNN and Vision Transformers
Vision transformers have shown excellent performance in computer vision tasks.
Prior-Guided One-shot Neural Architecture Search
In this paper, we present Prior-Guided One-shot NAS (PGONAS) to strengthen the ranking correlation of supernets.
Multi-trial Neural Architecture Search with Lottery Tickets
Neural architecture search (NAS) has brought significant progress in recent image recognition tasks.
Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation
Extensive experiments show that AMR establishes a new state-of-the-art performance on the PASCAL VOC 2012 dataset, surpassing not only current methods trained with the image-level of supervision but also some methods relying on stronger supervision, such as saliency label.
Adversarial Joint Training with Self-Attention Mechanism for Robust End-to-End Speech Recognition
Inspired by the extensive applications of the generative adversarial networks (GANs) in speech enhancement and ASR tasks, we propose an adversarial joint training framework with the self-attention mechanism to boost the noise robustness of the ASR system.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Explicit Connection Distillation
In this paper, we present Explicit Connection Distillation (ECD), a new KD framework, which addresses the knowledge distillation problem in a novel perspective of bridging dense intermediate feature connections between a student network and its corresponding teacher generated automatically in the training, achieving knowledge transfer goal via direct cross-network layer-to-layer gradients propagation, without need to define complex distillation losses and assume a pre-trained teacher model to be available.
CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition
In this work, we combine freely available corpora for German speech recognition, including yet unlabeled speech data, to a big dataset of over $1700$h of speech data.
Ranked #5 on
Speech Recognition
on TUDA
(using extra training data)
Speech Recognition
Audio and Speech Processing
Regularized Forward-Backward Decoder for Attention Models
Nowadays, attention models are one of the popular candidates for speech recognition.
