Search Results for author:
Found 11 papers, 4 papers with code
GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism
Additionally, this study explores a novel recurrent routing strategy that may inspire further advancements in enhancing the reasoning capabilities of language models.
QAEncoder: Towards Aligned Representation Learning in Question Answering System
Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses.
Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach
In this work, we analyze the order-preserving ability on the whole search space (global) and a sub-space of top architectures (local), and empirically show that the local order-preserving for current two-stage NAS methods still need to be improved.
Poisson Process for Bayesian Optimization
BayesianOptimization(BO) is a sample-efficient black-box optimizer, and extensive methods have been proposed to build the absolute function response of the black-box function through a probabilistic surrogate model, including Tree-structured Parzen Estimator (TPE), random forest (SMAC), and Gaussian process (GP).
EAutoDet: Efficient Architecture Search for Object Detection
In contrast, this paper introduces an efficient framework, named EAutoDet, that can discover practical backbone and FPN architectures for object detection in 1. 4 GPU-days.
Ranked #36 on
Object Detection In Aerial Images
on DOTA
(using extra training data)
ZARTS: On Zero-order Optimization for Neural Architecture Search
Also, we search on the search space of DARTS to compare with peer methods, and our discovered architecture achieves 97. 54% accuracy on CIFAR-10 and 75. 7% top-1 accuracy on ImageNet, which are state-of-the-art performance.
DAAS: Differentiable Architecture and Augmentation Policy Search
Neural architecture search (NAS) has been an active direction of automatic machine learning (Auto-ML), aiming to explore efficient network structures.
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation
Albeit being a prevalent architecture searching approach, differentiable architecture search (DARTS) is largely hindered by its substantial memory cost since the entire supernet resides in the memory.
DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
We call this approach DARTS-.
Ranked #21 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Ternary Weight Networks
We present a memory and computation efficient ternary weight networks (TWNs) - with weights constrained to +1, 0 and -1.
