Search Results for author:
Found 31 papers, 18 papers with code
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
On the GLaM dataset, DoReMi, which has no knowledge of downstream tasks, even matches the performance of using domain weights tuned on downstream tasks.
Symbolic Discovery of Optimization Algorithms
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
Ranked #1 on
Image Classification
on ImageNet
PyGlove: Efficiently Exchanging ML Ideas as Code
We also perform a case study of a large codebase where PyGlove led to an 80% reduction in the number of lines of code.
Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version
(i) Linear complexity: we introduce a novel patch attention with linear complexity.
Automated Deep Learning: Neural Architecture Search Is Not the End
That stated, NAS is not the be-all and end-all of AutoDL.
Recognizing Vector Graphics without Rasterization
In this paper, we consider a different data format for images: vector graphics.
Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision
The energy consumption of deep learning models is increasing at a breathtaking rate, which raises concerns due to potential negative effects on carbon neutrality in the context of global warming and climate change.
Rethinking Co-design of Neural Architectures and Hardware Accelerators
In our work, we target the optimization of hardware and software configurations on an industry-standard edge accelerator.
Isometric Propagation Network for Generalized Zero-shot Learning
To resolve this problem, we propose Isometric Propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces.
Supervision by Registration and Triangulation for Landmark Detection
End-to-end training is made possible by differentiable registration and 3D triangulation modules.
PyGlove: Symbolic Programming for Automated Machine Learning
As a result, AutoML can be reformulated as an automated process of symbolic manipulation.
NAHAS: Neural Architecture and Hardware Accelerator Search
In this paper, we study the importance of co-designing neural architectures and hardware accelerators.
MASP: Model-Agnostic Sample Propagation for Few-shot learning
Few-shot learning aims to train a classifier given only a few samples per class that are highly insufficient to describe the whole data distribution.
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm.
AutoHAS: Efficient Hyperparameter and Architecture Search
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures.
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
A variety of algorithms search architectures under different search space.
One-Shot Neural Architecture Search via Self-Evaluated Template Network
In this paper, we propose a Self-Evaluated Template Network (SETN) to improve the quality of the architecture candidates for evaluation so that it is more likely to cover competitive candidates.
Ranked #17 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Searching for A Robust Neural Architecture in Four GPU Hours
To avoid traversing all the possibilities of the sub-graphs, we develop a differentiable sampler over the DAG.
Ranked #18 on
Neural Architecture Search
on CIFAR-10
Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection
A typical approach is to (1) train a detector on the labeled images; (2) generate new training samples using this detector's prediction as pseudo labels of unlabeled images; (3) retrain the detector on the labeled samples and partial pseudo labeled samples.
Ranked #1 on
Facial Landmark Detection
on 300W (Full)
(using extra training data)
Network Pruning via Transformable Architecture Search
The maximum probability for the size in each distribution serves as the width and depth of the pruned network, whose parameters are learned by knowledge transfer, e. g., knowledge distillation, from the original networks.
Ranked #1 on
Network Pruning
on CIFAR-10
Auto-ReID: Searching for a Part-aware ConvNet for Person Re-Identification
We propose to automatically search for a CNN architecture that is specifically suitable for the reID task.
Ranked #9 on
Person Re-Identification
on CUHK03 detected
Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks
With asymptotic pruning, the information of the training set would be gradually concentrated in the remaining filters, so the subsequent training and pruning process would be stable.
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
Therefore, the network trained by our method has a larger model capacity to learn from the training data.
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors
In this paper, we present supervision-by-registration, an unsupervised approach to improve the precision of facial landmark detectors on both images and video.
Ranked #1 on
Facial Landmark Detection
on 300-VW (C)
Exploit the Unknown Gradually: One-Shot Video-Based Person Re-Identification by Stepwise Learning
We focus on the one-shot learning for video-based person re-Identification (re-ID).
Style Aggregated Network for Facial Landmark Detection
In this work, we propose a style-aggregated approach to deal with the large intrinsic variance of image styles for facial landmark detection.
Ranked #1 on
Facial Landmark Detection
on AFLW-Front
(Mean NME metric)
EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks
For most state-of-the-art architectures, Rectified Linear Unit (ReLU) becomes a standard component accompanied with each layer.
Ranked #12 on
Image Classification
on SVHN
Self-Paced Co-training
During co-training process, labels of unlabeled instances in the training pool are very likely to be false especially in the initial training rounds, while the standard co-training algorithm utilizes a “draw without replacement” manner and does not remove these false labeled instances from training.
PatchShuffle Regularization
This paper focuses on regularizing the training of the convolutional neural network (CNN).
Few-Example Object Detection with Model Communication
Experiments on PASCAL VOC'07, MS COCO'14, and ILSVRC'13 indicate that by using as few as three or four samples selected for each category, our method produces very competitive results when compared to the state-of-the-art weakly-supervised approaches using a large number of image-level labels.
Ranked #1 on
Weakly Supervised Object Detection
on COCO
More is Less: A More Complicated Network with Less Inference Complexity
In this paper, we present a novel and general network structure towards accelerating the inference process of convolutional neural networks, which is more complicated in network structure yet with less inference complexity.
