Search Results for author:
Found 21 papers, 16 papers with code
Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
We extend this result to non-diagonal LRNNs, which have recently shown promise in models such as DeltaNet.
Ensembling Finetuned Language Models for Text Classification
Finetuning is a common practice widespread across different communities to adapt pretrained models to particular tasks.
Dynamic Post-Hoc Neural Ensemblers
In this study, we explore employing neural networks as ensemble methods, emphasizing the significance of dynamic ensembling to leverage diverse model predictions adaptively.
GAMformer: In-Context Learning for Generalized Additive Models
Generalized Additive Models (GAMs) are widely recognized for their ability to create fully interpretable machine learning models for tabular data.
HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models
The increasing size of language models necessitates a thorough analysis across multiple dimensions to assess trade-offs among crucial hardware metrics such as latency, energy consumption, GPU memory usage, and performance.
Multi-objective Differentiable Neural Architecture Search
Pareto front profiling in multi-objective optimization (MOO), i. e. finding a diverse set of Pareto optimal solutions, is challenging, especially with expensive objectives like neural network training.
Neural Architecture Search: Insights from 1000 Papers
Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas.
Natural Language Understanding
Neural Architecture Search
+2
NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies
Zero-cost proxies (ZC proxies) are a recent architecture performance prediction technique aiming to significantly speed up algorithms for neural architecture search (NAS).
Neural Architecture Search for Dense Prediction Tasks in Computer Vision
The success of deep learning in recent years has lead to a rising demand for neural network architecture engineering.
NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy
The release of tabular benchmarks, such as NAS-Bench-101 and NAS-Bench-201, has significantly lowered the computational overhead for conducting scientific research in neural architecture search (NAS).
Multi-headed Neural Ensemble Search
Ensembles of CNN models trained with different seeds (also known as Deep Ensembles) are known to achieve superior performance over a single copy of the CNN.
Bag of Tricks for Neural Architecture Search
While neural architecture search methods have been successful in previous years and led to new state-of-the-art performance on various problems, they have also been criticized for being unstable, being highly sensitive with respect to their hyperparameters, and often not performing better than random search.
How Powerful are Performance Predictors in Neural Architecture Search?
Early methods in the rapidly developing field of neural architecture search (NAS) required fully training thousands of neural networks.
NASLib: A Modular and Flexible Neural Architecture Search Library
Neural Architecture Search (NAS) is one of the focal points for the Deep Learning community, but reproducing NAS methods is extremely challenging due to numerous low-level implementation details.
Smooth Variational Graph Embeddings for Efficient Neural Architecture Search
We evaluate the proposed approach on neural architectures defined by the ENAS approach, the NAS-Bench-101 and the NAS-Bench-201 search space and show that our smooth embedding space allows to directly extrapolate the performance prediction to architectures outside the seen domain (e. g. with more operations).
Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks
We show that surrogate NAS benchmarks can model the true performance of architectures better than tabular benchmarks (at a small fraction of the cost), that they lead to faithful estimates of how well different NAS methods work on the original non-surrogate benchmark, and that they can generate new scientific insight.
Neural Ensemble Search for Uncertainty Estimation and Dataset Shift
On a variety of classification tasks and modern architecture search spaces, we show that the resulting ensembles outperform deep ensembles not only in terms of accuracy but also uncertainty calibration and robustness to dataset shift.
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search
One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice.
Understanding and Robustifying Differentiable Architecture Search
Differentiable Architecture Search (DARTS) has attracted a lot of attention due to its simplicity and small search costs achieved by a continuous relaxation and an approximation of the resulting bi-level optimization problem.
AutoDispNet: Improving Disparity Estimation With AutoML
In this work, we show how to use and extend existing AutoML techniques to efficiently optimize large-scale U-Net-like encoder-decoder architectures.
Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal.
