Search Results for author: Julien Siems

Found 6 papers, 5 papers with code

Is Mamba Capable of In-Context Learning?

1 code implementation5 Feb 2024 Riccardo Grazzi, Julien Siems, Simon Schrodi, Thomas Brox, Frank Hutter

State of the art foundation models such as GPT-4 perform surprisingly well at in-context learning (ICL), a variant of meta-learning concerning the learned ability to solve tasks during a neural network forward pass, exploiting contextual information provided as input to the model.

In-Context Learning

Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models

1 code implementation NeurIPS 2023 Julien Siems, Konstantin Ditschuneit, Winfried Ripken, Alma Lindborg, Maximilian Schambach, Johannes S. Otterbach, Martin Genzel

Generalized Additive Models (GAMs) have recently experienced a resurgence in popularity due to their interpretability, which arises from expressing the target value as a sum of non-linear transformations of the features.

Additive models Time Series

Bayesian Optimization-based Combinatorial Assignment

1 code implementation31 Aug 2022 Jakob Weissteiner, Jakob Heiss, Julien Siems, Sven Seuken

In this paper, we address this shortcoming by presenting a Bayesian optimization-based combinatorial assignment (BOCA) mechanism.

Bayesian Optimization

Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks

1 code implementation ICLR 2022 Arber Zela, Julien Siems, Lucas Zimmer, Jovita Lukasik, Margret Keuper, Frank Hutter

We show that surrogate NAS benchmarks can model the true performance of architectures better than tabular benchmarks (at a small fraction of the cost), that they lead to faithful estimates of how well different NAS methods work on the original non-surrogate benchmark, and that they can generate new scientific insight.

Neural Architecture Search

NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

1 code implementation ICLR 2020 Arber Zela, Julien Siems, Frank Hutter

One-shot neural architecture search (NAS) has played a crucial role in making NAS methods computationally feasible in practice.

Benchmarking Neural Architecture Search

Cannot find the paper you are looking for? You can Submit a new open access paper.