Search Results for author:
Found 17 papers, 13 papers with code
VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting
Combining CNNs or ViTs, with RNNs for spatiotemporal forecasting, has yielded unparalleled results in predicting temporal and spatial dynamics.
Evolutionary Optimization of Model Merging Recipes
Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks.
Evolution Transformer: In-Context Evolutionary Optimization
Given a trajectory of evaluations and search distribution statistics, Evolution Transformer outputs a performance-improving update to the search distribution.
Large Language Models As Evolution Strategies
Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms.
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views
By combining two complementing models, LEVI effectively suppresses problematic features in both the fine-tuning data and pre-trained model and preserves useful features for new tasks.
NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications
Recently, the Deep Learning community has become interested in evolutionary optimization (EO) as a means to address hard optimization problems, e. g. meta-learning through long inner loop unrolls or optimizing non-differentiable operators.
PostRainBench: A comprehensive benchmark and a new model for precipitation forecasting
To address these limitations, we introduce the PostRainBench, a comprehensive multi-variable NWP post-processing benchmark consisting of three datasets for NWP post-processing-based precipitation forecasting.
PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting
Although the Transformer has been the dominant architecture for time series forecasting tasks in recent years, a fundamental challenge remains: the permutation-invariant self-attention mechanism within Transformers leads to a loss of temporal information.
Ranked #1 on
Time Series Forecasting
on ETTh2 (336) Multivariate
DEIR: Efficient and Robust Exploration through Discriminative-Model-Based Episodic Intrinsic Rewards
Exploration is a fundamental aspect of reinforcement learning (RL), and its effectiveness is a deciding factor in the performance of RL algorithms, especially when facing sparse extrinsic rewards.
Collective Intelligence for 2D Push Manipulations with Mobile Robots
While natural systems often present collective intelligence that allows them to self-organize and adapt to changes, the equivalent is missing in most artificial systems.
Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter
We show that current agents struggle to generalize, and introduce novel object-centric agents that improve over strong baselines.
Evolving Modular Soft Robots without Explicit Inter-Module Communication using Local Self-Attention
We show experimentally that the evolved robots are effective in the task of locomotion: thanks to self-attention, instances of the same controller embodied in the same robot can focus on different inputs.
EvoJAX: Hardware-Accelerated Neuroevolution
Evolutionary computation has been shown to be a highly effective method for training neural networks, particularly when employed at scale on CPU clusters.
Collective Intelligence for Deep Learning: A Survey of Recent Developments
In this review, we will provide a historical context of neural network research's involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its current capabilities.
The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
In complex systems, we often observe complex global behavior emerge from a collection of agents interacting with each other in their environment, with each individual agent acting only on locally available information, without knowing the full picture.
Learning Agile Locomotion via Adversarial Training
In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility.
Neuroevolution of Self-Interpretable Agents
Inattentional blindness is the psychological phenomenon that causes one to miss things in plain sight.
