Search Results for author: Risto Miikkulainen

Found 52 papers, 13 papers with code

Characterizing Dynamic Word Meaning Representations in the Brain

no code implementations COLING (CogALex) 2020 Nora Aguirre-Celis, Risto Miikkulainen

During sentence comprehension, humans adjust word meanings according to the combination of the concepts that occur in the sentence.

Accelerating Evolution Through Gene Masking and Distributed Search

no code implementations13 Feb 2023 Hormoz Shahrzad, Risto Miikkulainen

In building practical applications of evolutionary computation (EC), two optimizations are essential.

Efficient Activation Function Optimization through Surrogate Modeling

2 code implementations13 Jan 2023 Garrett Bingham, Risto Miikkulainen

Carefully designed activation functions can improve the performance of neural networks in many machine learning tasks.

Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search

1 code implementation25 Oct 2022 Xin Qiu, Risto Miikkulainen

This paper presents the first theoretical analysis of the behaviors of mutation, crossover and RL in black-box NAS, and proposes a new crossover operator based on the shortest edit path (SEP) in graph space.

Neural Architecture Search Reinforcement Learning (RL)

EVOTER: Evolution of Transparent Explainable Rule-sets

no code implementations21 Apr 2022 Hormoz Shahrzad, Babak Hodjat, Risto Miikkulainen

The approach is evaluated in several prediction/classification and prescription/policy search domains with and without a surrogate.

Effective Mutation Rate Adaptation through Group Elite Selection

no code implementations11 Apr 2022 Akarsh Kumar, Bo Liu, Risto Miikkulainen, Peter Stone

GESMR co-evolves a population of solutions and a population of MRs, such that each MR is assigned to a group of solutions.

Image Classification

DIAS: A Domain-Independent Alife-Based Problem-Solving System

no code implementations14 Mar 2022 Babak Hodjat, Hormoz Shahrzad, Risto Miikkulainen

A domain-independent problem-solving system based on principles of Artificial Life is introduced.

Artificial Life

Simple Genetic Operators are Universal Approximators of Probability Distributions (and other Advantages of Expressive Encodings)

no code implementations19 Feb 2022 Elliot Meyerson, Xin Qiu, Risto Miikkulainen

The conclusion is that, across evolutionary computation areas as diverse as genetic programming, neuroevolution, genetic algorithms, and theory, expressive encodings can be a key to understanding and realizing the full power of evolution.

NeuroComb: Improving SAT Solving with Graph Neural Networks

no code implementations26 Oct 2021 Wenxi Wang, Yang Hu, Mohit Tiwari, Sarfraz Khurshid, Kenneth McMillan, Risto Miikkulainen

Aiming to make GNN improvements practical, this paper proposes an approach called NeuroComb, which builds on two insights: (1) predictions of important variables and clauses can be combined with dynamic branching into a more effective hybrid branching strategy, and (2) it is sufficient to query the neural model only once for the predictions before the SAT solving starts.

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks

1 code implementation18 Sep 2021 Garrett Bingham, Risto Miikkulainen

By analytically tracking the mean and variance of signals as they propagate through the network, AutoInit appropriately scales the weights at each layer to avoid exploding or vanishing signals.

Meta-Learning Neural Architecture Search +1

Evolving GAN Formulations for Higher Quality Image Synthesis

no code implementations17 Feb 2021 Santiago Gonzalez, Mohak Kant, Risto Miikkulainen

Generative Adversarial Networks (GANs) have extended deep learning to complex generation and translation tasks across different data modalities.

Image-to-Image Translation Translation

The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings

no code implementations ICLR 2021 Elliot Meyerson, Risto Miikkulainen

This paper frames a general prediction system as an observer traveling around a continuous space, measuring values at some locations, and predicting them at others.

Multi-Task Learning

Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model

1 code implementation5 Oct 2020 Xin Qiu, Risto Miikkulainen

This framework, RED, builds an error detector on top of the base classifier and estimates uncertainty of the detection scores using Gaussian Processes.

Gaussian Processes

Effective Regularization Through Loss-Function Metalearning

no code implementations2 Oct 2020 Santiago Gonzalez, Risto Miikkulainen

Evolutionary optimization, such as the TaylorGLO method, can be used to discover novel, customized loss functions for deep neural networks, resulting in improved performance, faster training, and improved data utilization.

Creative AI Through Evolutionary Computation: Principles and Examples

no code implementations4 Aug 2020 Risto Miikkulainen

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new.

Generalization of Agent Behavior through Explicit Representation of Context

no code implementations18 Jun 2020 Cem C. Tutum, Suhaib Abdulquddos, Risto Miikkulainen

In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations.

Autonomous Driving

Discovering Parametric Activation Functions

no code implementations5 Jun 2020 Garrett Bingham, Risto Miikkulainen

Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks.

Image Classification

From Prediction to Prescription: Evolutionary Optimization of Non-Pharmaceutical Interventions in the COVID-19 Pandemic

1 code implementation28 May 2020 Risto Miikkulainen, Olivier Francon, Elliot Meyerson, Xin Qiu, Elisa Canzani, Babak Hodjat

Several models have been developed to predict how the COVID-19 pandemic spreads, and how it could be contained with non-pharmaceutical interventions (NPIs) such as social distancing restrictions and school and business closures.

Evolutionary Optimization of Deep Learning Activation Functions

no code implementations17 Feb 2020 Garrett Bingham, William Macke, Risto Miikkulainen

The choice of activation function can have a large effect on the performance of a neural network.

Regularized Evolutionary Population-Based Training

no code implementations11 Feb 2020 Jason Liang, Santiago Gonzalez, Hormoz Shahrzad, Risto Miikkulainen

This paper presents an algorithm called Evolutionary Population-Based Training (EPBT) that interleaves the training of a DNN's weights with the metalearning of loss functions.

Image Classification Knowledge Distillation

Improving Neural Network Learning Through Dual Variable Learning Rates

no code implementations9 Feb 2020 Elizabeth Liner, Risto Miikkulainen

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR).

Optimizing Loss Functions Through Multivariate Taylor Polynomial Parameterization

1 code implementation31 Jan 2020 Santiago Gonzalez, Risto Miikkulainen

Metalearning of deep neural network (DNN) architectures and hyperparameters has become an increasingly important area of research.

Improved Training Speed, Accuracy, and Data Utilization via Loss Function Optimization

no code implementations25 Sep 2019 Santiago Gonzalez, Risto Miikkulainen

As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning.

AutoML Image Classification

Enhanced Optimization with Composite Objectives and Novelty Pulsation

no code implementations7 Jun 2019 Hormoz Shahrzad, Babak Hodjat, Camille Dollé, Andrei Denissov, Simon Lau, Donn Goodhew, Justin Dyer, Risto Miikkulainen

This paper improves this approach further by introducing novelty pulsation, i. e. a systematic method to alternate between novelty selection and local optimization.

Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel

2 code implementations ICLR 2020 Xin Qiu, Elliot Meyerson, Risto Miikkulainen

In many such tasks, the point prediction is not enough: the uncertainty (i. e. risk or confidence) of that prediction must also be estimated.

Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains

1 code implementation NeurIPS 2019 Elliot Meyerson, Risto Miikkulainen

As deep learning applications continue to become more diverse, an interesting question arises: Can general problem solving arise from jointly learning several such diverse tasks?

Multi-Task Learning

Improved Training Speed, Accuracy, and Data Utilization Through Loss Function Optimization

2 code implementations27 May 2019 Santiago Gonzalez, Risto Miikkulainen

As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning.

AutoML Image Classification

Functional Generative Design of Mechanisms with Recurrent Neural Networks and Novelty Search

no code implementations25 Mar 2019 Cameron R. Wolfe, Cem C. Tutum, Risto Miikkulainen

However, while static designs are easily produced with 3D printing, functional designs with moving parts are more difficult to generate: The search space is too high-dimensional, the resolution of the 3D-printed parts is not adequate, and it is difficult to predict the physical behavior of imperfect 3D-printed mechanisms.

Creative AI Through Evolutionary Computation

no code implementations12 Jan 2019 Risto Miikkulainen

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new.

Faster Training by Selecting Samples Using Embeddings

no code implementations27 Sep 2018 Santiago Gonzalez, Joshua Landgraf, Risto Miikkulainen

Long training times have increasingly become a burden for researchers by slowing down the pace of innovation, with some models taking days or weeks to train.

Image Classification

A Comparison of the Taguchi Method and Evolutionary Optimization in Multivariate Testing

no code implementations25 Aug 2018 Jingbo Jiang, Diego Legrand, Robert Severn, Risto Miikkulainen

Its performance is compared to that of the Taguchi method in several simulated conditions, including an orthogonal one designed to favor the Taguchi method, and two realistic conditions with dependences between variables.

Functional Generative Design: An Evolutionary Approach to 3D-Printing

no code implementations19 Apr 2018 Cem C. Tutum, Supawit Chockchowwat, Etienne Vouga, Risto Miikkulainen

The proposed methodology for discovering solutions to this problem consists of three components: First, an effective search space is learned through a variational autoencoder (VAE); second, a surrogate model for functional designs is built; and third, a genetic algorithm is used to simultaneously update the hyperparameters of the surrogate and to optimize the designs using the updated surrogate.

From Nodes to Networks: Evolving Recurrent Neural Networks

no code implementations12 Mar 2018 Aditya Rawal, Risto Miikkulainen

Gated recurrent networks such as those composed of Long Short-Term Memory (LSTM) nodes have recently been used to improve state of the art in many sequential processing tasks such as speech recognition and machine translation.

Language Modelling Machine Translation +3

Enhanced Optimization with Composite Objectives and Novelty Selection

no code implementations10 Mar 2018 Hormoz Shahrzad, Daniel Fink, Risto Miikkulainen

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular.

Enhancing Evolutionary Conversion Rate Optimization via Multi-armed Bandit Algorithms

no code implementations10 Mar 2018 Xin Qiu, Risto Miikkulainen

Traffic is allocated to candidate solutions using a multi-armed bandit algorithm, using more traffic on those evaluations that are most useful.

Evolutionary Architecture Search For Deep Multitask Networks

no code implementations10 Mar 2018 Jason Liang, Elliot Meyerson, Risto Miikkulainen

Multitask learning, i. e. learning several tasks at once with the same neural network, can improve performance in each of the tasks.

Neural Architecture Search

Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering

no code implementations ICLR 2018 Elliot Meyerson, Risto Miikkulainen

Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering.

Discovering Evolutionary Stepping Stones through Behavior Domination

no code implementations18 Apr 2017 Elliot Meyerson, Risto Miikkulainen

The conclusion is that behavior domination can help illuminate the complex dynamics of behavior-driven search, and can thus lead to the design of more scalable and robust algorithms.

Multiobjective Optimization

Conversion Rate Optimization through Evolutionary Computation

no code implementations1 Mar 2017 Risto Miikkulainen, Neil Iscoe, Aaron Shagrin, Ron Cordell, Sam Nazari, Cory Schoolland, Myles Brundage, Jonathan Epstein, Randy Dean, Gurmeet Lamba

Conversion optimization means designing a web interface so that as many users as possible take a desired action on it, such as register or purchase.

Reuse of Neural Modules for General Video Game Playing

no code implementations4 Dec 2015 Alexander Braylan, Mark Hollenbeck, Elliot Meyerson, Risto Miikkulainen

A general approach to knowledge transfer is introduced in which an agent controlled by a neural network adapts how it reuses existing networks as it learns in a new domain.

Atari Games Decision Making +1

Increasing Behavioral Complexity for Evolved Virtual Creatures with the ESP Method

no code implementations27 Oct 2015 Dan Lessin, Don Fussell, Risto Miikkulainen, Sebastian Risi

Since their introduction in 1994 (Sims), evolved virtual creatures (EVCs) have employed the coevolution of morphology and control to produce high-impact work in multiple fields, including graphics, evolutionary computation, robotics, and artificial life.

Artificial Life

Evolving Neural Networks through Augmenting Topologies

3 code implementations Evolutionary Computation 2002 2006 Kenneth O. Stanley, Risto Miikkulainen

An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights.

Neural Architecture Search

Cannot find the paper you are looking for? You can Submit a new open access paper.