Search Results for author: Risto Miikkulainen

Found 45 papers, 10 papers with code

Characterizing Dynamic Word Meaning Representations in the Brain

no code implementations COLING (CogALex) 2020 Nora Aguirre-Celis, Risto Miikkulainen

During sentence comprehension, humans adjust word meanings according to the combination of the concepts that occur in the sentence.

NeuroComb: Improving SAT Solving with Graph Neural Networks

no code implementations26 Oct 2021 Wenxi Wang, Yang Hu, Mohit Tiwari, Sarfraz Khurshid, Kenneth McMillan, Risto Miikkulainen

NeuroComb is therefore a practical approach to improving SAT solving through modern machine learning.

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks

1 code implementation18 Sep 2021 Garrett Bingham, Risto Miikkulainen

By analytically tracking the mean and variance of signals as they propagate through the network, AutoInit is able to appropriately scale the weights at each layer to avoid exploding or vanishing signals.

Meta-Learning Neural Architecture Search +1

Evolving GAN Formulations for Higher Quality Image Synthesis

no code implementations17 Feb 2021 Santiago Gonzalez, Mohak Kant, Risto Miikkulainen

Generative Adversarial Networks (GANs) have extended deep learning to complex generation and translation tasks across different data modalities.

Image-to-Image Translation Translation

Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model

no code implementations5 Oct 2020 Xin Qiu, Risto Miikkulainen

This framework, RED, builds an error detector on top of the base classifier and estimates uncertainty of the detection scores using Gaussian Processes.

Gaussian Processes

The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings

no code implementations ICLR 2021 Elliot Meyerson, Risto Miikkulainen

This paper frames a general prediction system as an observer traveling around a continuous space, measuring values at some locations, and predicting them at others.

Multi-Task Learning

Effective Regularization Through Loss-Function Metalearning

no code implementations2 Oct 2020 Santiago Gonzalez, Risto Miikkulainen

Evolutionary optimization, such as the TaylorGLO method, can be used to discover novel, customized loss functions for deep neural networks, resulting in improved performance, faster training, and improved data utilization.

Creative AI Through Evolutionary Computation: Principles and Examples

no code implementations4 Aug 2020 Risto Miikkulainen

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new.

Generalization of Agent Behavior through Explicit Representation of Context

no code implementations18 Jun 2020 Cem C. Tutum, Suhaib Abdulquddos, Risto Miikkulainen

In order to deploy autonomous agents in digital interactive environments, they must be able to act robustly in unseen situations.

Autonomous Driving

Discovering Parametric Activation Functions

no code implementations5 Jun 2020 Garrett Bingham, Risto Miikkulainen

Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks.

Image Classification

From Prediction to Prescription: Evolutionary Optimization of Non-Pharmaceutical Interventions in the COVID-19 Pandemic

1 code implementation28 May 2020 Risto Miikkulainen, Olivier Francon, Elliot Meyerson, Xin Qiu, Elisa Canzani, Babak Hodjat

Several models have been developed to predict how the COVID-19 pandemic spreads, and how it could be contained with non-pharmaceutical interventions (NPIs) such as social distancing restrictions and school and business closures.

Evolutionary Optimization of Deep Learning Activation Functions

no code implementations17 Feb 2020 Garrett Bingham, William Macke, Risto Miikkulainen

The choice of activation function can have a large effect on the performance of a neural network.

Effective Reinforcement Learning through Evolutionary Surrogate-Assisted Prescription

1 code implementation13 Feb 2020 Olivier Francon, Santiago Gonzalez, Babak Hodjat, Elliot Meyerson, Risto Miikkulainen, Xin Qiu, Hormoz Shahrzad

Using this data, it is possible to learn a surrogate model, and with that model, evolve a decision strategy that optimizes the outcomes.

Decision Making

Regularized Evolutionary Population-Based Training

no code implementations11 Feb 2020 Jason Liang, Santiago Gonzalez, Hormoz Shahrzad, Risto Miikkulainen

This paper presents an algorithm called Evolutionary Population-Based Training (EPBT) that interleaves the training of a DNN's weights with the metalearning of loss functions.

Knowledge Distillation

MDEA: Malware Detection with Evolutionary Adversarial Learning

no code implementations9 Feb 2020 Xiruo Wang, Risto Miikkulainen

Malware detection have used machine learning to detect malware in programs.

Malware Detection

Improving Neural Network Learning Through Dual Variable Learning Rates

no code implementations9 Feb 2020 Elizabeth Liner, Risto Miikkulainen

This paper introduces and evaluates a novel training method for neural networks: Dual Variable Learning Rates (DVLR).

Optimizing Loss Functions Through Multivariate Taylor Polynomial Parameterization

1 code implementation31 Jan 2020 Santiago Gonzalez, Risto Miikkulainen

Metalearning of deep neural network (DNN) architectures and hyperparameters has become an increasingly important area of research.

Improved Training Speed, Accuracy, and Data Utilization via Loss Function Optimization

no code implementations25 Sep 2019 Santiago Gonzalez, Risto Miikkulainen

As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning.

AutoML Image Classification

Enhanced Optimization with Composite Objectives and Novelty Pulsation

no code implementations7 Jun 2019 Hormoz Shahrzad, Babak Hodjat, Camille Dollé, Andrei Denissov, Simon Lau, Donn Goodhew, Justin Dyer, Risto Miikkulainen

This paper improves this approach further by introducing novelty pulsation, i. e. a systematic method to alternate between novelty selection and local optimization.

Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel

1 code implementation ICLR 2020 Xin Qiu, Elliot Meyerson, Risto Miikkulainen

In many such tasks, the point prediction is not enough: the uncertainty (i. e. risk or confidence) of that prediction must also be estimated.

Modular Universal Reparameterization: Deep Multi-task Learning Across Diverse Domains

1 code implementation NeurIPS 2019 Elliot Meyerson, Risto Miikkulainen

As deep learning applications continue to become more diverse, an interesting question arises: Can general problem solving arise from jointly learning several such diverse tasks?

Multi-Task Learning

Improved Training Speed, Accuracy, and Data Utilization Through Loss Function Optimization

2 code implementations27 May 2019 Santiago Gonzalez, Risto Miikkulainen

As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning.

AutoML Image Classification

Functional Generative Design of Mechanisms with Recurrent Neural Networks and Novelty Search

no code implementations25 Mar 2019 Cameron R. Wolfe, Cem C. Tutum, Risto Miikkulainen

However, while static designs are easily produced with 3D printing, functional designs with moving parts are more difficult to generate: The search space is too high-dimensional, the resolution of the 3D-printed parts is not adequate, and it is difficult to predict the physical behavior of imperfect 3D-printed mechanisms.

Creative AI Through Evolutionary Computation

no code implementations12 Jan 2019 Risto Miikkulainen

The main power of artificial intelligence is not in modeling what we already know, but in creating solutions that are new.

Faster Training by Selecting Samples Using Embeddings

no code implementations27 Sep 2018 Santiago Gonzalez, Joshua Landgraf, Risto Miikkulainen

Long training times have increasingly become a burden for researchers by slowing down the pace of innovation, with some models taking days or weeks to train.

Image Classification

A Comparison of the Taguchi Method and Evolutionary Optimization in Multivariate Testing

no code implementations25 Aug 2018 Jingbo Jiang, Diego Legrand, Robert Severn, Risto Miikkulainen

Its performance is compared to that of the Taguchi method in several simulated conditions, including an orthogonal one designed to favor the Taguchi method, and two realistic conditions with dependences between variables.

Functional Generative Design: An Evolutionary Approach to 3D-Printing

no code implementations19 Apr 2018 Cem C. Tutum, Supawit Chockchowwat, Etienne Vouga, Risto Miikkulainen

The proposed methodology for discovering solutions to this problem consists of three components: First, an effective search space is learned through a variational autoencoder (VAE); second, a surrogate model for functional designs is built; and third, a genetic algorithm is used to simultaneously update the hyperparameters of the surrogate and to optimize the designs using the updated surrogate.

From Nodes to Networks: Evolving Recurrent Neural Networks

no code implementations12 Mar 2018 Aditya Rawal, Risto Miikkulainen

Gated recurrent networks such as those composed of Long Short-Term Memory (LSTM) nodes have recently been used to improve state of the art in many sequential processing tasks such as speech recognition and machine translation.

Language Modelling Machine Translation +2

Evolutionary Architecture Search For Deep Multitask Networks

no code implementations10 Mar 2018 Jason Liang, Elliot Meyerson, Risto Miikkulainen

Multitask learning, i. e. learning several tasks at once with the same neural network, can improve performance in each of the tasks.

Neural Architecture Search

Enhancing Evolutionary Conversion Rate Optimization via Multi-armed Bandit Algorithms

no code implementations10 Mar 2018 Xin Qiu, Risto Miikkulainen

Traffic is allocated to candidate solutions using a multi-armed bandit algorithm, using more traffic on those evaluations that are most useful.

Enhanced Optimization with Composite Objectives and Novelty Selection

no code implementations10 Mar 2018 Hormoz Shahrzad, Daniel Fink, Risto Miikkulainen

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular.

Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering

no code implementations ICLR 2018 Elliot Meyerson, Risto Miikkulainen

Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering.

Discovering Evolutionary Stepping Stones through Behavior Domination

no code implementations18 Apr 2017 Elliot Meyerson, Risto Miikkulainen

The conclusion is that behavior domination can help illuminate the complex dynamics of behavior-driven search, and can thus lead to the design of more scalable and robust algorithms.

Multiobjective Optimization

Conversion Rate Optimization through Evolutionary Computation

no code implementations1 Mar 2017 Risto Miikkulainen, Neil Iscoe, Aaron Shagrin, Ron Cordell, Sam Nazari, Cory Schoolland, Myles Brundage, Jonathan Epstein, Randy Dean, Gurmeet Lamba

Conversion optimization means designing a web interface so that as many users as possible take a desired action on it, such as register or purchase.

Reuse of Neural Modules for General Video Game Playing

no code implementations4 Dec 2015 Alexander Braylan, Mark Hollenbeck, Elliot Meyerson, Risto Miikkulainen

A general approach to knowledge transfer is introduced in which an agent controlled by a neural network adapts how it reuses existing networks as it learns in a new domain.

Atari Games Decision Making +1

Increasing Behavioral Complexity for Evolved Virtual Creatures with the ESP Method

no code implementations27 Oct 2015 Dan Lessin, Don Fussell, Risto Miikkulainen, Sebastian Risi

Since their introduction in 1994 (Sims), evolved virtual creatures (EVCs) have employed the coevolution of morphology and control to produce high-impact work in multiple fields, including graphics, evolutionary computation, robotics, and artificial life.

Artificial Life

Evolving Neural Networks through Augmenting Topologies

3 code implementations Evolutionary Computation 2002 2006 Kenneth O. Stanley, Risto Miikkulainen

An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights.

Neural Architecture Search

Cannot find the paper you are looking for? You can Submit a new open access paper.