Search Results for author: Jürgen Schmidhuber

Found 98 papers, 62 papers with code

Language Agents as Optimizable Graphs

1 code implementation26 Feb 2024 Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, Jürgen Schmidhuber

Various human-designed prompt engineering techniques have been proposed to improve problem solvers based on Large Language Models (LLMs), yielding many disparate code bases.

Prompt Engineering

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention

1 code implementation13 Dec 2023 Róbert Csordás, Piotr Piękos, Kazuki Irie, Jürgen Schmidhuber

The costly self-attention layers in modern Transformers require memory and compute quadratic in sequence length.

Language Modelling

Automating Continual Learning

1 code implementation1 Dec 2023 Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

General-purpose learning systems should improve themselves in open-ended fashion in ever-changing environments.

Continual Learning Image Classification +2

Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotation

no code implementations14 Nov 2023 Lukas Tuggener, Thilo Stadelmann, Jürgen Schmidhuber

Humans and animals recognize objects irrespective of the beholder's point of view, which may drastically change their appearances.

Data Augmentation Semantic Segmentation

Unsupervised Musical Object Discovery from Audio

1 code implementation13 Nov 2023 Joonsu Gha, Vincent Herrmann, Benjamin Grewe, Jürgen Schmidhuber, Anand Gopalakrishnan

Our novel MusicSlots method adapts SlotAttention to the audio domain, to achieve unsupervised music decomposition.

Object Object Discovery +1

Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions

1 code implementation24 Oct 2023 Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

Recent studies of the computational power of recurrent neural networks (RNNs) reveal a hierarchy of RNN architectures, given real-time and finite-precision assumptions.

Approximating Two-Layer Feedforward Networks for Efficient Transformers

2 code implementations16 Oct 2023 Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber

Unlike prior work that compares MoEs with dense baselines under the compute-equal condition, our evaluation condition is parameter-equal, which is crucial to properly evaluate LMs.

Learning to Identify Critical States for Reinforcement Learning from Videos

1 code implementation ICCV 2023 Haozhe Liu, Mingchen Zhuge, Bing Li, Yuhui Wang, Francesco Faccio, Bernard Ghanem, Jürgen Schmidhuber

Recent work on deep reinforcement learning (DRL) has pointed out that algorithmic information about good policies can be extracted from offline data which lack explicit information about executed actions.

reinforcement-learning

Exploring the Promise and Limits of Real-Time Recurrent Learning

1 code implementation30 May 2023 Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber

To scale to such challenging tasks, we focus on certain well-known neural architectures with element-wise recurrence, allowing for tractable RTRL without approximation.

Large Language Model Programs

no code implementations9 May 2023 Imanol Schlag, Sainbayar Sukhbaatar, Asli Celikyilmaz, Wen-tau Yih, Jason Weston, Jürgen Schmidhuber, Xian Li

In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples.

Language Modelling Large Language Model +1

Accelerating Neural Self-Improvement via Bootstrapping

1 code implementation2 May 2023 Kazuki Irie, Jürgen Schmidhuber

Few-shot learning with sequence-processing neural networks (NNs) has recently attracted a new wave of attention in the context of large language models.

Few-Shot Learning

Topological Neural Discrete Representation Learning à la Kohonen

1 code implementation15 Feb 2023 Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

Unsupervised learning of discrete representations from continuous ones in neural networks (NNs) is the cornerstone of several applications today.

Representation Learning

Guiding Online Reinforcement Learning with Action-Free Offline Pretraining

1 code implementation30 Jan 2023 Deyao Zhu, Yuhui Wang, Jürgen Schmidhuber, Mohamed Elhoseiny

In this paper, we investigate the potential of using action-free offline datasets to improve online reinforcement learning, name this problem Reinforcement Learning with Action-Free Offline Pretraining (AFP-RL).

Offline RL reinforcement-learning +1

Eliminating Meta Optimization Through Self-Referential Meta Learning

no code implementations29 Dec 2022 Louis Kirsch, Jürgen Schmidhuber

We discuss the relationship of such systems to in-context and memory-based meta learning and show that self-referential neural networks require functionality to be reused in the form of parameter sharing.

Meta-Learning

Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networks

no code implementations29 Dec 2022 Vincent Herrmann, Louis Kirsch, Jürgen Schmidhuber

There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions.

On Narrative Information and the Distillation of Stories

1 code implementation22 Nov 2022 Dylan R. Ashley, Vincent Herrmann, Zachary Friggstad, Jürgen Schmidhuber

We then demonstrate how evolutionary algorithms can leverage this to extract a set of narrative templates and how these templates -- in tandem with a novel curve-fitting algorithm we introduce -- can reorder music albums to automatically induce stories in them.

Contrastive Learning Evolutionary Algorithms

Exploring through Random Curiosity with General Value Functions

1 code implementation18 Nov 2022 Aditya Ramesh, Louis Kirsch, Sjoerd van Steenkiste, Jürgen Schmidhuber

Furthermore, RC-GVF significantly outperforms previous methods in the absence of ground-truth episodic counts in the partially observable MiniGrid environments.

Efficient Exploration

Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks

no code implementations17 Nov 2022 Kazuki Irie, Jürgen Schmidhuber

Short-term memory in standard, general-purpose, sequence-processing recurrent neural networks (RNNs) is stored as activations of nodes or "neurons."

The Benefits of Model-Based Generalization in Reinforcement Learning

1 code implementation4 Nov 2022 Kenny Young, Aditya Ramesh, Louis Kirsch, Jürgen Schmidhuber

First, we provide a simple theorem motivating how learning a model as an intermediate step can narrow down the set of possible value functions more than learning a value function directly from data using the Bellman equation.

Model-based Reinforcement Learning reinforcement-learning +1

CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations

1 code implementation12 Oct 2022 Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber

While the original CTL is used to test length generalization or productivity, CTL++ is designed to test systematicity of NNs, that is, their capability to generalize to unseen compositions of known functions.

Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules

1 code implementation7 Oct 2022 Kazuki Irie, Jürgen Schmidhuber

Work on fast weight programmers has demonstrated the effectiveness of key/value outer product-based learning rules for sequentially generating a weight matrix (WM) of a neural net (NN) by another NN or itself.

Denoising Image Generation +1

Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter

1 code implementation5 Aug 2022 Aleksandar Stanić, Yujin Tang, David Ha, Jürgen Schmidhuber

We show that current agents struggle to generalize, and introduce novel object-centric agents that improve over strong baselines.

Meta-Learning

General Policy Evaluation and Improvement by Learning to Identify Few But Crucial States

1 code implementation4 Jul 2022 Francesco Faccio, Aditya Ramesh, Vincent Herrmann, Jean Harb, Jürgen Schmidhuber

In continuous control problems with infinitely many states, our value function minimizes its prediction error by simultaneously learning a small set of `probing states' and a mapping from actions produced in probing states to the policy's return.

Continuous Control Reinforcement Learning (RL) +1

Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules

2 code implementations3 Jun 2022 Kazuki Irie, Francesco Faccio, Jürgen Schmidhuber

Neural ordinary differential equations (ODEs) have attracted much attention as continuous-time counterparts of deep residual neural networks (NNs), and numerous extensions for recurrent NNs have been proposed.

Time Series Time Series Analysis +1

Upside-Down Reinforcement Learning Can Diverge in Stochastic Environments With Episodic Resets

1 code implementation13 May 2022 Miroslav Štrupl, Francesco Faccio, Dylan R. Ashley, Jürgen Schmidhuber, Rupesh Kumar Srivastava

Upside-Down Reinforcement Learning (UDRL) is an approach for solving RL problems that does not require value functions and uses only supervised learning, where the targets for given inputs in a dataset do not change over time.

reinforcement-learning Reinforcement Learning (RL)

Unsupervised Learning of Temporal Abstractions with Slot-based Transformers

1 code implementation25 Mar 2022 Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, Sjoerd van Steenkiste

The discovery of reusable sub-routines simplifies decision-making and planning in complex reinforcement learning problems.

Decision Making

All You Need Is Supervised Learning: From Imitation Learning to Meta-RL With Upside Down RL

1 code implementation24 Feb 2022 Kai Arulkumaran, Dylan R. Ashley, Jürgen Schmidhuber, Rupesh K. Srivastava

Upside down reinforcement learning (UDRL) flips the conventional use of the return in the objective function in RL upside down, by taking returns as input and predicting actions.

Imitation Learning Offline RL +2

The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention

1 code implementation11 Feb 2022 Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber

Linear layers in neural networks (NNs) trained by gradient descent can be expressed as a key-value memory system which stores all training datapoints and the initial weights, and produces outputs using unnormalised dot attention over the entire training experience.

Continual Learning Image Classification +2

Improving Baselines in the Wild

1 code implementation31 Dec 2021 Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

We share our experience with the recently released WILDS benchmark, a collection of ten datasets dedicated to developing models and training strategies which are robust to domain shifts.

Automatic Embedding of Stories Into Collections of Independent Media

1 code implementation3 Nov 2021 Dylan R. Ashley, Vincent Herrmann, Zachary Friggstad, Kory W. Mathewson, Jürgen Schmidhuber

We look at how machine learning techniques that derive properties of items in a collection of independent media can be used to automatically embed stories into such collections.

TAG

The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization

1 code implementation14 Oct 2021 Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber

Despite progress across a broad range of applications, Transformers have limited success in systematic generalization.

ListOps Systematic Generalization

Reward-Weighted Regression Converges to a Global Optimum

1 code implementation19 Jul 2021 Miroslav Štrupl, Francesco Faccio, Dylan R. Ashley, Rupesh Kumar Srivastava, Jürgen Schmidhuber

Reward-Weighted Regression (RWR) belongs to a family of widely known iterative Reinforcement Learning algorithms based on the Expectation-Maximization framework.

regression Reinforcement Learning (RL)

Bayesian brains and the Rényi divergence

no code implementations12 Jul 2021 Noor Sajid, Francesco Faccio, Lancelot Da Costa, Thomas Parr, Jürgen Schmidhuber, Karl Friston

Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters.

Bayesian Inference Variational Inference

Going Beyond Linear Transformers with Recurrent Fast Weight Programmers

5 code implementations NeurIPS 2021 Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

Transformers with linearised attention (''linear Transformers'') have demonstrated the practical scalability and effectiveness of outer product-based Fast Weight Programmers (FWPs) from the '90s.

Atari Games ListOps

Is it enough to optimize CNN architectures on ImageNet?

1 code implementation16 Mar 2021 Lukas Tuggener, Jürgen Schmidhuber, Thilo Stadelmann

In this work we challenge the notion that CNN architecture design solely based on ImageNet leads to generally effective convolutional neural network (CNN) architectures that perform well on a diverse set of datasets and application domains.

Image Classification

Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling

1 code implementation ICLR 2021 Đorđe Miladinović, Aleksandar Stanić, Stefan Bauer, Jürgen Schmidhuber, Joachim M. Buhmann

We show that augmenting the decoder of a hierarchical VAE by spatial dependency layers considerably improves density estimation over baseline convolutional architectures and the state-of-the-art among the models within the same class.

Density Estimation

Linear Transformers Are Secretly Fast Weight Programmers

9 code implementations22 Feb 2021 Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber

We show the formal equivalence of linearised self-attention mechanisms and fast weight controllers from the early '90s, where a ``slow" neural net learns by gradient descent to program the ``fast weights" of another net through sequences of elementary programming instructions which are additive outer products of self-invented activation patterns (today called keys and values).

Language Modelling Machine Translation +2

Meta Learning Backpropagation And Improving It

no code implementations NeurIPS 2021 Louis Kirsch, Jürgen Schmidhuber

Many concepts have been proposed for meta learning with neural networks (NNs), e. g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs.

Meta-Learning

On the Binding Problem in Artificial Neural Networks

no code implementations9 Dec 2020 Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber

Contemporary neural networks still fall short of human-level generalization, which extends far beyond our direct experiences.

Hierarchical Relational Inference

no code implementations7 Oct 2020 Aleksandar Stanić, Sjoerd van Steenkiste, Jürgen Schmidhuber

Common-sense physical reasoning in the real world requires learning about the interactions of objects and their dynamics.

Common Sense Reasoning

Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks

1 code implementation ICLR 2021 Róbert Csordás, Sjoerd van Steenkiste, Jürgen Schmidhuber

Neural networks (NNs) whose subnetworks implement reusable functions are expected to offer numerous advantages, including compositionality through efficient recombination of functional building blocks, interpretability, preventing catastrophic interference, etc.

Systematic Generalization

Recurrent Neural-Linear Posterior Sampling for Nonstationary Contextual Bandits

1 code implementation9 Jul 2020 Aditya Ramesh, Paulo Rauber, Michelangelo Conserva, Jürgen Schmidhuber

An agent in a nonstationary contextual bandit problem should balance between exploration and the exploitation of (periodic or structured) patterns present in its previous experiences.

Multi-Armed Bandits

Parameter-Based Value Functions

1 code implementation ICLR 2021 Francesco Faccio, Louis Kirsch, Jürgen Schmidhuber

We introduce a class of value functions called Parameter-Based Value Functions (PBVFs) whose inputs include the policy parameters.

Continuous Control Reinforcement Learning (RL)

Meta-Learning for Recalibration of EMG-Based Upper Limb Prostheses

no code implementations ICML Workshop LifelongML 2020 Krsto Proroković, Michael Wand, Jürgen Schmidhuber

An EMG-based upper limb prosthesis relies on a statistical pattern recognition system to map the EMG signal of residual forearm muscles into the appropriate hand movements.

Meta-Learning

Training Agents using Upside-Down Reinforcement Learning

7 code implementations5 Dec 2019 Rupesh Kumar Srivastava, Pranav Shyam, Filipe Mutz, Wojciech Jaśkowski, Jürgen Schmidhuber

Many of its general principles are outlined in a companion report; the goal of this paper is to develop a practical learning algorithm and show that this conceptually simple perspective on agent training can produce a range of rewarding behaviors for multiple episodic environments.

reinforcement-learning Reinforcement Learning (RL)

Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving

3 code implementations15 Oct 2019 Imanol Schlag, Paul Smolensky, Roland Fernandez, Nebojsa Jojic, Jürgen Schmidhuber, Jianfeng Gao

We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure.

Math Question Answering

R-SQAIR: Relational Sequential Attend, Infer, Repeat

no code implementations11 Oct 2019 Aleksandar Stanić, Jürgen Schmidhuber

Traditional sequential multi-object attention models rely on a recurrent mechanism to infer object relations.

Inductive Bias Object

Recurrent Neural Processes

2 code implementations13 Jun 2019 Timon Willi, Jonathan Masci, Jürgen Schmidhuber, Christian Osendorfer

We extend Neural Processes (NPs) to sequential data through Recurrent NPs or RNPs, a family of conditional state space models.

Gaussian Processes Time Series +1

A Perspective on Objects and Systematic Generalization in Model-Based RL

no code implementations3 Jun 2019 Sjoerd van Steenkiste, Klaus Greff, Jürgen Schmidhuber

In order to meet the diverse challenges in solving many real-world problems, an intelligent agent has to be able to dynamically construct a model of its environment.

Systematic Generalization

Learning to Reason with Third Order Tensor Products

1 code implementation NeurIPS 2018 Imanol Schlag, Jürgen Schmidhuber

We combine Recurrent Neural Networks with Tensor Product Representations to learn combinatorial representations of sequential data.

Test

Learning to Reason with Third-Order Tensor Products

1 code implementation29 Nov 2018 Imanol Schlag, Jürgen Schmidhuber

We combine Recurrent Neural Networks with Tensor Product Representations to learn combinatorial representations of sequential data.

Test

Investigating Object Compositionality in Generative Adversarial Networks

no code implementations ICLR 2019 Sjoerd van Steenkiste, Karol Kurach, Jürgen Schmidhuber, Sylvain Gelly

We present a minimal modification of a standard generator to incorporate this inductive bias and find that it reliably learns to generate images as compositions of objects.

Image Generation Inductive Bias +5

Recurrent World Models Facilitate Policy Evolution

no code implementations NeurIPS 2018 David Ha, Jürgen Schmidhuber

A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations.

reinforcement-learning Reinforcement Learning (RL)

DeepScores -- A Dataset for Segmentation, Detection and Classification of Tiny Objects

2 code implementations27 Mar 2018 Lukas Tuggener, Ismail Elezi, Jürgen Schmidhuber, Marcello Pelillo, Thilo Stadelmann

We present the DeepScores dataset with the goal of advancing the state-of-the-art in small objects recognition, and by placing the question of object recognition in the context of scene understanding.

General Classification Object +3

World Models

21 code implementations27 Mar 2018 David Ha, Jürgen Schmidhuber

We explore building generative neural network models of popular reinforcement learning environments.

Car Racing reinforcement-learning +1

GATED FAST WEIGHTS FOR ASSOCIATIVE RETRIEVAL

no code implementations ICLR 2018 Imanol Schlag, Jürgen Schmidhuber

We improve previous end-to-end differentiable neural networks (NNs) with fast weight memories.

Retrieval

Neural Expectation Maximization

1 code implementation NeurIPS 2017 Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber

Many real world tasks such as reasoning and physical interaction require identification and manipulation of conceptual entities.

Clustering

Highway and Residual Networks learn Unrolled Iterative Estimation

no code implementations22 Dec 2016 Klaus Greff, Rupesh K. Srivastava, Jürgen Schmidhuber

We demonstrate that this viewpoint directly leads to the construction of Highway and Residual networks.

Recurrent Highway Networks

5 code implementations ICML 2017 Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, Jürgen Schmidhuber

We introduce a novel theoretical analysis of recurrent networks based on Gersgorin's circle theorem that illuminates several modeling and optimization issues and improves our understanding of the LSTM cell.

Language Modelling

Tagger: Deep Unsupervised Perceptual Grouping

2 code implementations NeurIPS 2016 Klaus Greff, Antti Rasmus, Mathias Berglund, Tele Hotloo Hao, Jürgen Schmidhuber, Harri Valpola

We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features.

General Classification Segmentation

Lipreading with Long Short-Term Memory

no code implementations29 Jan 2016 Michael Wand, Jan Koutník, Jürgen Schmidhuber

Lipreading, i. e. speech recognition from visual-only recordings of a speaker's face, can be achieved with a processing pipeline based solely on neural networks, yielding significantly better accuracy than conventional methods.

Lipreading speech-recognition +1

Binding via Reconstruction Clustering

1 code implementation19 Nov 2015 Klaus Greff, Rupesh Kumar Srivastava, Jürgen Schmidhuber

Disentangled distributed representations of data are desirable for machine learning, since they are more expressive and can generalize from fewer examples.

Clustering Denoising +1

Training Very Deep Networks

3 code implementations NeurIPS 2015 Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber

Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success.

Image Classification

Highway Networks

3 code implementations3 May 2015 Rupesh Kumar Srivastava, Klaus Greff, Jürgen Schmidhuber

There is plenty of theoretical and empirical evidence that depth of neural networks is a crucial ingredient for their success.

Language Modelling

LSTM: A Search Space Odyssey

15 code implementations13 Mar 2015 Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R. Steunebrink, Jürgen Schmidhuber

Several variants of the Long Short-Term Memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995.

Handwriting Recognition Music Modeling +1

Understanding Locally Competitive Networks

no code implementations5 Oct 2014 Rupesh Kumar Srivastava, Jonathan Masci, Faustino Gomez, Jürgen Schmidhuber

Recently proposed neural network activation functions such as rectified linear, maxout, and local winner-take-all have allowed for faster and more effective training of deep neural architectures on large and complex datasets.

Retrieval

A Clockwork RNN

5 code implementations14 Feb 2014 Jan Koutník, Klaus Greff, Faustino Gomez, Jürgen Schmidhuber

Sequence prediction and classification are ubiquitous and challenging problems in machine learning that can require identifying complex dependencies between temporally distant inputs.

General Classification

My First Deep Learning System of 1991 + Deep Learning Timeline 1962-2013

no code implementations19 Dec 2013 Jürgen Schmidhuber

Deep Learning has attracted significant attention in recent years.

Compete to Compute

no code implementations NeurIPS 2013 Rupesh K. Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, Jürgen Schmidhuber

Local competition among neighboring neurons is common in biological neural networks (NNs).

Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification

no code implementations1 Sep 2013 Dan Cireşan, Jürgen Schmidhuber

Our Multi-Column Deep Neural Networks achieve best known recognition rates on Chinese characters from the ICDAR 2011 and 2013 offline handwriting competitions, approaching human performance.

General Classification

Measuring Intelligence through Games

no code implementations6 Sep 2011 Tom Schaul, Julian Togelius, Jürgen Schmidhuber

Artificial general intelligence (AGI) refers to research aimed at tackling the full problem of artificial intelligence, that is, create truly intelligent agents.

Motion Planning

Natural Evolution Strategies

1 code implementation22 Jun 2011 Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jürgen Schmidhuber

This paper presents Natural Evolution Strategies (NES), a recent family of algorithms that constitute a more principled approach to black-box optimization than established evolutionary algorithms.

Evolutionary Algorithms

Improving the Asymptotic Performance of Markov Chain Monte-Carlo by Inserting Vortices

no code implementations NeurIPS 2010 Yi Sun, Jürgen Schmidhuber, Faustino J. Gomez

We present a new way of converting a reversible finite Markov chain into a nonreversible one, with a theoretical guarantee that the asymptotic variance of the MCMC estimator based on the non-reversible chain is reduced.

Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks

no code implementations NeurIPS 2008 Alex Graves, Jürgen Schmidhuber

Offline handwriting recognition---the transcription of images of handwritten text---is an interesting task, in that it combines computer vision with sequence learning.

Handwriting Recognition

Unconstrained On-line Handwriting Recognition with Recurrent Neural Networks

no code implementations NeurIPS 2007 Alex Graves, Marcus Liwicki, Horst Bunke, Jürgen Schmidhuber, Santiago Fernández

On-line handwriting recognition is unusual among sequence labelling tasks in that the underlying generator of the observed data, i. e. the movement of the pen, is recorded directly.

Handwriting Recognition Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.