Search Results

Trending Latest Greatest
1
Card image cap
LSTM-based Deep Learning Models for Non-factoid Answer Selection
One direction is to define a more composite representation for questions and answers by combining convolutional neural network with the basic framework. Several variations of models are provided.

2
Card image cap
Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition
Named Entity Recognition (NER) is one of the most common tasks of the natural language processing. The purpose of NER is to find and classify tokens in text documents into predefined categories called tags, such as person names, quantity expressions, percentage expressions, names of locations, organizations, as well as expression of time, currency and others.

3
Card image cap
Regularizing and Optimizing LSTM Language Models
Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine translation, language modeling, and question answering. In this paper, we consider the specific problem of word-level language modeling and investigate strategies for regularizing and optimizing LSTM-based models.
4
Card image cap
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance. However, little is published which parameters and design choices should be evaluated or selected making the correct hyperparameter optimization often a "black art that requires expert experiences" (Snoek et al., 2012).

5
Card image cap
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF.
6
Card image cap
Chinese NER Using Lattice LSTM
We investigate a lattice-structured LSTM model for Chinese NER, which encodes a sequence of input characters as well as all potential words that match a lexicon. Compared with character-based methods, our model explicitly leverages word and word sequence information.

7
Card image cap
Machine Comprehension Using Match-LSTM and Answer Pointer
Machine comprehension of text is an important problem in natural language processing. We propose two ways of using Pointer Net for our task.

8
Card image cap
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance. However, little is published which parameters and design choices should be evaluated or selected making the correct hyperparameter optimization often a "black art that requires expert experiences" (Snoek et al., 2012).

9
Card image cap
Machine Comprehension Using Match-LSTM and Answer Pointer
Machine comprehension of text is an important problem in natural language processing. We propose two ways of using Pointer Net for our task.

10
Card image cap
Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging
In this paper we show that reporting a single performance score is insufficient to compare non-deterministic approaches. We demonstrate for common sequence tagging tasks that the seed value for the random number generator can result in statistically significant (p < 10^-4) differences for state-of-the-art systems.
11
Card image cap
Learning Natural Language Inference with LSTM
Natural language inference (NLI) is a fundamentally important task in natural language processing that has many applications. On the SNLI corpus, our model achieves an accuracy of 86.1%, outperforming the state of the art.

12
Card image cap
TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition
Building upon our experimental results, we then propose and investigate two different networks to further integrate spatiotemporal information: 1) temporal segment RNN and 2) Inception-style Temporal-ConvNet. We demonstrate that using both RNNs (using LSTMs) and Temporal-ConvNets on spatiotemporal feature matrices are able to exploit spatiotemporal dynamics to improve the overall performance.

13
Card image cap
Generating Sequences With Recurrent Neural Networks
This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued).

14
Card image cap
Machine Comprehension Using Match-LSTM and Answer Pointer
Machine comprehension of text is an important problem in natural language processing. We propose two ways of using Pointer Net for our task.

15
Card image cap
Factorization tricks for LSTM networks
We present two simple ways of reducing the number of parameters and accelerating the training of large Long Short-Term Memory (LSTM) networks: the first one is "matrix factorization by design" of LSTM matrix into the product of two smaller matrices, and the second one is partitioning of LSTM matrix, its inputs and states into the independent groups. Both approaches allow us to train large LSTM networks significantly faster to the near state-of the art perplexity while using significantly less RNN parameters.

16
Card image cap
Multivariate LSTM-FCNs for Time Series Classification
Over the past decade, multivariate time series classification has been receiving a lot of attention. We propose augmenting the existing univariate time series classification models, LSTM-FCN and ALSTM-FCN with a squeeze and excitation block to further improve performance.

17
Card image cap
Enhanced LSTM for Natural Language Inference
Reasoning and inference are central to human and artificial intelligence. Modeling inference in human language is very challenging.

18
Card image cap
LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection
Mechanical devices such as engines, vehicles, aircrafts, etc., are typically instrumented with numerous sensors to capture the behavior and health of the machine. We show that EncDec-AD is robust and can detect anomalies from predictable, unpredictable, periodic, aperiodic, and quasi-periodic time-series.

19
LSTM based Conversation Models
In this paper, we present a conversational model that incorporates both context and participant role for two-party conversations. Different architectures are explored for integrating participant role and context information into a Long Short-term Memory (LSTM) language model.

20
Card image cap
Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
In our approach, the encoding of sentence is a two-stage process. Instead of using target sentence to attend words in source sentence, we utilized the sentence's first-stage representation to attend words appeared in itself, which is called "Inner-Attention" in our paper .

21
Card image cap
Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition
Named Entity Recognition (NER) is one of the most common tasks of the natural language processing. The purpose of NER is to find and classify tokens in text documents into predefined categories called tags, such as person names, quantity expressions, percentage expressions, names of locations, organizations, as well as expression of time, currency and others.

22
Card image cap
LSTM Fully Convolutional Networks for Time Series Classification
We propose the augmentation of fully convolutional networks with long short term memory recurrent neural network (LSTM RNN) sub-modules for time series classification. We also explore the usage of attention mechanism to improve time series classification with the Attention Long Short Term Memory Fully Convolutional Network (ALSTM-FCN).

23
Card image cap
Named Entity Recognition with Bidirectional LSTM-CNNs
Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineering and lexicons to achieve high performance. In this paper, we present a novel neural network architecture that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.

24
Card image cap
A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder
The detection of anomalous executions is valuable for reducing potential hazards in assistive manipulation. Multimodal sensory signals can be helpful for detecting a wide range of anomalies.

25
Card image cap
Multivariate LSTM-FCNs for Time Series Classification
Over the past decade, multivariate time series classification has been receiving a lot of attention. We propose augmenting the existing univariate time series classification models, LSTM-FCN and ALSTM-FCN with a squeeze and excitation block to further improve performance.

26
Card image cap
LSTM Fully Convolutional Networks for Time Series Classification
We propose the augmentation of fully convolutional networks with long short term memory recurrent neural network (LSTM RNN) sub-modules for time series classification. We also explore the usage of attention mechanism to improve time series classification with the Attention Long Short Term Memory Fully Convolutional Network (ALSTM-FCN).

27
Card image cap
Bidirectional LSTM-CRF Models for Sequence Tagging
In this paper, we propose a variety of Long Short-Term Memory (LSTM) based models for sequence tagging. It can also use sentence level tag information thanks to a CRF layer.

28
Card image cap
Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Recurrent Neural Networks (RNNs) have become the state-of-the-art choice for extracting patterns from temporal sequences. In this work, we introduce the Phased LSTM model, which extends the LSTM unit by adding a new time gate.
29
Card image cap
Text-based LSTM networks for Automatic Music Composition
In this paper, we introduce new methods and discuss results of text-based LSTM (Long Short-Term Memory) networks for automatic music composition. The proposed network is designed to learn relationships within text documents that represent chord progressions and drum tracks in two case studies.

30
Card image cap
Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Recurrent Neural Networks (RNNs) have become the state-of-the-art choice for extracting patterns from temporal sequences. In this work, we introduce the Phased LSTM model, which extends the LSTM unit by adding a new time gate.
31
Card image cap
Text-based LSTM networks for Automatic Music Composition
In this paper, we introduce new methods and discuss results of text-based LSTM (Long Short-Term Memory) networks for automatic music composition. The proposed network is designed to learn relationships within text documents that represent chord progressions and drum tracks in two case studies.

32
Card image cap
Bidirectional LSTM-CRF for Clinical Concept Extraction
Extraction of concepts present in patient clinical records is an essential step in clinical research. The 2010 i2b2/VA Workshop on Natural Language Processing Challenges for clinical records presented concept extraction (CE) task, with aim to identify concepts (such as treatments, tests, problems) and classify them into predefined categories.

33
Card image cap
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF.
34
Card image cap
Bidirectional LSTM-CRF for Clinical Concept Extraction
Automated extraction of concepts from patient clinical records is an essential facilitator of clinical research. For this reason, the 2010 i2b2/VA Natural Language Processing Challenges for Clinical Records introduced a concept extraction task aimed at identifying and classifying concepts into predefined categories (i.e., treatments, tests and problems).

35
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Also, most NMT systems have difficulty with rare words. These issues have hindered NMT's use in practical deployments and services, where both accuracy and speed are essential.

36
Actionable and Political Text Classification using Word Embeddings and LSTM
The first is that of Actionability, where we build models to classify social media messages from customers of service providers as Actionable or Non-Actionable. We build models for over 30 different languages for actionability, and most of the models achieve accuracy around 85%, with some reaching over 90% accuracy.

37
Card image cap
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network architecture that benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM, CNN and CRF.
38
Card image cap
LSTM Benchmarks for Deep Learning Frameworks
This study provides benchmarks for different implementations of LSTM units between the deep learning frameworks PyTorch, TensorFlow, Lasagne and Keras. The comparison includes cuDNN LSTMs, fused LSTM variants and less optimized, but more flexible LSTM implementations.

39
Card image cap
Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model
Data-driven saliency has recently gained a lot of attention thanks to the use of Convolutional Neural Networks for predicting gaze fixations. In this paper we go beyond standard approaches to saliency prediction, in which gaze maps are computed with a feed-forward network, and present a novel model which can predict accurate saliency maps by incorporating neural attentive mechanisms.

40
Card image cap
Named Entity Recognition with Bidirectional LSTM-CNNs
Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineering and lexicons to achieve high performance. In this paper, we present a novel neural network architecture that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.

41
Card image cap
Sequence-Level Knowledge Distillation
We demonstrate that standard knowledge distillation applied to word-level prediction can be effective for NMT, and also introduce two novel sequence-level versions of knowledge distillation that further improve performance, and somewhat surprisingly, seem to eliminate the need for beam search (even when applied on the original teacher model). Applying weight pruning on top of knowledge distillation results in a student model that has 13 times fewer parameters than the original teacher model, with a decrease of 0.4

42
Card image cap
Learning Natural Language Inference with LSTM
Natural language inference (NLI) is a fundamentally important task in natural language processing that has many applications. On the SNLI corpus, our model achieves an accuracy of 86.1%, outperforming the state of the art.

43
Card image cap
Training Very Deep Networks
Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success. However, training becomes more difficult as depth increases, and training of very deep networks remains an open problem.
44
Card image cap
Towards Binary-Valued Gates for Robust LSTM Training
Long Short-Term Memory (LSTM) is one of the most widely used recurrent structures in sequence modeling. It aims to use gates to control information flow (e.g., whether to skip some information or not) in the recurrent computations, although its practical implementation based on soft gates only partially achieves this goal.
45
Card image cap
Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTM
In this paper, we propose a novel DNN-based video saliency prediction method. We further find from our database that there exists a temporal correlation of human attention with a smooth saliency transition across video frames.