Search Results for author: Tomas Mikolov

Found 39 papers, 26 papers with code

Preserving Semantics in Textual Adversarial Attacks

1 code implementation8 Nov 2022 David Herel, Hugo Cisneros, Tomas Mikolov

The goal of this kind of adversarial attack is to modify the input text to fool a classifier while maintaining the original meaning of the text.

Adversarial Attack Sentence Embedding +1

Benchmarking Learning Efficiency in Deep Reservoir Computing

2 code implementations29 Sep 2022 Hugo Cisneros, Josef Sivic, Tomas Mikolov

In this paper, we introduce a benchmark of increasingly difficult tasks together with a data efficiency metric to measure how quickly machine learning models learn from training data.


Visualizing computation in large-scale cellular automata

no code implementations1 Apr 2021 Hugo Cisneros, Josef Sivic, Tomas Mikolov

Emergent processes in complex systems such as cellular automata can perform computations of increasing complexity, and could possibly lead to artificial evolution.


Emergence of Self-Reproducing Metabolisms as Recursive Algorithms in an Artificial Chemistry

no code implementations15 Mar 2021 Germán Kruszewski, Tomas Mikolov

One of the main goals of Artificial Life is to research the conditions for the emergence of life, not necessarily as it is, but as it could be.

Artificial Life

Classification of Complex Systems Based on Transients

no code implementations31 Aug 2020 Barbora Hudcova, Tomas Mikolov

In order to develop systems capable of modeling artificial life, we need to identify, which systems can produce complex behavior.

Artificial Life Classification +1

Evaluating Online Continual Learning with CALM

1 code implementation7 Apr 2020 Germán Kruszewski, Ionut-Teodor Sorodoc, Tomas Mikolov

Online Continual Learning (OCL) studies learning over a continuous data stream without observing any single example more than once, a setting that is closer to the experience of humans and systems that must learn "on-the-wild".

Continual Learning Language Modelling

Combinatory Chemistry: Towards a Simple Model of Emergent Evolution

1 code implementation17 Mar 2020 Germán Kruszewski, Tomas Mikolov

An explanatory model for the emergence of evolvable units must display emerging structures that (1) preserve themselves in time (2) self-reproduce and (3) tolerate a certain amount of variation when reproducing.

Artificial Life

Evolving Structures in Complex Systems

1 code implementation4 Nov 2019 Hugo Cisneros, Josef Sivic, Tomas Mikolov

In this paper we propose an approach for measuring growth of complexity of emerging patterns in complex systems such as cellular automata.

Artificial Life

Place Deduplication with Embeddings

no code implementations29 Sep 2019 Carl Yang, Do Huy Hoang, Tomas Mikolov, Jiawei Han

Thanks to the advancing mobile location services, people nowadays can post about places to share visiting experience on-the-go.

Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion

4 code implementations EMNLP 2018 Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Herve Jegou, Edouard Grave

Continuous word representations learned separately on distinct languages can be aligned so that their words become comparable in a common space.

regression Retrieval +2

Learning Word Vectors for 157 Languages

2 code implementations LREC 2018 Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, Tomas Mikolov

Distributed word representations, or word vectors, have recently been applied to many tasks in natural language processing, leading to state-of-the-art performance.

Advances in Pre-Training Distributed Word Representations

5 code implementations LREC 2018 Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin

Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl.

Fast Linear Model for Knowledge Graph Embeddings

1 code implementation30 Oct 2017 Armand Joulin, Edouard Grave, Piotr Bojanowski, Maximilian Nickel, Tomas Mikolov

This paper shows that a simple baseline based on a Bag-of-Words (BoW) representation learns surprisingly good knowledge graph embeddings.

General Classification Knowledge Base Completion +2

Learning Simpler Language Models with the Differential State Framework

no code implementations26 Mar 2017 Alexander G. Ororbia II, Tomas Mikolov, David Reitter

The Differential State Framework (DSF) is a simple and high-performing design that unifies previously introduced gated neural models.

Language Modelling

CommAI: Evaluating the first steps towards a useful general AI

no code implementations31 Jan 2017 Marco Baroni, Armand Joulin, Allan Jabri, Germàn Kruszewski, Angeliki Lazaridou, Klemen Simonic, Tomas Mikolov

With machine learning successfully applied to new daunting problems almost every day, general AI starts looking like an attainable goal.

BIG-bench Machine Learning Continual Learning +2 Compressing text classification models

42 code implementations12 Dec 2016 Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, Tomas Mikolov

We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory.

General Classification Quantization +2

Variable Computation in Recurrent Neural Networks

no code implementations18 Nov 2016 Yacine Jernite, Edouard Grave, Armand Joulin, Tomas Mikolov

Recurrent neural networks (RNNs) have been used extensively and with increasing success to model various types of sequential data.

Enriching Word Vectors with Subword Information

49 code implementations TACL 2017 Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov

A vector representation is associated to each character $n$-gram; words being represented as the sum of these representations.

Word Embeddings Word Similarity

A Roadmap towards Machine Intelligence

1 code implementation25 Nov 2015 Tomas Mikolov, Armand Joulin, Marco Baroni

The development of intelligent machines is one of the biggest unsolved challenges in computer science.

Learning Simple Algorithms from Examples

1 code implementation23 Nov 2015 Wojciech Zaremba, Tomas Mikolov, Armand Joulin, Rob Fergus

We present an approach for learning simple algorithms such as copying, multi-digit addition and single digit multiplication directly from examples.


Alternative structures for character-level RNNs

1 code implementation19 Nov 2015 Piotr Bojanowski, Armand Joulin, Tomas Mikolov

The first one consists on conditioning the character level representation on the previous word representation.

Language Modelling

Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets

3 code implementations NeurIPS 2015 Armand Joulin, Tomas Mikolov

Despite the recent achievements in machine learning, we are still very far from achieving real artificial intelligence.

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

19 code implementations19 Feb 2015 Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merriënboer, Armand Joulin, Tomas Mikolov

One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.

Question Answering Reading Comprehension

Learning Longer Memory in Recurrent Neural Networks

5 code implementations24 Dec 2014 Tomas Mikolov, Armand Joulin, Sumit Chopra, Michael Mathieu, Marc'Aurelio Ranzato

In this paper, we show that learning longer term patterns in real data, such as in natural language, is perfectly possible using gradient descent.

Language Modelling

Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews

4 code implementations17 Dec 2014 Grégoire Mesnil, Tomas Mikolov, Marc'Aurelio Ranzato, Yoshua Bengio

Sentiment analysis is a common task in natural language processing that aims to detect polarity of a text document (typically a consumer review).

Binary Classification General Classification +1

Distributed Representations of Sentences and Documents

27 code implementations16 May 2014 Quoc V. Le, Tomas Mikolov

Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models.

Question Answering Sentiment Analysis +1

Zero-Shot Learning by Convex Combination of Semantic Embeddings

2 code implementations19 Dec 2013 Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, Jeffrey Dean

In other cases the semantic embedding space is established by an independent natural language processing task, and then the image transformation into that space is learned in a second stage.

Multi-label zero-shot learning

Distributed Representations of Words and Phrases and their Compositionality

49 code implementations NeurIPS 2013 Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean

Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible.

Exploiting Similarities among Languages for Machine Translation

8 code implementations17 Sep 2013 Tomas Mikolov, Quoc V. Le, Ilya Sutskever

Dictionaries and phrase tables are the basis of modern statistical machine translation systems.

Machine Translation Translation

Efficient Estimation of Word Representations in Vector Space

76 code implementations16 Jan 2013 Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean

We propose two novel model architectures for computing continuous vector representations of words from very large data sets.

Word Similarity

On the difficulty of training Recurrent Neural Networks

2 code implementations21 Nov 2012 Razvan Pascanu, Tomas Mikolov, Yoshua Bengio

There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994).

Cannot find the paper you are looking for? You can Submit a new open access paper.