Search Results for author:
Found 42 papers, 27 papers with code
Collapse of Self-trained Language Models
In various fields of knowledge creation, including science, new ideas often build on pre-existing information.
Large Language Models: A Survey
Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022.
Advancing State of the Art in Language Modeling
In this paper, we propose a simple framework that should help advance the state of the art in language modeling in terms of generalization.
Ranked #7 on
Language Modelling
on WikiText-103
Preserving Semantics in Textual Adversarial Attacks
Our method outperforms existing sentence encoders used in adversarial attacks by achieving 1. 2x - 5. 1x better real attack success rate.
Benchmarking Learning Efficiency in Deep Reservoir Computing
In this paper, we introduce a benchmark of increasingly difficult tasks together with a data efficiency metric to measure how quickly machine learning models learn from training data.
Emergence of Novelty in Evolutionary Algorithms
This leads to an emergence of a novel behavior of the agents.
Visualizing computation in large-scale cellular automata
Emergent processes in complex systems such as cellular automata can perform computations of increasing complexity, and could possibly lead to artificial evolution.
Emergence of Self-Reproducing Metabolisms as Recursive Algorithms in an Artificial Chemistry
One of the main goals of Artificial Life is to research the conditions for the emergence of life, not necessarily as it is, but as it could be.
Classification of Complex Systems Based on Transients
In order to develop systems capable of modeling artificial life, we need to identify, which systems can produce complex behavior.
Evaluating Online Continual Learning with CALM
Online Continual Learning (OCL) studies learning over a continuous data stream without observing any single example more than once, a setting that is closer to the experience of humans and systems that must learn "on-the-wild".
Combinatory Chemistry: Towards a Simple Model of Emergent Evolution
An explanatory model for the emergence of evolvable units must display emerging structures that (1) preserve themselves in time (2) self-reproduce and (3) tolerate a certain amount of variation when reproducing.
Evolving Structures in Complex Systems
In this paper we propose an approach for measuring growth of complexity of emerging patterns in complex systems such as cellular automata.
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
In this paper, we focus on the problem of adapting word vector-based models to new textual data.
Place Deduplication with Embeddings
Thanks to the advancing mobile location services, people nowadays can post about places to share visiting experience on-the-go.
Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
Continuous word representations learned separately on distinct languages can be aligned so that their words become comparable in a common space.
Learning Word Vectors for 157 Languages
Distributed word representations, or word vectors, have recently been applied to many tasks in natural language processing, leading to state-of-the-art performance.
Ranked #12 on
Only Connect Walls Dataset Task 1 (Grouping)
on OCW
(using extra training data)
Advances in Pre-Training Distributed Word Representations
Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl.
Fast Linear Model for Knowledge Graph Embeddings
This paper shows that a simple baseline based on a Bag-of-Words (BoW) representation learns surprisingly good knowledge graph embeddings.
Learning Simpler Language Models with the Differential State Framework
The Differential State Framework (DSF) is a simple and high-performing design that unifies previously introduced gated neural models.
CommAI: Evaluating the first steps towards a useful general AI
With machine learning successfully applied to new daunting problems almost every day, general AI starts looking like an attainable goal.
FastText.zip: Compressing text classification models
We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory.
Variable Computation in Recurrent Neural Networks
Recurrent neural networks (RNNs) have been used extensively and with increasing success to model various types of sequential data.
Enriching Word Vectors with Subword Information
A vector representation is associated to each character $n$-gram; words being represented as the sum of these representations.
Bag of Tricks for Efficient Text Classification
This paper explores a simple and efficient baseline for text classification.
Ranked #1 on
Sentiment Analysis
on Sogou News
Emotion Recognition in Conversation
General Classification
+2
A Roadmap towards Machine Intelligence
The development of intelligent machines is one of the biggest unsolved challenges in computer science.
Learning Simple Algorithms from Examples
We present an approach for learning simple algorithms such as copying, multi-digit addition and single digit multiplication directly from examples.
Alternative structures for character-level RNNs
The first one consists on conditioning the character level representation on the previous word representation.
Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
Despite the recent achievements in machine learning, we are still very far from achieving real artificial intelligence.
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
Learning Longer Memory in Recurrent Neural Networks
In this paper, we show that learning longer term patterns in real data, such as in natural language, is perfectly possible using gradient descent.
Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews
Sentiment analysis is a common task in natural language processing that aims to detect polarity of a text document (typically a consumer review).
Distributed Representations of Sentences and Documents
Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models.
Ranked #4 on
Question Answering
on QASent
Zero-Shot Learning by Convex Combination of Semantic Embeddings
In other cases the semantic embedding space is established by an independent natural language processing task, and then the image transformation into that space is learned in a second stage.
Ranked #8 on
Multi-label zero-shot learning
on Open Images V4
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling
We propose a new benchmark corpus to be used for measuring progress in statistical language modeling.
Ranked #22 on
Language Modelling
on One Billion Word
DeViSE: A Deep Visual-Semantic Embedding Model
Modern visual recognition systems are often limited in their ability to scale to large numbers of object categories.
Ranked #13 on
Zero-Shot Action Recognition
on Kinetics
Distributed Representations of Words and Phrases and their Compositionality
Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible.
Exploiting Similarities among Languages for Machine Translation
Dictionaries and phrase tables are the basis of modern statistical machine translation systems.
Efficient Estimation of Word Representations in Vector Space
We propose two novel model architectures for computing continuous vector representations of words from very large data sets.
On the difficulty of training Recurrent Neural Networks
There are two widely known issues with properly training Recurrent Neural Networks, the vanishing and the exploding gradient problems detailed in Bengio et al. (1994).
