Search Results for author:
Found 35 papers, 15 papers with code
Implicit Chain of Thought Reasoning via Knowledge Distillation
In this work, we explore an alternative reasoning approach: instead of explicitly producing the chain of thought reasoning steps, we use the language model's internal hidden states to perform implicit reasoning.
Differentiable Tree Operations Promote Compositional Generalization
To facilitate the learning of these symbolic sequences, we introduce a differentiable tree interpreter that compiles high-level symbolic tree operations into subsymbolic matrix operations on tensors.
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by composing known parts.
Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages
Machine translation has seen rapid progress with the advent of Transformer-based models.
Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems
What explains the dramatic progress from 20th-century to 21st-century AI, and how can the remaining limitations of current AI be overcome?
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
We apply these analyses to four neural language models (an LSTM, a Transformer, Transformer-XL, and GPT-2).
Distributed neural encoding of binding to thematic roles
A framework and method are proposed for the study of constituent composition in fMRI.
Scalable knowledge base completion with superposition memories
We present Harmonic Memory Networks (HMem), a neural architecture for knowledge base completion that models entities as weighted sums of pairwise bindings between an entity's neighbors and corresponding relations.
Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization
On several syntactic and semantic probing tasks, we demonstrate the emergent structural information in the role vectors and improved syntactic interpretability in the TPR layer outputs.
Compositional Processing Emerges in Neural Networks Solving Math Problems
A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition.
Invertible Tree Embeddings using a Cryptographic Role Embedding Scheme
We present a novel method for embedding trees in a vector space based on Tensor-Product Representations (TPRs) which allows for inversion: the retrieval of the original tree structure and nodes from the vectorial embedding.
Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language
In this paper, we propose a new model architecture for learning multi-modal neuro-symbolic representations for video captioning.
Universal linguistic inductive biases via meta-learning
To facilitate computational modeling aimed at addressing this question, we introduce a framework for giving particular linguistic inductive biases to a neural network model; such a model can then be used to empirically explore the effects of those inductive biases.
HUBERT Untangles BERT to Improve Transfer across NLP Tasks
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model.
Discovering the Compositional Structure of Vector Representations with Role Learning Networks
How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations?
Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving
We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure.
Ranked #1 on
Question Answering
on Mathematics Dataset
Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
The encoder of TP-N2F employs TPR `binding' to encode natural-language symbolic structure in vector space and the decoder uses TPR `unbinding' to generate, in symbolic space, a sequential program represented by relational tuples, each consisting of a relation (or operation) and a number of arguments.
Natural- to formal-language generation using Tensor Product Representations
Generating formal-language represented by relational tuples, such as Lisp programs or mathematical expressions, from a natural-language input is an extremely challenging task because it requires to explicitly capture discrete symbolic structural information from the input to generate the output.
RNNs implicitly implement tensor-product representations
Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regularities (analogies).
RNNs Implicitly Implement Tensor Product Representations
Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regularities (analogies).
Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations
Neural models of Knowledge Base data have typically employed compositional representations of graph objects: entity and relation embeddings are systematically combined to evaluate the truth of a candidate Knowedge Base entry.
Ranked #1 on
Knowledge Graphs
on FB15k
A Simple Recurrent Unit with Reduced Tensor Product Representations
idely used recurrent units, including Long-short Term Memory (LSTM) and the Gated Recurrent Unit (GRU), perform well on natural language tasks, but their ability to learn structured representations is still questionable.
Predicting the Argumenthood of English Prepositional Phrases
Distinguishing between arguments and adjuncts of a verb is a longstanding, nontrivial problem.
Learning and analyzing vector encoding of symbolic representations
We present a formal language with expressions denoting general symbol structures and queries which access information in those structures.
Discrete symbolic optimization and Boltzmann sampling by continuous neural dynamics: Gradient Symbolic Computation
Gradient Symbolic Computation is proposed as a means of solving discrete global optimization problems using a neurally plausible continuous stochastic dynamical system.
A Neural-Symbolic Approach to Design of CAPTCHA
To address this, this paper promotes image/visual captioning based CAPTCHAs, which is robust against machine-learning-based attacks.
Tensor Product Generation Networks for Deep NLP Modeling
We present a new approach to the design of deep networks for natural language processing (NLP), based on the general technique of Tensor Product Representations (TPRs) for encoding and processing symbol structures in distributed neural networks.
Question-Answering with Grammatically-Interpretable Representations
In our application of TPRN, internal representations learned by end-to-end optimization in a deep neural network performing a textual question-answering (QA) task can be interpreted using basic concepts from linguistic theory.
Basic Reasoning with Tensor Product Representations
In this paper we present the initial development of a general theory for mapping inference in predicate logic to computation over Tensor Product Representations (TPRs; Smolensky (1990), Smolensky & Legendre (2006)).
Reasoning in Vector Space: An Exploratory Study of Question Answering
Question answering tasks have shown remarkable progress with distributed vector representation.
