Search Results for author:
Found 46 papers, 19 papers with code
On Target Representation in Continuous-output Neural Machine Translation
We explore pretrained embeddings and also introduce knowledge transfer from the discrete Transformer model using embeddings in Euclidean and non-Euclidean spaces.
Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens
We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation.
DAG Learning on the Permutahedron
We propose a continuous optimization framework for discovering a latent directed acyclic graph (DAG) from observational data.
Discrete Latent Structure in Neural Networks
Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees, sequences, or matchings.
How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?
While it is often assumed that by using BPE, NMT systems are capable of handling OOV words, the effectiveness of BPE in translating OOV words has not been explicitly measured.
Modeling Structure with Undirected Neural Networks
In this paper, we combine the representational strengths of factor graphs and of neural networks, proposing undirected neural networks (UNNs): a flexible framework for specifying computations that can be performed in any order.
Sparse Communication via Mixed Distributions
Neural networks and other machine learning models compute continuous representations, while humans communicate mostly through discrete symbols.
Sparse Continuous Distributions and Fenchel-Young Losses
In contrast, for finite domains, recent work on sparse alternatives to softmax (e. g., sparsemax, $\alpha$-entmax, and fusedmax), has led to distributions with varying support.
Learning Binary Decision Trees by Argmin Differentiation
We address the problem of learning binary decision trees that partition data for some downstream task.
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning
Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously uncovering linguistic insights about the data.
Learning Binary Trees via Sparse Relaxation
In this work we derive a novel sparse relaxation for binary tree learning.
Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
In this paper, we propose a new training strategy which replaces these estimators by an exact yet efficient marginalization.
Sparse and Continuous Attention Mechanisms
Exponential families are widely used in machine learning; they include many distributions in continuous and discrete domains (e. g., Gaussian, Dirichlet, Poisson, and categorical distributions via the softmax transformation).
Ranked #36 on
Visual Question Answering (VQA)
on VQA v2 test-std
Sparse and Structured Visual Attention
Visual attention mechanisms are widely used in multimodal tasks, as visual question answering (VQA).
LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction
Structured prediction requires manipulating a large number of combinatorial structures, e. g., dependency trees or alignments, either as latent or output variables.
Adaptively Sparse Transformers
Findings of the quantitative and qualitative analysis of our approach include that heads in different layers learn different sparsity preferences and tend to be more diverse in their attention distributions than softmax Transformers.
Ranked #1 on
Machine Translation
on IWSLT2017 German-English
Notes on Latent Structure Models and SPIGOT
These notes aim to shed light on the recently proposed structured projected intermediate gradient optimization technique (SPIGOT, Peng et al., 2018).
Latent Structure Models for Natural Language Processing
Latent structure models are a powerful tool for modeling compositional data, discovering linguistic structure, and building NLP pipelines.
Learning with Fenchel-Young Losses
Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally structured prediction.
Interpretable Structure Induction via Sparse Attention
Neural network methods are experiencing wide adoption in NLP, thanks to their empirical performance on many tasks.
Towards Dynamic Computation Graphs via Sparse Latent Structure
Deep NLP models benefit from underlying structures in the data---e. g., parse trees---typically extracted using off-the-shelf parsers.
Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function.
SparseMAP: Differentiable Sparse Structured Inference
Structured prediction requires searching over a combinatorial number of structures.
A Regularized Framework for Sparse and Structured Neural Attention
Modern neural networks are often augmented with an attention mechanism, which tells the network where to focus within the input.
Multi-output Polynomial Networks and Factorization Machines
On recommendation system tasks, we show how to combine our algorithm with a reduction from ordinal regression to multi-output classification and show that the resulting algorithm outperforms simple baselines in terms of ranking accuracy.
Argument Mining with Structured SVMs and RNNs
We propose a novel factor graph model for argument mining, designed for settings in which the argumentative relations in a document do not necessarily form a tree structure.
Conversational Markers of Constructive Discussions
We exploit conversational patterns reflecting the flow of ideas and the balance between the participants, as well as their linguistic choices.
Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions
Changing someone's opinion is arguably one of the most important challenges of social interaction.
Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game
Interpersonal relations are fickle, with close friendships often dissolving into enmity.
QUOTUS: The Structure of Political Media Coverage as Revealed by Quoting Patterns
By encoding bias patterns in a low-rank space we provide an analysis of the structure of political media coverage.
Using a machine learning model to assess the complexity of stress systems
We address the task of stress prediction as a sequence tagging problem.
API design for machine learning software: experiences from the scikit-learn project
Scikit-learn is an increasingly popular machine learning li- brary.
The Romanian Neuter Examined Through A Two-Gender N-Gram Classification System
A recent analysis of the Romanian gender system described in (Bateman and Polinsky, 2010), based on older observations, argues that there are two lexically unspecified noun classes in the singular and two different ones in the plural and that what is generally called neuter in Romanian shares the class in the singular with masculines, and the class in the plural with feminines based not only on agreement features but also on form.
