Search Results for author: Luke Vilnis

Found 21 papers, 8 papers with code

Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models

no code implementations18 Oct 2022 Luke Vilnis, Yury Zemlyanskiy, Patrick Murray, Alexandre Passos, Sumit Sanghai

Decoding methods for large language models often trade-off between diversity of outputs and parallelism of computation.

Language Modelling Machine Translation

Capacity and Bias of Learned Geometric Embeddings for Directed Graphs

1 code implementation NeurIPS 2021 Michael Boratko, Dongxu Zhang, Nicholas Monath, Luke Vilnis, Kenneth Clarkson, Andrew McCallum

While vectors in Euclidean space can theoretically represent any graph, much recent work shows that alternatives such as complex, hyperbolic, order, or box embeddings have geometric properties better suited to modeling real-world graphs.

Knowledge Base Completion Multi-Label Classification

Improving Local Identifiability in Probabilistic Box Embeddings

1 code implementation NeurIPS 2020 Shib Sankar Dasgupta, Michael Boratko, Dongxu Zhang, Luke Vilnis, Xiang Lorraine Li, Andrew McCallum

Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment.

Representing Joint Hierarchies with Box Embeddings

1 code implementation AKBC 2020 Dhruvesh Patel, Shib Sankar Dasgupta, Michael Boratko, Xiang Li, Luke Vilnis, Andrew McCallum

Box Embeddings [Vilnis et al., 2018, Li et al., 2019] represent concepts with hyperrectangles in $n$-dimensional space and are shown to be capable of modeling tree-like structures efficiently by training on a large subset of the transitive closure of the WordNet hypernym graph.

Smoothing the Geometry of Probabilistic Box Embeddings

no code implementations ICLR 2019 Xiang Li, Luke Vilnis, Dongxu Zhang, Michael Boratko, Andrew McCallum

However, the hard edges of the boxes present difficulties for standard gradient based optimization; that work employed a special surrogate function for the disjoint case, but we find this method to be fragile.

Inductive Bias

Embedded-State Latent Conditional Random Fields for Sequence Labeling

no code implementations CONLL 2018 Dung Thai, Sree Harsha Ramesh, Shikhar Murty, Luke Vilnis, Andrew McCallum

Complex textual information extraction tasks are often posed as sequence labeling or \emph{shallow parsing}, where fields are extracted using local labels made consistent through probabilistic inference in a graphical model with constrained transitions.

Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking

2 code implementations ACL 2018 Shikhar Murty*, Patrick Verga*, Luke Vilnis, Irena Radovanovic, Andrew McCallum

Extraction from raw text to a knowledge base of entities and fine-grained types is often cast as prediction into a flat set of entity and type labels, neglecting the rich hierarchies over types and entities contained in curated ontologies.

Entity Linking Entity Typing

Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures

no code implementations ACL 2018 Luke Vilnis, Xiang Li, Shikhar Murty, Andrew McCallum

Embedding methods which enforce a partial order or lattice structure over the concept space, such as Order Embeddings (OE) (Vendrov et al., 2016), are a natural way to model transitive relational data (e. g. entailment graphs).

Inductive Bias Knowledge Graphs

Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning

6 code implementations ICLR 2018 Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, Andrew McCallum

Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information.


Finer Grained Entity Typing with TypeNet

no code implementations15 Nov 2017 Shikhar Murty, Patrick Verga, Luke Vilnis, Andrew McCallum

We consider the challenging problem of entity typing over an extremely fine grained set of types, wherein a single mention or entity can have many simultaneous and often hierarchically-structured types.

Entity Typing

Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection

no code implementations NAACL 2018 Haw-Shiuan Chang, ZiYun Wang, Luke Vilnis, Andrew McCallum

Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, coreference, relation extraction, and question answering.

Hypernym Discovery Question Answering +1

Low-Rank Hidden State Embeddings for Viterbi Sequence Labeling

no code implementations2 Aug 2017 Dung Thai, Shikhar Murty, Trapit Bansal, Luke Vilnis, David Belanger, Andrew McCallum

In textual information extraction and other sequence labeling tasks it is now common to use recurrent neural networks (such as LSTM) to form rich embedded representations of long-term input co-occurrence patterns.

named-entity-recognition Named Entity Recognition

Improved Representation Learning for Predicting Commonsense Ontologies

no code implementations1 Aug 2017 Xiang Li, Luke Vilnis, Andrew McCallum

Recent work in learning ontologies (hierarchical and partially-ordered structures) has leveraged the intrinsic geometry of spaces of learned representations to make predictions that automatically obey complex structural constraints.

Representation Learning

Adding Gradient Noise Improves Learning for Very Deep Networks

4 code implementations21 Nov 2015 Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens

This success is partially attributed to architectural innovations such as convolutional and long short-term memory networks.

Question Answering

Generating Sentences from a Continuous Space

14 code implementations CONLL 2016 Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Jozefowicz, Samy Bengio

The standard recurrent neural network language model (RNNLM) generates sentences one word at a time and does not work from an explicit global sentence representation.

Language Modelling

Learning Dynamic Feature Selection for Fast Sequential Prediction

no code implementations IJCNLP 2015 Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.

named-entity-recognition NER +3

Bethe Projections for Non-Local Inference

no code implementations4 Mar 2015 Luke Vilnis, David Belanger, Daniel Sheldon, Andrew McCallum

Many inference problems in structured prediction are naturally solved by augmenting a tractable dependency structure with complex, non-local auxiliary objectives.

Handwriting Recognition Structured Prediction +1

Word Representations via Gaussian Embedding

1 code implementation20 Dec 2014 Luke Vilnis, Andrew McCallum

Current work in lexical distributed representations maps each word to a point vector in low-dimensional space.

Training for Fast Sequential Prediction Using Dynamic Feature Selection

no code implementations30 Oct 2014 Emma Strubell, Luke Vilnis, Andrew McCallum

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.

Part-Of-Speech Tagging

Cannot find the paper you are looking for? You can Submit a new open access paper.