Search Results for author:
Found 20 papers, 9 papers with code
Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions
The softmax layer produces the distribution based on the dot products of a single hidden state and the embeddings of words in the vocabulary.
Unsupervised Partial Sentence Matching for Cited Text Identification
Given a citation in the body of a research paper, cited text identification aims to find the sentences in the cited paper that are most relevant to the citing sentence.
REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy
If a LLM's entropy is higher than the asymptotic entropy (i. e., the LLM is more uncertain than it should be), the THF model predicts a high hallucination hazard, which leads to a lower p threshold in REAL sampling.
To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenders
Specifically, the similarity structure of the global item embeddings in the softmax layer sometimes forces the single hidden state embedding to be close to new items when copying is a better choice, while sometimes forcing the hidden state to be close to the items from the input inappropriately.
Encoding Multi-Domain Scientific Papers by Ensembling Multiple CLS Tokens
Many useful tasks on scientific documents, such as topic classification and citation prediction, involve corpora that span multiple scientific domains.
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond
Is the output softmax layer, which is adopted by most language models (LMs), always the best way to compute the next word probability?
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling
Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint.
Augmenting Scientific Creativity with Retrieval across Knowledge Domains
Exposure to ideas in domains outside a scientist's own may benefit her in reformulating existing research problems in novel ways and discovering new application domains for existing solution ideas.
Open Aspect Target Sentiment Classification with Natural Language Prompts
For many business applications, we often seek to analyze sentiments associated with any arbitrary aspects of commercial products, despite having a very limited amount of labels or even without any labels at all.
Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications
Most unsupervised NLP models represent each word with a single point or single region in semantic space, while the existing multi-sense word embeddings cannot represent longer word sequences like phrases or sentences.
Changing the Mind of Transformers for Topically-Controllable Language Generation
Our framework consists of two components: (1) a method that produces a set of candidate topics by predicting the centers of word clusters in the possible continuations, and (2) a text generation model whose output adheres to the chosen topics.
Multi-facet Universal Schema
To address the violation of the USchema assumption, we propose multi-facet universal schema that uses a neural model to represent each sentence pattern as multiple facet embeddings and encourage one of these facet embeddings to be close to that of another sentence pattern if they co-occur with the same entity pair.
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Can one build a knowledge graph (KG) for all products in the world?
Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition
Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks.
The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures
Materials science literature contains millions of materials synthesis procedures described in unstructured natural language text.
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery.
Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings
Word sense induction (WSI), which addresses polysemy by unsupervised discovery of multiple word senses, resolves ambiguities for downstream NLP tasks and also makes word representations more interpretable.
Automatically Extracting Action Graphs from Materials Science Synthesis Procedures
In this work, we present a system for automatically extracting structured representations of synthesis procedures from the texts of materials science journal articles that describe explicit, experimental syntheses of inorganic compounds.
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection
Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, coreference, relation extraction, and question answering.
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy.
