Search Results for author:
Found 16 papers, 3 papers with code
Bit Cipher -- A Simple yet Powerful Word Representation System that Integrates Efficiently with Language Models
While Large Language Models (LLMs) become ever more dominant, classic pre-trained word embeddings sustain their relevance through computational efficiency and nuanced linguistic interpretation.
Explicit Foundation Model Optimization with Self-Attentive Feed-Forward Neural Units
We will discuss a general result about feed-forward neural networks and then extend this solution to compositional (mult-layer) networks, which are applied to a simplified transformer block containing feed-forward and self-attention layers.
Reducing the Need for Backpropagation and Discovering Better Optima With Explicit Optimizations of Neural Networks
Iterative differential approximation methods that rely upon backpropagation have enabled the optimization of neural networks; however, at present, they remain computationally expensive, especially when training models at scale.
EigenNoise: A Contrastive Prior to Warm-Start Representations
In this work, we present a naive initialization scheme for word vectors based on a dense, independent co-occurrence model and provide preliminary results that suggest it is competitive and warrants further investigation.
To Know by the Company Words Keep and What Else Lies in the Vicinity
However, we use the solution to demonstrate a seemingly-universal existence of a property that word vectors exhibit and which allows for the prophylactic discernment of biases in data -- prior to their absorption by DL models.
A general solution to the preferential selection model
We provide a general analytic solution to Herbert Simon's 1955 model for time-evolving novelty functions.
A Computational Framework for Multi-Modal Social Action Identification
We demonstrate how these methods can be used diagnostically-by researchers, government officials and the public-to understand peaceful and violent collective action at very fine-grained levels of time and geography.
Is space a word, too?
We offer a resolution to these issues by exhibiting how the dark matter of word segmentation, i. e., space, punctuation, etc., connect the Zipf-Mandelbrot law to Simon's mechanistic process.
Boundary-based MWE segmentation with text partitioning
This work presents a fine-grained, text-chunking algorithm designed for the task of multiword expressions (MWEs) segmentation.
Zipf's law is a consequence of coherent language production
The task of text segmentation may be undertaken at many levels in text analysis---paragraphs, sentences, words, or even letters.
Benchmarking sentiment analysis methods for large-scale texts: A case for using continuum-scored words and word shift graphs
The emergence and global adoption of social media has rendered possible the real-time estimation of population-scale sentiment, bearing profound implications for our understanding of human behavior.
Sifting Robotic from Organic Text: A Natural Language Approach for Detecting Automation on Twitter
Twitter, a popular social media outlet, has evolved into a vast source of linguistic data, rich with opinion, sentiment, and discussion.
Identifying missing dictionary entries with frequency-conserving context models
With our predictions we then engage the editorial community of the Wiktionary and propose short lists of potential missing entries for definition, developing a breakthrough, lexical extraction technique, and expanding our knowledge of the defined English lexicon of phrases.
Text mixing shapes the anatomy of rank-frequency distributions: A modern Zipfian mechanics for natural language
Natural languages are full of rules and exceptions.
Zipf's law holds for phrases, not words
With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in scaling.
Human language reveals a universal positivity bias
Using human evaluation of 100, 000 words spread across 24 corpora in 10 languages diverse in origin and culture, we present evidence of a deep imprint of human sociality in language, observing that (1) the words of natural human language possess a universal positivity bias; (2) the estimated emotional content of words is consistent between languages under translation; and (3) this positivity bias is strongly independent of frequency of word usage.
