Search Results for author:
Found 68 papers, 27 papers with code
Representing Unordered Data Using Multiset Automata and Complex Numbers
Unordered, variable-sized inputs arise in many settings across multiple fields.
Improving Rare Word Translation With Dictionaries and Attention Masking
In machine translation, rare words continue to be a problem for the dominant encoder-decoder architecture, especially in low-resource and out-of-domain translation settings.
Language Complexity and Speech Recognition Accuracy: Orthographic Complexity Hurts, Phonological Complexity Doesn't
We investigate what linguistic factors affect the performance of Automatic Speech Recognition (ASR) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
PILA: A Historical-Linguistic Dataset of Proto-Italic and Latin
To assist historical linguists in the study of Italic sound change, we introduce the Proto-Italic to Latin (PILA) dataset, which consists of roughly 3, 000 pairs of forms from Proto-Italic and Latin.
Killkan: The Automatic Speech Recognition Dataset for Kichwa with Morphosyntactic Information
This dataset, the ASR model, and the code used to develop them will be publicly available.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Nostra Domina at EvaLatin 2024: Improving Latin Polarity Detection through Data Augmentation
This paper describes submissions from the team Nostra Domina to the EvaLatin 2024 shared task of emotion polarity detection.
We're Calling an Intervention: Exploring the Fundamental Hurdles in Adapting Language Models to Nonstandard Text
We present a suite of experiments that allow us to understand the underlying challenges of language model adaptation to nonstandard text.
Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers
Deriving formal bounds on the expressivity of transformers, as well as studying transformers that are constructed to implement known algorithms, are both effective methods for better understanding the computational power of transformers.
Transformers as Transducers
We study the sequence-to-sequence mapping capacity of transformers by relating them to finite transducers, and find that they can express surprisingly large classes of transductions.
DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages
This allows for a comprehensive evaluation of NLP system performance on different language varieties.
Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines
Rhetoric, both spoken and written, involves not only content but also style.
What Formal Languages Can Transformers Express? A Survey
As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as formal languages.
BERTwich: Extending BERT's Capabilities to Model Dialectal and Noisy Text
Real-world NLP applications often deal with nonstandard text (e. g., dialectal, informal, or misspelled text).
Efficient Algorithms for Recognizing Weighted Tree-Adjoining Languages
From these, we also immediately obtain stringsum and allsum algorithms for TAG, LIG, PAA, and EPDA.
Masked Hard-Attention Transformers Recognize Exactly the Star-Free Languages
The expressive power of transformers over inputs of unbounded size can be studied through their ability to recognize classes of formal languages.
Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns
Attention, specifically scaled dot-product attention, has proven effective for natural language, but it does not have a mechanism for handling hierarchical patterns of arbitrary nesting depth, which limits its ability to recognize certain syntactic structures.
Universal Automatic Phonetic Transcription into the International Phonetic Alphabet
This paper presents a state-of-the-art model for transcribing speech in any language into the International Phonetic Alphabet (IPA).
Convergence and Diversity in the Control Hierarchy
Furthermore, using an even stricter notion of equivalence called d-strong equivalence, we make precise the intuition that a CFG controlling a CFG is a TAG, a PDA controlling a PDA is an embedded PDA, and a PDA controlling a CFG is a LIG.
Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to Dialects and Closely-Related Languages
In this work, we induce character-level noise in various forms when fine-tuning BERT to enable zero-shot cross-lingual transfer to unseen dialects and languages.
Tighter Bounds on the Expressivity of Transformer Encoders
Characterizing neural networks in terms of better-understood formal systems has the potential to yield new insights into the power and limitations of these networks.
Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata
A current goal in the graph neural network literature is to enable transformers to operate on graph-structured data, given their success on language and vision tasks.
A Continuum of Generation Tasks for Investigating Length Bias and Degenerate Repetition
Language models suffer from various degenerate behaviors.
Algorithms for Weighted Pushdown Automata
Weighted pushdown automata (WPDAs) are at the core of many natural language processing tasks, like syntax-based statistical machine translation and transition-based dependency parsing.
The Surprising Computational Power of Nondeterministic Stack RNNs
Second, it can recognize languages with much larger alphabet sizes than one might expect given the size of its stack alphabet.
Overcoming a Theoretical Limitation of Self-Attention
We examine this limitation using two languages: PARITY, the language of bit strings with an odd number of 1s, and FIRST, the language of bit strings starting with a 1.
Learning Hierarchical Structures with Differentiable Nondeterministic Stacks
Learning hierarchical structures in sequential data -- from simple algorithmic patterns to natural language -- in a reliable, generalizable way remains a challenging problem for neural language models.
Syntax-Based Attention Masking for Neural Machine Translation
We present a simple method for extending transformers to source-side trees.
Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution
In this paper, we investigate the driving factors behind concatenation, a simple but effective data augmentation method for low-resource neural machine translation.
Named Tensor Notation
We propose a notation for tensors with named axes, which relieves the author, reader, and future implementers of machine learning models from the burden of keeping track of the order of axes and the purpose of each.
Translating Recursive Probabilistic Programs to Factor Graph Grammars
It is natural for probabilistic programs to use conditionals to express alternative substructures in models, and loops (recursion) to express repeated substructures in models.
Factor Graph Grammars
We propose the use of hyperedge replacement graph grammars for factor graphs, or factor graph grammars (FGGs) for short.
Look It Up: Bilingual Dictionaries Improve Neural Machine Translation
Despite advances in neural machine translation (NMT) quality, rare words continue to be problematic.
Learning Context-Free Languages with Nondeterministic Stack RNNs
We present a differentiable stack data structure that simultaneously and tractably encodes an exponential number of stack configurations, based on Lang's algorithm for simulating nondeterministic pushdown automata.
Representing Unordered Data Using Complex-Weighted Multiset Automata
Unordered, variable-sized inputs arise in many settings across multiple fields.
Efficiency through Auto-Sizing: Notre Dame NLP's Submission to the WNGT 2019 Efficiency Task
We investigated the impact of auto-sizing (Murray and Chiang, 2015; Murray et al., 2019) to the Transformer network (Vaswani et al., 2017) with the goal of substantially reducing the number of parameters in the model.
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation
Neural sequence-to-sequence models, particularly the Transformer, are the state of the art in machine translation.
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units
Operations using sparse structures are common in natural language models at the input and output layers, because these models operate on sequences over discrete alphabets.
Measuring Human Perception to Improve Handwritten Document Transcription
The subtleties of human perception, as measured by vision scientists through the use of psychophysics, are important clues to the internal workings of visual recognition.
Correcting Length Bias in Neural Machine Translation
We study two problems in neural machine translation (NMT).
Neural Machine Translation of Text from Non-Native Speakers
Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data, especially when the system is trained only on clean data.
Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource
Most work on part-of-speech (POS) tagging is focused on high resource languages, or examines low-resource and active learning settings through simulated studies.
Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention
Natural language sentences, being hierarchical, can be represented at different levels of granularity, like words, subwords, or characters.
Composing Finite State Transducers on GPUs
Weighted finite-state transducers (FSTs) are frequently used in language processing to handle tasks such as part-of-speech tagging and speech recognition.
Leveraging translations for speech transcription in low-resource settings
Recently proposed data collection frameworks for endangered language documentation aim not only to collect speech in the language of interest, but also to collect translations into a high-resource language that will render the collected resource interpretable.
Weighted DAG Automata for Semantic Graphs
Graphs have a variety of uses in natural language processing, particularly as representations of linguistic meaning.
Tied Multitask Learning for Neural Speech Translation
We explore multitask models for neural translation of speech, augmenting them in order to reflect two intuitive notions.
Improving Lexical Choice in Neural Machine Translation
We explore two solutions to the problem of mistranslating rare words in neural machine translation.
Spoken Term Discovery for Language Documentation using Translations
Vast amounts of speech data collected for language documentation and research remain untranscribed and unsearchable, but often a small amount of speech may have text translations available.
Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation
We present a simple method to improve neural translation of a low-resource language pair using parallel data from a related, also low-resource, language pair.
Top-Rank Enhanced Listwise Optimization for Statistical Machine Translation
We propose a listwise learning framework for structure prediction problems such as machine translation.
Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information.
A case study on using speech-to-translation alignments for language documentation
For many low-resource or endangered languages, spoken language resources are more likely to be annotated with translations than with transcriptions.
DyNet: The Dynamic Neural Network Toolkit
In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives.
Decoding with Finite-State Transducers on GPUs
Weighted finite automata and transducers (including hidden Markov models and conditional random fields) are widely used in natural language processing (NLP) to perform tasks such as morphological analysis, part-of-speech tagging, chunking, named entity recognition, speech recognition, and others.
An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languages
For many low-resource languages, spoken language resources are more likely to be annotated with translations than with transcriptions.
Growing Graphs with Hyperedge Replacement Graph Grammars
In experiments on large real world networks, we show that random graphs, generated from extracted graph grammars, exhibit a wide range of properties that are very similar to the original graphs.
Auto-Sizing Neural Networks: With Applications to n-gram Language Models
Neural networks have been shown to improve performance across a range of natural-language tasks.
