Search Results for author:
Found 43 papers, 23 papers with code
CIAug: Equipping Interpolative Augmentation with Curriculum Learning
CIAug achieves state-of-the-art results over existing interpolative augmentation methods on 10 benchmark datasets across 4 languages in text classification and named-entity recognition tasks.
Probing Subphonemes in Morphology Models
Transformers have achieved state-of-the-art performance in morphological inflection tasks, yet their ability to generalize across languages and morphological rules remains limited.
Splintering Nonconcatenative Languages for Better Tokenization
Common subword tokenization algorithms like BPE and UnigramLM assume that text can be split into meaningful units by concatenative measures alone.
Token-Level Privacy in Large Language Models
We evaluate dchi-stencil using state-of-the-art language models and diverse datasets, achieving comparable and even better trade-off between utility and privacy compared to existing methods.
How Much is Enough? The Diminishing Returns of Tokenization Training Data
For Russian text, we observe diminishing returns after training a tokenizer from 200GB of data, which is approximately 33% more than when training on English.
Information Types in Product Reviews
Information in text is communicated in a way that supports a goal for its reader.
Don't Touch My Diacritics
The common practice of preprocessing text before feeding it into NLP models introduces many decision points which have unintended consequences on model performance.
OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation
Manual parallelization of code remains a significant challenge due to the complexities of modern software systems and the widespread adoption of multi-core architectures.
Protecting Privacy in Classifiers by Token Manipulation
Using language models as a remote service entails sending private information to an untrusted provider.
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge
Our empirical findings show that the accuracy of UniMorph Labeller is 98%, and that, in all language models studied (including ALBERT, BERT, RoBERTa, and DeBERTa), alien tokenization leads to poorer generalizations compared to morphological tokenization for semantic compositionality of word meanings.
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
We explore threshold vocabulary trimming in Byte-Pair Encoding subword tokenization, a postprocessing step that replaces rare subwords with their component subwords.
BiVert: Bidirectional Vocabulary Evaluation using Relations for Machine Translation
Evaluation of this task is crucial to determine the quality of the translation.
Greed is All You Need: An Evaluation of Tokenizer Inference Methods
While subword tokenizers such as BPE and WordPiece are typically used to build vocabularies for NLP models, the method of decoding text into a sequence of tokens from these vocabularies is often left unspecified, or ill-suited to the method in which they were constructed.
Tokenization Is More Than Compression
Tokenization is a foundational step in natural language processing (NLP) tasks, bridging raw text and language models.
MPIrigen: MPI Code Generation through Domain-Specific Language Models
This study first investigates the performance of state-of-the-art language models in generating MPI-based parallel programs.
MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
Specifically, we start with HPC as a domain and build an HPC-specific LM, named MonoCoder, which is orders of magnitude smaller than existing LMs but delivers better performance on non-HPC and HPC codes.
Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies
Our paper is the first to link LLM misgendering to tokenization and deficient neopronoun grammar, indicating that LLMs unable to correctly treat neopronouns as pronouns are more prone to misgender.
Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark
We introduce Universal NER (UNER), an open, community-driven project to develop gold-standard NER benchmarks in many languages.
Ranked #1 on
Named Entity Recognition (NER)
on UNER v1 (Danish)
Analyzing Cognitive Plausibility of Subword Tokenization
Subword tokenization has become the de-facto standard for tokenization, although comparative evaluations of subword vocabulary quality across languages are scarce.
Emptying the Ocean with a Spoon: Should We Edit Models?
We call into question the recently popularized method of direct model editing as a means of correcting factual errors in LLM generations.
Scope is all you need: Transforming LLMs for HPC Code
With easier access to powerful compute resources, there is a growing trend in the field of AI for software development to develop larger and larger language models (LLMs) to address a variety of programming tasks.
Advising OpenMP Parallelization via a Graph-Based Approach with Transformers
Specifically, we propose a novel approach, called OMPify, to detect and predict the OpenMP pragmas and shared-memory attributes in parallel code, given its serial version.
MPI-rical: Data-Driven MPI Distributed Parallelism Assistance with Transformers
Message Passing Interface (MPI) plays a crucial role in distributed memory parallelization across multiple nodes.
Incorporating Context into Subword Vocabularies
Most current popular subword tokenizers are trained based on word frequency statistics over a corpus, without considering information about co-occurrence or context.
Lost in Space Marking
We look at a decision taken early in training a subword tokenizer, namely whether it should be the word-initial token that carries a special mark, or the word-final one.
UniMorph 4.0: Universal Morphology
The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema.
Learning to Parallelize in a Shared-Memory Environment with Transformers
As a result, there is a growing need to utilize these architectures by introducing shared memory parallelization schemes to software applications.
Integrating Approaches to Word Representation
The problem of representing the atomic elements of language in modern neural learning systems is one of the central challenges of the field of natural language processing.
Learning to Look Inside: Augmenting Token-Based Encoders with Character-Level Information
Commonly-used transformer language models depend on a tokenization schema which sets an unchangeable subword vocabulary prior to pre-training, destined to be applied to all downstream tasks regardless of domain shift, novel word formations, or other sources of vocabulary mismatch.
Restoring Hebrew Diacritics Without a Dictionary
We demonstrate that it is feasible to diacritize Hebrew script without any human-curated resources other than plain diacritized text.
Will it Unblend?
Natural language processing systems often struggle with out-of-vocabulary (OOV) terms, which do not appear in training data.
UniMorph 3.0: Universal Morphology
The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema.
Learning to Faithfully Rationalize by Construction
In NLP this often entails extracting snippets of an input text `responsible for' corresponding model output; when such a snippet comprises tokens that indeed informed the model's prediction, it is a faithful explanation.
NYTWIT: A Dataset of Novel Words in the New York Times
We present baseline results for both uncontextual and contextual prediction of novelty class, showing that there is room for improvement even for state-of-the-art NLP systems.
Attending Form and Context to Generate Specialized Out-of-VocabularyWords Representations
We propose a new contextual-compositional neural network layer that handles out-of-vocabulary (OOV) words in natural language processing (NLP) tagging tasks.
Attention is not not Explanation
We show that even when reliable adversarial distributions can be found, they don't perform well on the simple diagnostic, indicating that prior work does not disprove the usefulness of attention mechanisms for explainability.
Character Eyes: Seeing Language through Character-Level Taggers
Character-level models have been used extensively in recent years in NLP tasks as both supplements and replacements for closed-vocabulary token-level word representations.
Predicting Semantic Relations using Global Graph Properties
Semantic graphs, such as WordNet, are resources which curate natural language on two distinguishable layers.
Ranked #18 on
Link Prediction
on WN18RR
Si O No, Que Penses? Catalonian Independence and Linguistic Identity on Social Media
We also find that Catalan is used more often in referendum-related discourse than in other contexts, contrary to prior findings on language variation.
Sí o no, què penses? Catalonian Independence and Linguistic Identity on Social Media
We also find that Catalan is used more often in referendum-related discourse than in other contexts, contrary to prior findings on language variation.
Mimicking Word Embeddings using Subword RNNs
In this paper, we present MIMICK, an approach to generating OOV word embeddings compositionally, by learning a function from spellings to distributional embeddings.
The Yahoo Query Treebank, V. 1.0
A description and annotation guidelines for the Yahoo Webscope release of Query Treebank, Version 1. 0, May 2016.
