Search Results for author:
Found 24 papers, 10 papers with code
Identifying Elements Essential for BERT's Multilinguality
We aim to identify architectural properties of BERT and linguistic properties of languages that are necessary for BERT to become multilingual.
Wine is not v i n. On the Compatibility of Tokenizations across Languages
The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.
An Information-Theoretic Approach and Dataset for Probing Gender Stereotypes in Multilingual Masked Language Models
Bias research in NLP is a rapidly growing and developing field.
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models
While Ferret seamlessly integrates regional understanding into the Large Language Model (LLM) to facilitate its referring and grounding capability, it poses certain limitations: constrained by the pre-trained fixed visual encoder and failed to perform well on broader tasks.
Ranked #83 on
Visual Question Answering
on MM-Vet
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #30 on
Visual Question Answering
on MM-Vet
Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages
Prior work on extracting MNE datasets from parallel corpora required resources such as large monolingual corpora or word aligners that are unavailable or perform poorly for underresourced languages.
BERT Cannot Align Characters
We show that the closer two languages are, the better BERT can align them on the character level.
Locating Language-Specific Information in Contextualized Embeddings
Multilingual pretrained language models (MPLMs) exhibit multilinguality and are well suited for transfer across languages.
Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages
The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.
Graph Algorithms for Multiparallel Word Alignment
With the advent of end-to-end deep learning approaches in machine translation, interest in word alignments initially decreased; however, they have again become a focus of research more recently.
ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus
With more than 7000 languages worldwide, multilingual natural language processing (NLP) is essential both from an academic and commercial perspective.
Static Embeddings as Efficient Knowledge Bases?
Recent research investigates factual knowledge stored in large pretrained language models (PLMs).
Position Information in Transformers: An Overview
Transformers are arguably the main workhorse in recent Natural Language Processing research.
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
(i) Can mBERT be used as a multilingual knowledge base?
Subword Sampling for Low Resource Word Alignment
This method's hypothesis is that the aggregation of different granularities of text for certain language pairs can help word-level alignment.
Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted Attention
Self-Attention Networks (SANs) are an integral part of successful neural architectures such as Transformer (Vaswani et al., 2017), and thus of pretrained language models such as BERT (Devlin et al., 2019) or GPT-3 (Brown et al., 2020).
Monolingual and Multilingual Reduction of Gender Bias in Contextualized Representations
Pretrained language models (PLMs) learn stereotypes held by humans and reflected in text from their training corpora, including gender bias.
Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs
We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation.
Ranked #5 on
KG-to-Text Generation
on AGENDA
Identifying Necessary Elements for BERT's Multilinguality
We aim to identify architectural properties of BERT and linguistic properties of languages that are necessary for BERT to become multilingual.
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Pretrained language models have achieved a new state of the art on many NLP tasks, but there are still many open questions about how and why they work so well.
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings
We find that alignments created from embeddings are superior for four and comparable for two language pairs compared to those produced by traditional statistical aligners, even with abundant parallel data; e. g., contextualized embeddings achieve a word alignment F1 for English-German that is 5 percentage points higher than eflomal, a high-quality statistical aligner, trained on 100k parallel sentences.
Analytical Methods for Interpretable Ultradense Word Embeddings
In this work, we investigate three methods for making word spaces interpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and DensRay, a new method we propose.
Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
A simple and effective context-based multilingual embedding learner is Levy et al. (2017)'s S-ID (sentence ID) method.
Embedding Learning Through Multilingual Concept Induction
We present a new method for estimating vector space representations of words: embedding learning by concept induction.
