Search Results for author:
Found 20 papers, 5 papers with code
Analyzing BERT’s Knowledge of Hypernymy via Prompting
The high performance of large pretrained language models (LLMs) such as BERT on NLP tasks has prompted questions about BERT’s linguistic capabilities, and how they differ from humans’.
Don’t Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
The representations in large language models contain multiple types of gender information.
GPT-2-based Human-in-the-loop Theatre Play Script Generation
We experiment with adapting generative language models for the generation of long coherent narratives in the form of theatre plays.
Debiasing Algorithm through Model Adaptation
Large language models are becoming the go-to solution for the ever-growing number of tasks.
Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation
We study the effect of tokenization on gender bias in machine translation, an aspect that has been largely overlooked in previous works.
Closing the loop: Autonomous experiments enabled by machine-learning-based online data analysis in synchrotron beamline environments
Our focus lies on the beamline integration of ML-based online data analysis and closed-loop feedback.
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages
Multilingual language models have recently gained attention as a promising solution for representing multiple languages in a single model.
Independent Components of Word Embeddings Represent Semantic Features
Independent Component Analysis (ICA) is an algorithm originally developed for finding separate sources in a mixed signal, such as a recording of multiple people in the same room speaking at the same time.
Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
The representations in large language models contain multiple types of gender information.
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes
The evaluated information is encoded in a shared cross-lingual embedding space.
THEaiTRE 1.0: Interactive generation of theatre play scripts
We present the first version of a system for interactive generation of theatre play scripts.
Introducing Orthogonal Constraint in Structural Probes
With the recent success of pre-trained models in NLP, a significant focus was put on interpreting their representations.
Syntax Representation in Word Embeddings and Neural Networks -- A Survey
Neural networks trained on natural language processing tasks capture syntax even though it is not provided as a supervision signal.
Measuring Memorization Effect in Word-Level Neural Networks Probing
In classical probing, a classifier is trained on the representations to extract the target linguistic information.
THEaiTRE: Artificial Intelligence to Write a Theatre Play
We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts.
Universal Dependencies according to BERT: both more specific and more general
This work focuses on analyzing the form and extent of syntactic abstraction captured by BERT by extracting labeled dependency trees from self-attentions.
Inducing Syntactic Trees from BERT Representations
We use the English model of BERT and explore how a deletion of one word in a sentence changes representations of other words.
Derivational Morphological Relations in Word Embeddings
Derivation is a type of a word-formation process which creates new words from existing ones by adding, changing or deleting affixes.
From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions
We inspect the multi-head self-attention in Transformer NMT encoders for three source languages, looking for patterns that could have a syntactic interpretation.
Input Combination Strategies for Multi-Source Transformer Decoder
In multi-source sequence-to-sequence tasks, the attention mechanism can be modeled in several ways.
