Search Results for author:
Found 30 papers, 20 papers with code
Disentangling Textual and Acoustic Features of Neural Speech Representations
Neural speech models build deeply entangled internal representations, which capture a variety of features (e. g., fundamental frequency, loudness, syntactic category, or semantic content of a word) in a distributed encoding.
Encoding of lexical tone in self-supervised models of spoken language
Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological, syntactic and semantic levels, to speaker characteristics.
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited.
Quantifying the Plausibility of Context Reliance in Neural Machine Translation
Establishing whether language models can use contextual information in a human-plausible way is important to ensure their trustworthiness in real-world settings.
Wave to Syntax: Probing spoken language models for syntax
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures.
Putting Natural in Natural Language Processing
Human language is firstly spoken and only secondarily written.
Quantifying Context Mixing in Transformers
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models.
Learning English with Peppa Pig
In the real world the coupling between the linguistic and the visual modality is loose, and often confounded by correlations with non-semantic aspects of the speech signal.
Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations
The perturbed data, models, and code are available for reproduction at https://github. com/cmry/augtox
ZR-2021VG: Zero-Resource Speech Challenge, Visually-Grounded Language Modelling track, 2021 edition
We present the visually-grounded language modelling track that was introduced in the Zero-Resource Speech challenge, 2021 edition, 2nd round.
Discrete representations in neural models of spoken language
The distributed and continuous representations used by neural networks are at odds with representations employed in linguistics, which are typically symbolic.
Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques
This survey provides an overview of the evolution of visually grounded models of spoken language over the last 20 years.
Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling
Written language contains stylistic cues that can be exploited to automatically infer a variety of potentially sensitive author information.
Textual Supervision for Visually Grounded Spoken Language Understanding
Visually-grounded models of spoken language understanding extract semantic information directly from speech, without relying on transcriptions.
Learning to Understand Child-directed and Adult-directed Speech
Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signal itself, such as prosodic and phonemic variation.
Analyzing analytical methods: The case of phonology in neural models of spoken language
Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method.
Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
However, we do find that using an external machine translation model to generate the synthetic data sets results in better performance.
Correlating neural and symbolic representations of language
Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP.
Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language.
Symbolic inductive bias for visually grounded learning of spoken language
A widespread approach to processing spoken language is to first automatically transcribe it into text.
Lessons learned in multilingual grounded language learning
Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language.
Revisiting the Hierarchical Multiscale LSTM
Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art language model that learns interpretable structure from character-level input.
Style Obfuscation by Invariance
The task of obfuscating writing style using sequence models has previously been investigated under the framework of obfuscation-by-transfer, where the input text is explicitly rewritten in another style.
On the difficulty of a distributional semantics of spoken language
In the domain of unsupervised learning most work on speech has focused on discovering low-level constructs such as phoneme inventories or word-like units.
Encoding of phonology in a recurrent neural model of grounded speech
We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech.
Representations of language in a model of visually grounded speech signal
We present a visually grounded model of speech perception which projects spoken utterances and images to a joint semantic space.
From phonemes to images: levels of representation in a recurrent neural model of visually-grounded language learning
We present a model of visually-grounded language learning based on stacked gated recurrent neural networks which learns to predict visual features given an image description in the form of a sequence of phonemes.
Representation of linguistic form and function in recurrent neural networks
We present novel methods for analyzing the activation patterns of RNNs from a linguistic point of view and explore the types of linguistic structure they learn.
Learning language through pictures
We propose Imaginet, a model of learning visually grounded representations of language from coupled textual and visual input.
Text segmentation with character-level text embeddings
To demonstrate the usefulness of the learned text embeddings, we use them as features in a supervised character level text segmentation and labeling task: recognizing spans of text containing programming language code.
