Search Results for author:
Found 178 papers, 84 papers with code
Separating Hate Speech and Offensive Language Classes via Adversarial Debiasing
Research to tackle hate speech plaguing online media has made strides in providing solutions, analyzing bias and curating data.
Few-Shot Text Generation with Natural Language Instructions
Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.
Multidomain Pretrained Language Models for Green NLP
Thus, instead of training multiple models, we can train a single multidomain model saving on computational resources and training time.
Don’t Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings
The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.
Wine is not v i n. On the Compatibility of Tokenizations across Languages
The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.
Cross-Lingual Constituency Parsing for Middle High German: A Delexicalized Approach
However, training an automatic syntactic analysis system for ancient languages solely relying on annotated parse data is a formidable task due to the inherent challenges in building treebanks for such languages.
Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding
We conduct comprehensive experiments on diverse cross-lingual language understanding tasks (sentiment classification, paraphrase identification, and natural language inference) and empirically analyze the variation trends of prompt-based finetuning performance in cross-lingual transfer across different few-shot and full-data settings.
Natural Language Inference
Natural Language Understanding
+4
Politeness Stereotypes and Attack Vectors: Gender Stereotypes in Japanese and Korean Language Models
Non-English bias research, however, is still in its infancy with most work focusing on English.
On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
By conducting extensive experiments on different language pairs, including similar and distant, high and low-resource languages, we find that our method alleviates the copying problem, thus improving the translation performance on low-resource languages.
Evaluate What You Can't Evaluate: Unassessable Generated Responses Quality
There are risks in using eference-free evaluators based on LLMs to evaluate the quality of dialogue responses.
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
To the best of our knowledge, this is the first work comprehensively evaluating distillation objectives in both settings.
RET-LLM: Towards a General Read-Write Memory for Large Language Models
Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) through their extensive parameters and comprehensive data utilization.
mPLM-Sim: Unveiling Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models
Recent multilingual pretrained language models (mPLMs) have been shown to encode strong language-specific signals, which are not explicitly provided during pretraining.
Open-Ended Question Answering
Zero-Shot Cross-Lingual Transfer
Language-Agnostic Bias Detection in Language Models
Pretrained language models (PLMs) are key components in NLP, but they contain strong social biases.
A study of conceptual language similarity: comparison and evaluation
An interesting line of research in natural language processing (NLP) aims to incorporate linguistic typology to bridge linguistic diversity and assist the research of low-resource languages.
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs
This demonstrates the benefits of colexification for multilingual NLP.
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages
The NLP community has mainly focused on scaling Large Language Models (LLMs) vertically, i. e., making them better for about 100 languages.
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages
While natural language processing tools have been developed extensively for some of the world's languages, a significant portion of the world's over 7000 languages are still neglected.
A Crosslingual Investigation of Conceptualization in 1335 Languages
The resulting measure for the conceptual similarity of two languages is complementary to standard genealogical, typological, and surface similarity measures.
NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis
In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model.
Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages
This can for instance be observed when finetuning PLMs on one language and evaluating them on data in a closely related language variety with no standardized orthography.
A Survey of Corpora for Germanic Low-Resource Languages and Dialects
In this work, we instead focus on low-resource languages and in particular non-standardized low-resource languages.
LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction
Our models outperform 10x larger language models without instruction tuning on various tasks such as story/recipe generation and long-form question answering.
Sociocultural knowledge is needed for selection of shots in hate speech detection tasks
We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for the countries of Brazil, Germany, India and Kenya, to aid training and interpretability of models.
MenuCraft: Interactive Menu System Design with Large Language Models
Menu system design is a challenging task involving many design options and various human factors.
Construction Grammar Provides Unique Insight into Neural Language Models
Construction Grammar (CxG) has recently been used as the basis for probing studies that have investigated the performance of large pretrained language models (PLMs) with respect to the structure and meaning of constructions.
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages
Multilingual Pretrained Language Models (MPLMs) have shown their strong multilinguality in recent empirical cross-lingual transfer studies.
PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism
We investigate response generation for multi-turn dialogue in generative-based chatbots.
Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology
We propose a fully unsupervised method to detect bias in contextualized embeddings.
Hengam: An Adversarially Trained Transformer for Persian Temporal Tagging
We create Hengam in the following concrete steps: (1) we develop HengamTagger, an extensible rule-based tool that can extract temporal expressions from a set of diverse language-specific patterns for any language of interest.
Ranked #1 on
Temporal Tagging
on HengamCorpus
MEAL: Stable and Active Learning for Few-Shot Prompting
Few-shot classification has made great strides due to foundation models that, through priming and prompting, are highly effective few-shot learners.
This joke is [MASK]: Recognizing Humor and Offense with Prompting
Humor is a magnetic component in everyday human interactions and communications.
The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative
Construction Grammar (CxG) is a paradigm from cognitive linguistics emphasising the connection between syntax and semantics.
Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging
An established method for training a POS tagger in such a scenario is to create a labeled training set by transferring from high-resource languages.
SilverAlign: MT-Based Silver Data Algorithm For Evaluating Word Alignment
Word alignments are essential for a variety of NLP tasks.
Federated Continual Learning for Text Classification via Selective Inter-client Transfer
The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data.
Modeling Content-Emotion Duality via Disentanglement for Empathetic Conversation
To solve the task, it is essential to model the content-emotion duality of a dialogue, which is composed of the content view (i. e., what personal experiences are described) and the emotion view (i. e., the feelings of the speaker on these experiences).
Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings
The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.
Analyzing Hate Speech Data along Racial, Gender and Intersectional Axes
To tackle the rising phenomenon of hate speech, efforts have been made towards data curation and analysis.
Flow-Adapter Architecture for Unsupervised Machine Translation
The primary novelties of our model are: (a) capturing language-specific sentence representations separately for each language using normalizing flows and (b) using a simple transformation of these latent representations for translating from one language to another.
Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?
The practical success of much of NLP depends on the availability of training data.
CaMEL: Case Marker Extraction without Labels
We introduce CaMEL (Case Marker Extraction without Labels), a novel and challenging task in computational morphology that is especially relevant for low-resource languages.
ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations
Since conventional knowledge embedding models cannot take full advantage of the abundant textual information, there have been extensive research efforts in enhancing knowledge embedding using texts.
Graph Neural Networks for Multiparallel Word Alignment
First, we create a multiparallel word alignment graph, joining all bilingual word alignment pairs in one graph.
Geographic Adaptation of Pretrained Language Models
Evaluation on three tasks, namely fine-tuned as well as zero-shot geolocation prediction and zero-shot prediction of dialect features, shows that geoadaptation is very effective: e. g., we obtain state-of-the-art performance in supervised geolocation prediction and report massive gains over geographically uninformed PLMs on zero-shot geolocation prediction.
Modular and Parameter-Efficient Multimodal Fusion with Prompting
Recent research has made impressive progress in large-scale multimodal pre-training.
CoDA21: Evaluating Language Understanding Capabilities of NLP Models With Context-Definition Alignment
Pretrained language models (PLMs) have achieved superhuman performance on many benchmarks, creating a need for harder tasks.
Semantic-Oriented Unlabeled Priming for Large-Scale Language Models
Due to the high costs associated with finetuning large language models, various recent works propose to adapt them to specific tasks without any parameter updates through in-context learning.
Towards a Broad Coverage Named Entity Resource: A Data-Efficient Approach for Many Diverse Languages
Prior work on extracting MNE datasets from parallel corpora required resources such as large monolingual corpora or word aligners that are unavailable or perform poorly for underresourced languages.
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers.
True Few-Shot Learning with Prompts -- A Real-World Perspective
Prompt-based approaches are strong at few-shot learning.
BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief
We show that, in a controlled experimental setting, these two mechanisms result in more consistent beliefs in the overall system, improving both the accuracy and consistency of its answers over time.
Active Learning for Argument Mining: A Practical Approach
Despite considerable recent progress, the creation of well-balanced and diverse resources remains a time-consuming and costly challenge in Argument Mining.
Scene Graph Generation for Better Image Captioning?
We investigate the incorporation of visual relationships into the task of supervised image caption generation by proposing a model that leverages detected objects and auto-generated visual relationships to describe images in natural language.
BERT Cannot Align Characters
We show that the closer two languages are, the better BERT can align them on the character level.
Locating Language-Specific Information in Contextualized Embeddings
Multilingual pretrained language models (MPLMs) exhibit multilinguality and are well suited for transfer across languages.
Graph Algorithms for Multiparallel Word Alignment
With the advent of end-to-end deep learning approaches in machine translation, interest in word alignments initially decreased; however, they have again become a focus of research more recently.
Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages
The size of the vocabulary is a central design choice in large pretrained language models, with respect to both performance and memory requirements.
Continuous Entailment Patterns for Lexical Inference in Context
If we allow for tokens outside the PLM's vocabulary, patterns can be adapted more flexibly to a PLM's idiosyncrasies.
Ranked #1 on
Few-Shot NLI
on SherLIiC
Discrete and Soft Prompting for Multilingual Models
It has been shown for English that discrete and soft prompting perform strongly in few-shot learning with pretrained language models (PLMs).
ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus
With more than 7000 languages worldwide, multilingual natural language processing (NLP) is essential both from an academic and commercial perspective.
Data Centric Domain Adaptation for Historical Text with OCR Errors
We propose new methods for in-domain and cross-domain Named Entity Recognition (NER) on historical data for Dutch and French.
Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity
The increasing polarization of online political discourse calls for computational tools that automatically detect and monitor ideological divides in social media.
Multi-source Neural Topic Modeling in Multi-view Embedding Spaces
Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity in short-text or small collection of documents.
Generating Datasets with Pretrained Language Models
To obtain high-quality sentence embeddings from pretrained language models (PLMs), they must either be augmented with additional pretraining objectives or finetuned on a large set of labeled text pairs.
Ranked #7 on
Semantic Textual Similarity
on SICK
Static Embeddings as Efficient Knowledge Bases?
Recent research investigates factual knowledge stored in large pretrained language models (PLMs).
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce.
Position Information in Transformers: An Overview
Transformers are arguably the main workhorse in recent Natural Language Processing research.
Language Models for Lexical Inference in Context
Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences, i. e., sentences that only differ in one expression.
Ranked #2 on
Few-Shot NLI
on SherLIiC
Improving Scene Graph Classification by Exploiting Knowledge from Texts
We show that by fine-tuning the classification pipeline with the extracted knowledge from texts, we can achieve ~8x more accurate results in scene graph classification, ~3x in object classification, and ~1. 5x in predicate classification, compared to the supervised baselines with only 1% of the annotated images.
Does He Wink or Does He Nod? A Challenging Benchmark for Evaluating Word Understanding of Language Models
Recent progress in pretraining language models on large corpora has resulted in large performance gains on many NLP tasks.
Measuring and Improving Consistency in Pretrained Language Models
In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge?
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
(i) Can mBERT be used as a multilingual knowledge base?
Superbizarre Is Not Superb: Derivational Morphology Improves BERT's Interpretation of Complex Words
How does the input segmentation of pretrained language models (PLMs) affect their interpretations of complex words?
A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters
Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT.
Few-Shot Text Generation with Pattern-Exploiting Training
Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.
Subword Sampling for Low Resource Word Alignment
This method's hypothesis is that the aggregation of different granularities of text for certain language pairs can help word-level alignment.
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained language model and map the predicted words to labels.
Dynamic Contextualized Word Embeddings
Static word embeddings that represent words by a single vector cannot capture the variability of word meaning in different linguistic and extralinguistic contexts.
TopicBERT for Energy Efficient Document Classification
Our work thus focuses on optimizing the computational cost of fine-tuning for document classification.
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners
When scaled to hundreds of billions of parameters, pretrained language models such as GPT-3 (Brown et al., 2020) achieve remarkable few-shot performance.
Investigating Pretrained Language Models for Graph-to-Text Generation
We show that the PLMs BART and T5 achieve new state-of-the-art results and that task-adaptive pretraining strategies improve their performance even further.
Ranked #1 on
KG-to-Text Generation
on WebNLG (All)
Automatic Domain Adaptation Outperforms Manual Domain Adaptation for Predicting Financial Outcomes
In our experiments, we demonstrate that the automatically adapted sentiment dictionary outperforms the previous state of the art in predicting the financial outcomes excess return and volatility.
Neural Topic Modeling with Continual Lifelong Learning
To address the problem, we propose a lifelong learning framework for neural topic modeling that can continuously process streams of document collections, accumulate topics and guide future topic modeling tasks by knowledge transfer from several sources to better deal with the sparse data.
Explainable and Discourse Topic-aware Neural Language Understanding
Marrying topic models and language models exposes language understanding to a broader source of document-level context beyond sentences via topics.
Are Pretrained Language Models Symbolic Reasoners Over Knowledge?
How can pretrained language models (PLMs) learn factual knowledge from the training set?
Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs
We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation.
Ranked #5 on
KG-to-Text Generation
on AGENDA
Unsupervised Embedding-based Detection of Lexical Semantic Changes
This paper describes EmbLexChange, a system introduced by the "Life-Language" team for SemEval-2020 Task 1, on unsupervised detection of lexical-semantic changes.
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Khandelwal et al. (2020) use a k-nearest-neighbor (kNN) component to improve language model performance.
DagoBERT: Generating Derivational Morphology with a Pretrained Language Model
Can pretrained language models (PLMs) generate derivationally complex words?
Identifying Necessary Elements for BERT's Multilinguality
We aim to identify architectural properties of BERT and linguistic properties of languages that are necessary for BERT to become multilingual.
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning.
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Pretrained language models have achieved a new state of the art on many NLP tasks, but there are still many open questions about how and why they work so well.
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings
We find that alignments created from embeddings are superior for four and comparable for two language pairs compared to those produced by traditional statistical aligners, even with abundant parallel data; e. g., contextualized embeddings achieve a word alignment F1 for English-German that is 5 percentage points higher than eflomal, a high-quality statistical aligner, trained on 100k parallel sentences.
Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA
Domain adaptation of Pretrained Language Models (PTLMs) is typically achieved by unsupervised pretraining on target-domain text.
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e. g., Radford et al., 2019).
Multipurpose Intelligent Process Automation via Conversational Assistant
Such intelligent agents can assist the user by answering specific questions and executing routine tasks that are ordinarily performed in a natural language (i. e., customer support).
Extending Machine Language Models toward Human-Level Language Understanding
We take language to be a part of a system for understanding and communicating about situations.
Morphological Segmentation Inside-Out
Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output.
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
We present a novel way of injecting factual knowledge about entities into the pretrained BERT model (Devlin et al., 2019): We align Wikipedia2Vec entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors.
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity
We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre-trained sentence encoders into sentence meta-embeddings.
Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly
We find that PLMs do not distinguish between negated ("Birds cannot [MASK]") and non-negated ("Birds can [MASK]") cloze questions.
BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance
In this work, we transfer this idea to pretrained language models: We introduce BERTRAM, a powerful architecture based on BERT that is capable of inferring high-quality embeddings for rare words that are suitable as input representations for deep language models.
Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in BioNLP-OST 2019
Our RE system ranked first in the SeeDev-binary Relation Extraction Task with F1-score of 0. 3738.
BioNLP-OST 2019 RDoC Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions
This paper presents our system details and results of participation in the RDoC Tasks of BioNLP-OST 2019.
Type-aware Convolutional Neural Networks for Slot Filling
In particular, we explore different ways of integrating the named entity types of the relation arguments into a neural network for relation classification, including a joint training and a structured prediction approach.
Multi-source Multi-view Transfer Learning in Neural Topic Modeling with Pretrained Topic and Word Embeddings
Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity problem in short text or small collection of documents.
Generating Multi-Sentence Abstractive Summaries of Interleaved Texts
The interleaved posts are encoded hierarchically, i. e., word-to-word (words in a post) followed by post-to-post (posts in a channel).
Multi-view and Multi-source Transfers in Neural Topic Modeling with Pretrained Topic and Word Embeddings
Though word embeddings and topics are complementary representations, several past works have only used pre-trained word embeddings in (neural) topic modeling to address data sparsity problem in short text or small collection of documents.
Neural Architectures for Fine-Grained Propaganda Detection in News
To address the tasks of sentence (SLC) and fragment level (FLC) propaganda detection, we explore different neural architectures (e. g., CNN, LSTM-CRF and BERT) and extract linguistic (e. g., part-of-speech, named entity, readability, sentiment, emotion, etc.
Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings
Word embeddings typically represent different meanings of a word in a single conflated vector.
A Hierarchical Decoder with Three-level Hierarchical Attention to Generate Abstractive Summaries of Interleaved Texts
Interleaved texts, where posts belonging to different threads occur in one sequence, are a common occurrence, e. g., online chat conversations.
SherLIiC: A Typed Event-Focused Lexical Inference Benchmark for Evaluating Natural Language Inference
We present SherLIiC, a testbed for lexical inference in context (LIiC), consisting of 3985 manually annotated inference rule candidates (InfCands), accompanied by (i) ~960k unlabeled InfCands, and (ii) ~190k typed textual relations between Freebase entities extracted from the large entity-linked corpus ClueWeb09.
Ranked #1 on
Lexical Entailment
on SherLIiC
Fine-Grained Argument Unit Recognition and Classification
In this work, we argue that the task should be performed on a more fine-grained level of sequence labeling.
An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing
To this end, we present the first approach to unsupervised text generation from KGs and show simultaneously how it can be used for unsupervised semantic parsing.
Ranked #1 on
Unsupervised KG-to-Text Generation
on VG graph-text
Analytical Methods for Interpretable Ultradense Word Embeddings
In this work, we investigate three methods for making word spaces interpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and DensRay, a new method we propose.
Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking
Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks.
Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts
Learning high-quality embeddings for rare words is a hard problem because of sparse context information.
Learning Semantic Representations for Novel Words: Leveraging Both Form and Context
The general problem setting is that word embeddings are induced on an unlabeled training corpus and then a model is trained that embeds novel words into this induced embedding space.
CIS at TAC Cold Start 2015: Neural Networks and Coreference Resolution for Slot Filling
Especially, it focuses on the coreference and classification component.
Multilingual Embeddings Jointly Induced from Contexts and Concepts: Simple, Strong and Scalable
A simple and effective context-based multilingual embedding learner is Levy et al. (2017)'s S-ID (sentence ID) method.
Aligning Very Small Parallel Corpora Using Cross-Lingual Word Embeddings and a Monogamy Objective
Count-based word alignment methods, such as the IBM models or fast-align, struggle on very small parallel corpora.
Multi-Multi-View Learning: Multilingual and Multi-Representation Entity Typing
For representation, we consider representations based on the context distribution of the entity (i. e., on its embedding), on the entity's name (i. e., on its surface form) and on its description in Wikipedia.
Neural Relation Extraction Within and Across Sentence Boundaries
iDepNN models the shortest and augmented dependency paths via recurrent and recursive neural networks to extract relationships within (intra-) and across (inter-) sentence boundaries.
Ranked #1 on
Relation Extraction
on MUC6
textTOvec: Deep Contextualized Neural Autoregressive Topic Models of Language with Distributed Compositional Prior
We address two challenges of probabilistic topic modelling in order to better estimate the probability of a word in a given context, i. e., P(word|context): (1) No Language Structure in Context: Probabilistic topic models ignore word order by summarizing a given context as a "bag-of-word" and consequently the semantics of words in the context is lost.
Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting
Neural state-of-the-art sequence-to-sequence (seq2seq) models often do not perform well for small training sets.
Interpretable Textual Neuron Representations for NLP
Input optimization methods, such as Google Deep Dream, create interpretable representations of neurons for computer vision DNNs.
Document Informed Neural Autoregressive Topic Models with Distributional Prior
Here, we extend a neural autoregressive topic model to exploit the full context information around words in a document in a language modeling fashion.
Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging
Character-level models of tokens have been shown to be effective at dealing with within-token noise and out-of-vocabulary words.
Document Informed Neural Autoregressive Topic Models
Context information around words helps in determining their actual meaning, for example "networks" used in contexts of artificial neural networks or biological neuron networks.
LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation
Recurrent neural networks (RNNs) are temporal networks and cumulative in nature that have shown promising results in various natural language processing tasks.
News Article Teaser Tweets and How to Generate Them
In this work, we define the task of teaser generation and provide an evaluation benchmark and baseline systems for the process of generating teasers.
Evaluating Word Embeddings in Multi-label Classification Using Fine-grained Name Typing
We propose a new evaluation method for word embeddings based on multi-label classification given a word embedding.
Replicated Siamese LSTM in Ticketing System for Similarity Learning and Retrieval in Asymmetric Texts
The task is challenging due to significant term mismatch in the query and ticket pairs of asymmetric lengths, where subject is a short text but description and solution are multi-sentence texts.
Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs
Large scale knowledge graphs (KGs) such as Freebase are generally incomplete.
Joint Bootstrapping Machines for High Confidence Relation Extraction
Semi-supervised bootstrapping techniques for relationship extraction from text iteratively expand a set of initial seed instances.
Relationship Extraction (Distant Supervised)
Vocal Bursts Intensity Prediction
End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions
This work deals with SciTail, a natural entailment challenge derived from a multi-choice question answering problem.
Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages
Morphological segmentation for polysynthetic languages is challenging, because a word may consist of many individual morphemes and training data can be extremely scarce.
Neural Architectures for Open-Type Relation Argument Extraction
In this work, we introduce the task of Open-Type Relation Argument Extraction (ORAE): Given a corpus, a query entity Q and a knowledge base relation (e. g.,"Q authored notable work with title X"), the model has to extract an argument of non-standard entity type (entities that cannot be extracted by a standard named entity tagger, e. g. X: the title of a book or a work of art) from the corpus.
Embedding Learning Through Multilingual Concept Induction
We present a new method for estimating vector space representations of words: embedding learning by concept induction.
Evaluating neural network explanation methods using hybrid documents and morphological agreement
The behavior of deep neural networks (DNNs) is hard to understand.
Deep Temporal-Recurrent-Replicated-Softmax for Topical Trends over Time
We also introduce a metric (named as SPAN) to quantify the capability of dynamic topic model to capture word evolution in topics over time.
Impact of Coreference Resolution on Slot Filling
In this paper, we demonstrate the importance of coreference resolution for natural language processing on the example of the TAC Slot Filling shared task.
Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
We hypothesize that this is because the attention in CNNs has been mainly implemented as attentive pooling (i. e., it is applied to pooling) rather than as attentive convolution (i. e., it is integrated into convolution).
Corpus-level Fine-grained Entity Typing
This paper addresses the problem of corpus-level entity typing, i. e., inferring from a large corpus that an entity is a member of a class such as "food" or "artist".
Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification
We introduce globally normalized convolutional neural networks for joint entity classification and relation extraction.
Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models
We present a semi-supervised way of training a character-based encoder-decoder recurrent neural network for morphological reinflection, the task of generating one inflected word form from another.
Past, Present, Future: A Computational Investigation of the Typology of Tense in 1000 Languages
We present SuperPivot, an analysis method for low-resource languages that occur in a superparallel corpus, i. e., in a corpus that contains an order of magnitude more languages than parallel corpora currently in use.
One-Shot Neural Cross-Lingual Transfer for Paradigm Completion
We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a neural encoder-decoder model, the state of the art for the monolingual task.
Comparative Study of CNN and RNN for Natural Language Processing
Deep neural networks (DNN) have revolutionized the field of natural language processing (NLP).
Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching
This work studies comparatively two typical sentence matching tasks: textual entailment (TE) and answer selection (AS), observing that weaker phrase alignments are more critical in TE, while stronger phrase alignments deserve more attention in AS.
Multi-level Representations for Fine-Grained Typing of Knowledge Base Entities
Entities are essential elements of natural language.
Joint Semantic Synthesis and Morphological Analysis of the Derived Word
Since morphology obeys the principle of compositionality, the semantics of the word can be systematically derived from the meaning of its parts.
Noise Mitigation for Neural Entity Typing and Relation Extraction
For the second noise type, we propose ways to improve the integration of noisy entity type predictions into relation extraction.
Exploring Different Dimensions of Attention for Uncertainty Detection
Neural networks with attention have proven effective for many natural language processing tasks.
Neural Multi-Source Morphological Reinflection
We explore the task of multi-source morphological reinflection, which generalizes the standard, single-source version.
Corpus-level Fine-grained Entity Typing Using Contextual Information
This paper addresses the problem of corpus-level entity typing, i. e., inferring from a large corpus that an entity is a member of a class such as "fo