Search Results for author:
Found 19 papers, 11 papers with code
Attention Alignment and Flexible Positional Embeddings Improve Transformer Length Extrapolation
This suggests that a flexible positional embedding design and attention alignment can go a long way toward Transformer length extrapolation.
Advancing Regular Language Reasoning in Linear Recurrent Neural Networks
In recent studies, linear recurrent neural networks (LRNNs) have achieved Transformer-level performance in natural language and long-range modeling, while offering rapid parallel training and constant inference cost.
Structured Dialogue Discourse Parsing
In addition, unlike in previous work, we do not rely on hand-crafted features; this improves the model's robustness.
Ranked #1 on
Discourse Parsing
on STAC
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification.
Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings
The use of positional embeddings in transformer language models is widely accepted.
Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation
Unlike recurrent models, conventional wisdom has it that Transformers cannot perfectly model regular languages.
Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysis
Length extrapolation permits training a transformer language model on short sequences that preserves perplexities when tested on substantially longer sequences.
On Task-Adaptive Pretraining for Dialogue Response Selection
Recent advancements in dialogue response selection (DRS) are based on the \textit{task-adaptive pre-training (TAP)} approach, by first initializing their model with BERT~\cite{devlin-etal-2019-bert}, and adapt to dialogue data with dialogue-specific or fine-grained pre-training tasks.
Training Discrete Deep Generative Models via Gapped Straight-Through Estimator
While deep generative models have succeeded in image processing, natural language processing, and reinforcement learning, training that involves discrete random variables remains challenging due to the high variance of its gradient estimation process.
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation
Relative positional embeddings (RPE) have received considerable attention since RPEs effectively model the relative distance among tokens and enable length extrapolation.
Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection
In this paper, we are the first to propose a~\textbf{zero-shot} dialogue disentanglement solution.
Are you doing what I say? On modalities alignment in ALFRED
ALFRED is a recently proposed benchmark that requires a model to complete tasks in simulated house environments specified by instructions in natural language.
Automatic Speech Verification Spoofing Detection
Automatic speech verification (ASV) is the technology to determine the identity of a person based on their voice.
Just Ask:An Interactive Learning Framework for Vision and Language Navigation
We demonstrate the proposed strategy is substantially more realistic and data-efficient compared to previously proposed pre-exploration techniques.
BCWS: Bilingual Contextual Word Similarity
This paper introduces the first dataset for evaluating English-Chinese Bilingual Contextual Word Similarity, namely BCWS (https://github. com/MiuLab/BCWS).
CLUSE: Cross-Lingual Unsupervised Sense Embeddings
The model is evaluated on the Stanford Contextual Word Similarity (SCWS) dataset to ensure the quality of monolingual sense embeddings.
xSense: Learning Sense-Separated Sparse Representations and Textual Definitions for Explainable Word Sense Networks
This paper focuses on interpreting the embeddings for various aspects, including sense separation in the vector dimensions and definition generation.
Speaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning
Language understanding (LU) and dialogue policy learning are two essential components in conversational systems.
Dynamic Time-Aware Attention to Speaker Roles and Contexts for Spoken Language Understanding
However, the previous model only paid attention to the content in history utterances without considering their temporal information and speaker roles.
