Search Results for author:
Found 19 papers, 12 papers with code
LaTIM: Measuring Latent Token-to-Token Interactions in Mamba Models
State space models (SSMs), such as Mamba, have emerged as an efficient alternative to transformers for long-context sequence modeling.
AdaSplash: Adaptive Sparse Flash Attention
The computational cost of softmax-based attention in transformers limits their applicability to long-context tasks.
How Effective are State Space Models for Machine Translation?
Transformers are the current architecture of choice for NLP, but their attention layers do not scale well to long contexts.
xTower: A Multilingual LLM for Explaining and Correcting Translation Errors
While machine translation (MT) systems are achieving increasingly strong performance on benchmarks, they often produce translations with errors and anomalies.
CREST: A Joint Framework for Rationalization and Counterfactual Text Generation
Selective rationales and counterfactual examples have emerged as two effective, complementary classes of interpretability methods for analyzing and training NLP models.
The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics
Neural metrics for machine translation evaluation, such as COMET, exhibit significant improvements in their correlation with human judgments, as compared to traditional metrics based on lexical overlap, such as BLEU.
CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
We present the joint contribution of IST and Unbabel to the WMT 2022 Shared Task on Quality Estimation (QE).
Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
Learning to Scaffold: Optimizing Model Explanations for Teaching
In this work, leveraging meta-learning techniques, we extend this idea to improve the quality of the explanations themselves, specifically by optimizing explanations such that student models more effectively learn to simulate the original model.
Predicting Attention Sparsity in Transformers
Transformers' quadratic complexity with respect to the input sequence length has motivated a body of work on efficient sparse approximations to softmax.
Sparse Continuous Distributions and Fenchel-Young Losses
In contrast, for finite domains, recent work on sparse alternatives to softmax (e. g., sparsemax, $\alpha$-entmax, and fusedmax), has led to distributions with varying support.
Sparse and Continuous Attention Mechanisms
Exponential families are widely used in machine learning; they include many distributions in continuous and discrete domains (e. g., Gaussian, Dirichlet, Poisson, and categorical distributions via the softmax transformation).
Ranked #35 on
Visual Question Answering (VQA)
on VQA v2 test-std
Evaluating Sentence Segmentation in Different Datasets of Neuropsychological Language Tests in Brazilian Portuguese
Automatic analysis of connected speech by natural language processing techniques is a promising direction for diagnosing cognitive impairments.
Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task
We present the contribution of the Unbabel team to the WMT 2019 Shared Task on Quality Estimation.
Portuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks
Word embeddings have been found to provide meaningful representations for words in an efficient way; therefore, they have become common in Natural Language Processing sys- tems.
