Search Results for author:
Found 15 papers, 4 papers with code
MatFormer: Nested Transformer for Elastic Inference
Furthermore, we observe that smaller encoders extracted from a universal MatFormer-based ViT (MatViT) encoder preserve the metric-space structure for adaptive large-scale retrieval.
Deep Neural Networks for Bot Detection
In this paper, we propose a deep neural network based on contextual long short-term memory (LSTM) architecture that exploits both content and metadata to detect bots at the tweet level: contextual features are extracted from user metadata and fed as auxiliary input to LSTM deep nets processing the tweet text.
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Ranked #1 on
Question Answering
on StrategyQA
MiTTenS: A Dataset for Evaluating Misgendering in Translation
Misgendering is the act of referring to someone in a way that does not reflect their gender identity.
Training Autoencoders by Alternating Minimization
We present DANTE, a novel method for training neural networks, in particular autoencoders, using the alternating minimization principle.
DANTE: Deep AlterNations for Training nEural networks
We present DANTE, a novel method for training neural networks using the alternating minimization principle.
Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
Over the last few years two promising research directions in low-resource neural machine translation (NMT) have emerged.
Exploring Routing Strategies for Multilingual Mixture-of-Experts Models
Sparsely-Gated Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training computation.
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, web-mined text datasets covering hundreds of languages.
MURAL: Multimodal, Multitask Retrieval Across Languages
Both image-caption pairs and translation pairs provide the means to learn deep representations of and connections between languages.
Ranked #1 on
Semantic Image-Text Similarity
on CxC
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference
On WMT, our task-MoE with 32 experts (533M parameters) outperforms the best performing token-level MoE model (token-MoE) by +1. 0 BLEU on average across 30 language pairs.
A Loss Curvature Perspective on Training Instabilities of Deep Learning Models
In this work, we study the evolution of the loss Hessian across many classification tasks in order to understand the effect the curvature of the loss has on the training dynamics.
A Loss Curvature Perspective on Training Instability in Deep Learning
In this work, we study the evolution of the loss Hessian across many classification tasks in order to understand the effect the curvature of the loss has on the training dynamics.
MURAL: Multimodal, Multitask Representations Across Languages
Both image-caption pairs and translation pairs provide the means to learn deep representations of and connections between languages.
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer
Despite remarkable advancements in few-shot generalization in natural language processing, most models are developed and evaluated primarily in English.
