Search Results for author:
Found 14 papers, 8 papers with code
LLM Performance Predictors are good initializers for Architecture Search
We show that HS-NAS performs very similar to SOTA NAS across benchmarks, reduces search hours by 50% roughly, and in some cases, improves latency, GFLOPs, and model size.
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
In addition, the proposed method achieves the SOTA performance in NAS for building fast machine translation models, yielding better latency-BLEU tradeoff compared to HAT, state-of-the-art NAS for MT.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka. ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs.
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Furthermore, existing MoE works do not consider computational constraints (e. g., FLOPs, latency) to guide their design.
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints
In this work, we study the more challenging open-domain setting consisting of low frequency user prompt patterns (or broad prompts, e. g., prompt about 93rd academy awards) and demonstrate the effectiveness of character-based language models.
Automatic Detection of Entity-Manipulated Text using Factual Knowledge
We propose a neural network based detector that detects manipulated news articles by reasoning about the facts mentioned in the article.
Contrastive Learning of Sociopragmatic Meaning in Social Media
Recent progress in representation and contrastive learning in NLP has not widely considered the class of \textit{sociopragmatic meaning} (i. e., meaning in interaction within different language communities).
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
The problem of comparing two bodies of text and searching for words that differ in their usage between them arises often in digital humanities and computational social science.
Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing
We describe models focused at the understudied problem of translating between monolingual and code-mixed language pairs.
Automatic Detection of Machine Generated Text: A Critical Survey
Detectors that can distinguish text generated by TGM from human written text play a vital role in mitigating such misuse of TGMs.
Contextualized Diachronic Word Representations
We devise a novel attentional model, based on Bernoulli word embeddings, that are conditioned on contextual extra-linguistic (social) features such as network, spatial and socio-economic variables, which are associated with Twitter users, as well as topic-based features.
What Does BERT Learn about the Structure of Language?
BERT is a recent language representation model that has surprisingly performed well in diverse language understanding benchmarks.
ELMoLex: Connecting ELMo and Lexicon Features for Dependency Parsing
We augment the deep Biaffine (BiAF) parser (Dozat and Manning, 2016) with novel features to perform competitively: we utilize an indomain version of ELMo features (Peters et al., 2018) which provide context-dependent word representations; we utilize disambiguated, embedded, morphosyntactic features from lexicons (Sagot, 2018), which complements the existing feature set.
