Search Results for author:
Found 17 papers, 2 papers with code
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
In this paper, we introduce Direct Nash Optimization (DNO), a provable and scalable algorithm that marries the simplicity and stability of contrastive learning with theoretical generality from optimizing general preferences.
Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agents
We show that users spend a lot of ``effort'' on these questions in terms of signals like clicks and session length, and that they are also challenging for GPT-4.
Orca-Math: Unlocking the potential of SLMs in Grade School Math
Ensembling provides a substantial boost in accuracy but at a significant cost increase with multiple calls to the model (e. g., Phi-GSM uses top-48 to boost the performance from 68. 2 to 81. 5).
Ranked #35 on
Arithmetic Reasoning
on GSM8K
Axiomatic Preference Modeling for Longform Question Answering
The remarkable abilities of large language models (LLMs) like GPT-4 partially stem from post-training processes like Reinforcement Learning from Human Feedback (RLHF) involving human preferences encoded in a reward model.
Orca 2: Teaching Small Language Models How to Reason
Research on training small LMs has often relied on imitation learning to replicate the output of more capable models.
Ranked #1 on
Crass AI
on BIG-bench
Overview of the TREC 2023 Product Product Search Track
The focus this year was the creation of a reusable collection and evaluation of the impact of the use of metadata and multi-modal data on retrieval accuracy.
Automatic Pair Construction for Contrastive Post-training
Remarkably, our automatic contrastive post-training further improves the performance of Orca, already a state-of-the-art instruction learning model tuned with GPT-4 outputs, to outperform ChatGPT.
Nugget 2D: Dynamic Contextual Compression for Scaling Decoder-only Language Models
Standard Transformer-based language models (LMs) scale poorly to long contexts.
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from multiple information corpora ("external memories"), with the option to "plug in" new memory at inference time.
Zero-shot Clarifying Question Generation for Conversational Search
In this work, we innovatively explore generating clarifying questions in a zero-shot setting to overcome the cold start problem and we propose a constrained clarifying question generation system which uses both question templates and query facets to guide the effective and precise question generation.
Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention
Transformers have achieved new heights modeling natural language as a sequence of text tokens.
Ranked #42 on
Question Answering
on HotpotQA
Generic Intent Representation in Web Search
This paper presents GEneric iNtent Encoder (GEN Encoder) which learns a distributed representation space for user intent in search.
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Deep neural IR models, in contrast, compare the whole query to the document and are, therefore, typically employed only for late stage re-ranking.
An Axiomatic Approach to Regularizing Neural Ranking Models
The training of these models involve a search for appropriate parameter values based on large quantities of labeled examples.
Optimizing Query Evaluations using Reinforcement Learning for Web Search
In web search, typically a candidate generation step selects a small set of documents---from collections containing as many as billions of web pages---that are subsequently ranked and pruned before being presented to the user.
