Search Results for author:
Found 14 papers, 7 papers with code
Orca 2: Teaching Small Language Models How to Reason
Research on training small LMs has often relied on imitation learning to replicate the output of more capable models.
Ranked #1 on
Crass AI
on BIG-bench
Teaching Language Models to Hallucinate Less with Synthetic Tasks
We also find that optimizing the system message rather than the model weights can be critical; fine-tuning the entire model on the synthetic task can counterintuitively increase hallucination.
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
While existing token-level early exit methods show promising results for online inference, they cannot be readily applied for batch inferencing and Key-Value caching.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka. ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs.
Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions
Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly.
Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions
The adoption of pre-trained language models to generate action plans for embodied agents is a promising research strategy.
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
In this work, we propose a method to leverage weak/noisy labels (e. g., risk scores generated by machine rules for detecting malware) that are cheaper to obtain for anomaly detection.
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions.
Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021
The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
However, a "head-to-toe assessment" regarding the extent of redundancy in ViTs, and how much we could gain by thoroughly mitigating such, has been absent for this field.
NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment
Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions.
Uncertainty-aware Self-training for Few-shot Text Classification
Recent success of pre-trained language models crucially hinges on fine-tuning them on large amounts of labeled data for the downstream task, that are typically expensive to acquire or difficult to access for many applications.
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Deep and large pre-trained language models are the state-of-the-art for various natural language processing tasks.
Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching
We frame this task as a binary (relevant/irrelevant) classification problem, and present an adversarial training framework to alleviate label imbalance issue.
