Search Results for author:
Found 10 papers, 8 papers with code
Effective Long-Context Scaling of Foundation Models
We also examine the impact of various design choices in the pretraining process, including the data mix and the training curriculum of sequence lengths -- our ablation experiments suggest that having abundant long texts in the pretrain dataset is not the key to achieving strong performance, and we empirically verify that long context continual pretraining is more efficient and similarly effective compared to pretraining from scratch with long sequences.
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
Ranked #2 on
Question Answering
on PubChemQA
When should we prefer Decision Transformers for Offline Reinforcement Learning?
Three popular algorithms for offline RL are Conservative Q-Learning (CQL), Behavior Cloning (BC), and Decision Transformer (DT), from the class of Q-Learning, Imitation Learning, and Sequence Modeling respectively.
AUTODIAL: Efficient Asynchronous Task-Oriented Dialogue Model
As large dialogue models become commonplace in practice, the problems surrounding high compute requirements for training, inference and larger memory footprint still persists.
DiscoSense: Commonsense Reasoning with Discourse Connectives
We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives.
Ranked #8 on
Sentence Completion
on HellaSwag
Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey
While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks.
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics.
Adaptive Transformers for Learning Multimodal Representations
The usage of transformers has grown from learning about language semantics to forming meaningful visiolinguistic representations.
On Generalizing Detection Models for Unconstrained Environments
We address the problem of incremental learning in object detection on the India Driving Dataset (IDD).
Ranked #2 on
Object Detection
on BDD100K val
Incremental Learning in Person Re-Identification
In this paper, we propose a model that can be used for multiple tasks in Person Re-Identification, provide state-of-the-art results on a variety of tasks and still achieve considerable accuracy subsequently.
Ranked #69 on
Person Re-Identification
on DukeMTMC-reID
