Trending Research
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy.
LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding
alibabaresearch/advancedliteratemachinery
•
•
The core of LayoutLLM is a layout instruction tuning strategy, which is specially designed to enhance the comprehension and utilization of document layouts.
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Leeroo-AI/mergoo
•
•
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Ranked #30 on
Question Answering
on TriviaQA
Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
In this work, we highlight the following pitfall of prefilling: for batches containing high-varying prompt lengths, significant computation is wasted by the standard practice of padding sequences to the maximum length.
Arc2Face: A Foundation Model of Human Faces
This paper presents Arc2Face, an identity-conditioned face foundation model, which, given the ArcFace embedding of a person, can generate diverse photo-realistic images with an unparalleled degree of face similarity than existing models.
Ranked #1 on
Diffusion Personalization Tuning Free
on AgeDB
Rho-1: Not All Tokens Are What You Need
After fine-tuning, Rho-1-1B and 7B achieved state-of-the-art results of 40. 6% and 51. 8% on MATH dataset, respectively - matching DeepSeekMath with only 3% of the pretraining tokens.
Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
fudan-generative-vision/champ
•
•
In this study, we introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework to enhance shape alignment and motion guidance in curernt human generative techniques.
LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition
This enables our method - namely LAndmark-based Facial Self-supervised learning LAFS), to learn key representation that is more critical for face recognition.
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation.
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
picsart-ai-research/streamingt2v
•
•
To overcome these limitations, we introduce StreamingT2V, an autoregressive approach for long video generation of 80, 240, 600, 1200 or more frames with smooth transitions.
