Trending Research
Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
In this work, we highlight the following pitfall of prefilling: for batches containing high-varying prompt lengths, significant computation is wasted by the standard practice of padding sequences to the maximum length.
SchurVINS: Schur Complement-Based Lightweight Visual Inertial Navigation System
To this end, we propose a novel filter-based VINS framework named SchurVINS, which could guarantee both high accuracy by building a complete residual model and low computational complexity with Schur complement.
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB).
Rho-1: Not All Tokens Are What You Need
After fine-tuning, Rho-1-1B and 7B achieved state-of-the-art results of 40. 6% and 51. 8% on MATH dataset, respectively - matching DeepSeekMath with only 3% of the pretraining tokens.
LLoCO: Learning Long Contexts Offline
We introduce LLoCO, a technique that combines context compression, retrieval, and parameter-efficient finetuning using LoRA.
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
openbmb/minicpm
•
•
For data scaling, we introduce a Warmup-Stable-Decay (WSD) learning rate scheduler (LRS), conducive to continuous training and domain adaptation.
DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations
The Q-Formers are trained using paired images rather than the identical target, in which the reference image and the ground-truth image are with the same style or semantics.
A Survey of Large Language Models
To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size.
HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach
airi-institute/hairfastgan
•
•
Our paper addresses the complex task of transferring a hairstyle from a reference image to an input photo for virtual hair try-on.
UFO: A UI-Focused Agent for Windows OS Interaction
We introduce UFO, an innovative UI-Focused agent to fulfill user requests tailored to applications on Windows OS, harnessing the capabilities of GPT-Vision.
