Trending Research
Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
We also introduce the VoiceAssistant-400K dataset to fine-tune models optimized for speech output.
FLUX that Plays Music
feizc/fluxmusic
•
•
This paper explores a simple extension of diffusion-based rectified flow Transformers for text-to-music generation, termed as FluxMusic.
Ranked #2 on
Text-to-Music Generation
on MusicCaps
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling
Despite the reduced number of tokens, WavTokenizer achieves state-of-the-art reconstruction quality with outstanding UTMOS scores and inherently contains richer semantic information.
rerankers: A Lightweight Python Library to Unify Ranking Methods
This paper presents rerankers, a Python library which provides an easy-to-use interface to the most commonly used re-ranking approaches.
DGR-MIL: Exploring Diverse Global Representation in Multiple Instance Learning for Whole Slide Image Classification
Second, we propose two mechanisms to enforce the diversity among the global vectors to be more descriptive of the entire bag: (i) positive instance alignment and (ii) a novel, efficient, and theoretically guaranteed diversification learning paradigm.
Sapiens: Foundation for Human Vision Models
We present Sapiens, a family of models for four fundamental human-centric vision tasks -- 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction.
LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture
freedomintelligence/longllava
•
•
Expanding the long-context capabilities of Multi-modal Large Language Models~(MLLMs) is crucial for video understanding, high-resolution image understanding, and multi-modal agents.
NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results
caiyuanhao1998/retinexformer
•
•
This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results.
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders
nvlabs/eagle
•
•
We discover that simply concatenating visual tokens from a set of complementary vision encoders is as effective as more complex mixing architectures or strategies.
VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters
Surprisingly, without further adaptation in the time-series domain, the proposed VisionTS could achieve superior zero-shot forecasting performance compared to existing TSF foundation models.
