Trending Research
Efficient Streaming Language Models with Attention Sinks
In this paper, we first demonstrate that the emergence of attention sink is due to the strong attention scores towards initial tokens as a ``sink'' even if they are not semantically important.
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
dreamgaussian/dreamgaussian
•
•
In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significantly faster for 3D generative tasks.
Break-A-Scene: Extracting Multiple Concepts from a Single Image
Text-to-image model personalization aims to introduce a user-provided concept to the model, allowing its synthesis in diverse contexts.
Demystifying CLIP Data
We believe that the main ingredient to the success of CLIP is its data and not the model architecture or pre-training objective.
Text-to-3D using Gaussian Splatting
gsgen3d/gsgen
•
•
In this stage, we increase the number of Gaussians by compactness-based densification to enhance continuity and improve fidelity.
ProPainter: Improving Propagation and Transformer for Video Inpainting
sczhou/propainter
•
•
ICCV 2023
We also propose a mask-guided sparse video Transformer, which achieves high efficiency by discarding unnecessary and redundant tokens.
Ranked #1 on
Video Inpainting
on DAVIS
Representation Engineering: A Top-Down Approach to AI Transparency
In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience.
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters.
Bayesian Flow Networks
nnaisense/bayesian-flow-networks
•
•
Notably, the network inputs for discrete data lie on the probability simplex, and are therefore natively differentiable, paving the way for gradient-based sample guidance and few-step generation in discrete domains such as language modelling.
Ranked #3 on
Image Generation
on Binarized MNIST
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics.
Ranked #8 on
Math Word Problem Solving
on MATH
