Search Results for author:
Found 36 papers, 24 papers with code
Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
In this work, we introduce Janus-Pro, an advanced version of the previous work Janus.
Ranked #63 on
Visual Question Answering
on MM-Vet
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
DeepSeek-V3 Technical Report
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades.
Ranked #1 on
Referring Expression Comprehension
on RefCOCOg-test
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training.
Ranked #199 on
Visual Question Answering
on MM-Vet
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
In this paper, we introduce Janus, an autoregressive framework that unifies multimodal understanding and generation.
Ranked #166 on
Visual Question Answering
on MM-Vet
Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment
Despite the vital role reward models play in alignment, previous works have consistently overlooked their performance and used off-the-shelf reward models arbitrarily without verification, rendering the reward model ``\emph{an elephant in the room}''.
The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
As a result, previous debiasing methods mainly finetune or even pre-train language models on newly constructed anti-stereotypical datasets, which are high-cost.
Improving Long Text Understanding with Knowledge Distilled from Summarization Model
Long text understanding is important yet challenging for natural language processing.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.
InTeX: Interactive Text-to-texture Synthesis via Unified Depth-aware Inpainting
Text-to-texture synthesis has become a new frontier in 3D content creation thanks to the recent advances in text-to-image models.
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
2) 3D Backbone: We present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models.
Uncovering and Categorizing Social Biases in Text-to-SQL
In this work, we aim to uncover and categorize social biases in Text-to-SQL models.
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
We hope this model can set a new baseline for generalist vision and language models.
Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Prediction
In this paper, we propose a real-time semantic scene completion method with a feature aggregation strategy and conditioned prediction module.
Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement
Neural Radiance Fields (NeRF) have constituted a remarkable breakthrough in image-based 3D reconstruction.
Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios
However, current works pursuing sentence-level explanations rely heavily on annotated training data, which limits the development of interpretability to only a few tasks.
Understanding Self-Supervised Pretraining with Part-Aware Representation Learning
The study is mainly motivated by that random views, used in contrastive learning, and random masked (visible) patches, used in masked image modeling, are often about object parts.
Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition
While dynamic Neural Radiance Fields (NeRF) have shown success in high-fidelity 3D modeling of talking portraits, the slow training and inference speed severely obstruct their potential usage.
D$^3$ETR: Decoder Distillation for Detection Transformer
Specifically, Adaptive Matching applies bipartite matching to adaptively match the outputs of the teacher and the student in each decoder layer, while Fixed Matching fixes the correspondence between the outputs of the teacher and the student with the same object queries, with the teacher's fixed object queries fed to the decoder of the student as an auxiliary group.
CAE v2: Context Autoencoder with CLIP Target
That is to say, the smaller the model, the lower the mask ratio needs to be.
Group DETR v2: Strong Object Detector with Encoder-Decoder Pretraining
The training process consists of self-supervised pretraining and finetuning a ViT-Huge encoder on ImageNet-1K, pretraining the detector on Object365, and finally finetuning it on COCO.
Ranked #7 on
Object Detection
on COCO test-dev
Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment
Detection transformer (DETR) relies on one-to-one assignment, assigning one ground-truth object to one prediction, for end-to-end detection without NMS post-processing.
Conditional DETR V2: Efficient Detection Transformer with Box Queries
Inspired by Conditional DETR, an improved DETR with fast training convergence, that presented box queries (originally called spatial queries) for internal decoder layers, we reformulate the object query into the format of the box query that is a composition of the embeddings of the reference point and the transformation of the box with respect to the reference point.
Compressible-composable NeRF via Rank-residual Decomposition
To circumvent the hurdle, in this paper, we present an explicit neural field representation that enables efficient and convenient manipulation of models.
Point Scene Understanding via Disentangled Instance Mesh Reconstruction
Semantic scene reconstruction from point cloud is an essential and challenging task for 3D scene understanding.
MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation
For instance, our approach achieves a 66. 4\% mAP with the 0. 5 IoU threshold on the ScanNetV2 test set, which is 1. 9\% higher than the state-of-the-art method.
Ranked #7 on
3D Instance Segmentation
on S3DIS
Context Autoencoder for Self-Supervised Representation Learning
The pretraining tasks include two tasks: masked representation prediction - predict the representations for the masked patches, and masked patch reconstruction - reconstruct the masked patches.
Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective
Firstly, we transfer the voxelized scenes to point clouds by removing these visible empty voxels and adopt a deep point stream to capture semantic information from the scene efficiently.
Ranked #5 on
3D Semantic Scene Completion
on NYUv2
Conditional DETR for Fast Training Convergence
Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention.
Ranked #109 on
Object Detection
on COCO minival
Joint Implicit Image Function for Guided Depth Super-Resolution
Inspired by the recent progress in implicit neural representation, we propose to formulate the guided super-resolution as a neural implicit image interpolation problem, where we take the form of a general image interpolation but use a novel Joint Implicit Image Function (JIIF) representation to learn both the interpolation weights and values.
Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision
Our approach imposes the consistency on two segmentation networks perturbed with different initialization for the same input image.
Ranked #2 on
Semi-Supervised Semantic Segmentation
on WoodScape
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation.
Ranked #1 on
Semantic Segmentation
on TLCGIS
3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently.
3D Semantic Scene Completion from a single RGB image
Hallucination
Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
We empirically demonstrate that the proposed approach achieves competitive performance on various challenging semantic segmentation benchmarks: Cityscapes, ADE20K, LIP, PASCAL-Context, and COCO-Stuff.
Ranked #5 on
Semantic Segmentation
on LIP val
