Search Results for author:
Found 83 papers, 39 papers with code
VGNC: Reducing the Overfitting of Sparse-view 3DGS via Validation-guided Gaussian Number Control
Recently, many methods based on the 3D Gaussian Splatting (3DGS) framework have been proposed to address sparse-view 3D reconstruction.
TimeSearch: Hierarchical Video Search with Spotlight and Reflection for Human-like Long Video Understanding
Motivated by human hierarchical temporal search strategies, we propose \textbf{TimeSearch}, a novel framework enabling LVLMs to understand long videos in a human-like manner.
CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Driving
Recent progress in driving video generation has shown significant potential for enhancing self-driving systems by providing scalable and controllable training data.
MC-LLaVA: Multi-Concept Personalized Vision-Language Model
To reduce the costs related to joint training, we propose a personalized textual prompt that uses visual token information to initialize concept tokens.
DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation
We further propose a personalizer enhancer during distillation to enhance the influence of embeddings on facial animation.
Concept-as-Tree: Synthetic Data is All You Need for VLM Personalization
Vision-Language Models (VLMs) have demonstrated exceptional performance in various multi-modal tasks.
On Quantizing Neural Representation for Variable-Rate Video Coding
This work introduces variable-rate INR-VC for the first time and lays a theoretical foundation for future research in rate-distortion optimization, advancing the field of video coding technology.
CMamba: Learned Image Compression with State Space Models
In this paper, we propose a hybrid Convolution and State Space Models (SSMs) based image compression framework, termed \textit{CMamba}, to achieve superior rate-distortion performance with low computational complexity.
SliceOcc: Indoor 3D Semantic Occupancy Prediction with Vertical Slice Representation
To utilize these slice features, we propose SliceOcc, an RGB camera-based model specifically tailored for indoor 3D semantic occupancy prediction.
Fast-RF-Shimming: Accelerate RF Shimming in 7T MRI using Deep Learning
This approach offers a faster and more efficient solution to RF shimming challenges in UHF MRI.
Towards Loss-Resilient Image Coding for Unstable Satellite Networks
Geostationary Earth Orbit (GEO) satellite communication demonstrates significant advantages in emergency short burst data services.
MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders
Visual encoders are fundamental components in vision-language models (VLMs), each showcasing unique strengths derived from various pre-trained visual foundation models.
Adaptive Rate Control for Deep Video Compression with Rate-Distortion Prediction
In this paper, we propose a neural network-based $\lambda$-domain rate control scheme for deep video compression, which determines the coding parameter $\lambda$ for each to-be-coded frame based on the rate-distortion-$\lambda$ (R-D-$\lambda$) relationships directly learned from uncompressed frames, achieving high rate control accuracy efficiently without the need for pre-encoding.
GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians
In this paper, we introduce a method called GraphAvatar that utilizes Graph Neural Networks (GNN) to generate 3D Gaussians for the head avatar.
ASGDiffusion: Parallel High-Resolution Generation with Asynchronous Structure Guidance
To solve the above limitations, we introduce a novel method named ASGDiffusion for parallel HR generation with Asynchronous Structure Guidance (ASG) using pre-trained diffusion models.
MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussian Splatting
We attach the 2D Gaussians to the triangular mesh of the FLAME model and connect additional 3D Gaussians to those 2D Gaussians where the rendering quality of 2DGS is inadequate, creating a mixed 2D-3D Gaussian representation.
[CLS] Attention is All You Need for Training-Free Visual Token Pruning: Make VLM Inference Faster
Most existing methods assess the importance of visual tokens based on the text-visual cross-attentions in LLMs.
EMD: Explicit Motion Modeling for High-Quality Street Gaussian Splatting
To address this, we propose Explicit Motion Decomposition (EMD), which models the motions of dynamic objects by introducing learnable motion embeddings to the Gaussians, enhancing the decomposition in street scenes.
GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting
Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets.
MC-LLaVA: Multi-Concept Personalized Vision-Language Model
To reduce the costs related to joint training, we propose a personalized textual prompt that uses visual token information to initialize concept tokens.
PLGS: Robust Panoptic Lifting with 3D Gaussian Splatting
In this paper, we propose a new method called PLGS that enables 3DGS to generate consistent panoptic segmentation masks from noisy 2D segmentation masks while maintaining superior efficiency compared to NeRF-based methods.
High-Efficiency Neural Video Compression via Hierarchical Predictive Learning
This feature-space processing operates from the lowest to the highest scale of each frame, completely eliminating the need for the complexity-intensive motion estimation and compensation techniques that have been standard in video codecs for decades.
All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation
Experimental results show that MPA achieves performance comparable to state-of-the-art methods in both task-specific and multi-objective optimization across human viewing and machine analysis tasks.
ThermalGaussian: Thermal 3D Gaussian Splatting
In this work, we propose ThermalGaussian, the first thermal 3DGS approach capable of rendering high-quality images in RGB and thermal modalities.
Accelerating block-level rate control for learned image compression
Despite the unprecedented compression efficiency achieved by deep learned image compression (LIC), existing methods usually approximate the desired bitrate by adjusting a single quality factor for a given input image, which may compromise the rate control results.
HINER: Neural Representation for Hyperspectral Image
For downstream classification on compressed HSI, we theoretically demonstrate the task accuracy is not only related to the classification loss but also to the reconstruction fidelity through a first-order expansion of the accuracy degradation, and accordingly adapt the reconstruction by introducing Adaptive Spectral Weighting.
DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstruction
With this dataset, we make the generative model more robust to cars.
3DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Views
(1) \textbf{High-Volume}: 2, 500 cars are meticulously scanned by 3D scanners, obtaining car images and point clouds with real-world dimensions; (2) \textbf{High-Quality}: Each car is captured in an average of 200 dense, high-resolution 360-degree RGB-D views, enabling high-fidelity 3D reconstruction; (3) \textbf{High-Diversity}: The dataset contains various cars from over 100 brands, collected under three distinct lighting conditions, including reflective, standard, and dark.
$\textit{S}^3$Gaussian: Self-Supervised Street Gaussians for Autonomous Driving
Photorealistic 3D reconstruction of street scenes is a critical technique for developing real-world simulators for autonomous driving.
Implicit Neural Image Field for Biological Microscopy Image Compression
The rapid pace of innovation in biological microscopy imaging has led to large images, putting pressure on data storage and impeding efficient sharing, management, and visualization.
Three-layer deep learning network random trees for fault detection in chemical production process
With the development of technology, the chemical production process is becoming increasingly complex and large-scale, making fault detection particularly important.
Towards Backward-Compatible Continual Learning of Image Compression
This paper explores the possibility of extending the capability of pre-trained neural image compressors (e. g., adapting to new data or target bitrates) without breaking backward compatibility, the ability to decode bitstreams encoded by the original model.
TinyLIC-High efficiency lossy image compression method
Image compression has been the subject of extensive research for several decades, resulting in the development of well-known standards such as JPEG, JPEG2000, and H. 264/AVC.
Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysis
To overcome this limitation in MLLMs, we introduce Proximity Question Answering (Proximity QA), a novel framework designed to enable MLLMs to infer the proximity relationship between objects in images.
Another Way to the Top: Exploit Contextual Clustering in Learned Image Coding
While convolution and self-attention are extensively used in learned image compression (LIC) for transform coding, this paper proposes an alternative called Contextual Clustering based LIC (CLIC) which primarily relies on clustering operations and local attention for correlation characterization and compact representation of an image.
Deep Covariance Alignment for Domain Adaptive Remote Sensing Image Segmentation
Compared with the existing category alignment methods, our CR aims to regularize the correlation between different dimensions of the features and thus performs more robustly when dealing with the divergent category features of imbalanced and inconsistent distributions.
RustNeRF: Robust Neural Radiance Field with Low-Quality Images
First, existing methods assume enough high-quality images are available for training the NeRF model, ignoring real-world image degradation.
Deep Hierarchical Video Compression
Recently, probabilistic predictive coding that directly models the conditional distribution of latent features across successive frames for temporal redundancy removal has yielded promising results.
MoEC: Mixture of Experts Implicit Neural Compression
However, manually designing the partition scheme for a complex scene is very challenging and fails to jointly learn the partition and INRs.
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder.
DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffuser
To address the above limitations, we propose DiffusionTalker, a diffusion-based method that utilizes contrastive learning to personalize 3D facial animation and knowledge distillation to accelerate 3D animation generation.
Frequency-Aware Re-Parameterization for Over-Fitting Based Image Compression
Over-fitting-based image compression requires weights compactness for compression and fast convergence for practical use, posing challenges for deep convolutional neural networks (CNNs) based methods.
NTO3D: Neural Target Object 3D Reconstruction with Segment Anything
NTO3D lifts the 2D masks and features of SAM into the 3D neural field for high-quality neural target object 3D reconstruction.
QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection
Multi-view 3D detection based on BEV (bird-eye-view) has recently achieved significant improvements.
Channel prior convolutional attention for medical image segmentation
An efficient Channel Prior Convolutional Attention (CPCA) method is proposed in this paper, supporting the dynamic distribution of attention weights in both channel and spatial dimensions.
ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation
Note that, our method can be regarded as a novel transfer paradigm for large-scale models, delivering promising results in adaptation to continually changing distributions.
A Comprehensive Comparison of Projections in Omnidirectional Super-Resolution
In these works, omnidirectional frames are projected from the 3D sphere to a 2D plane by Equi-Rectangular Projection (ERP).
WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal
To this end, we propose a method called Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal.
QARV: Quantization-Aware ResNet VAE for Lossy Image Compression
This paper addresses the problem of lossy image compression, a fundamental problem in image processing and information theory that is involved in many real-world applications.
Efficient Visual Computing with Camera RAW Snapshots
Conventional cameras capture image irradiance on a sensor and convert it to RGB images using an image signal processor (ISP).
BEV-SAN: Accurate BEV 3D Object Detection via Slice Attention Networks
In order to find them, we further propose a LiDAR-guided sampling strategy to leverage the statistical distribution of LiDAR to determine the heights of local slices.
BEV-LGKD: A Unified LiDAR-Guided Knowledge Distillation Framework for BEV 3D Object Detection
In this paper, we propose a unified framework named BEV-LGKD to transfer the knowledge in the teacher-student manner.
BEVUDA: Multi-geometric Space Alignments for Domain Adaptive BEV 3D Object Detection
In this paper, we propose a Multi-space Alignment Teacher-Student (MATS) framework to ease the domain shift accumulation, which consists of a Depth-Aware Teacher (DAT) and a Geometric-space Aligned Student (GAS) model.
Rate-Distortion Optimized Post-Training Quantization for Learned Image Compression
Quantizing a floating-point neural network to its fixed-point representation is crucial for Learned Image Compression (LIC) because it improves decoding consistency for interoperability and reduces space-time complexity for implementation.
Complementary consistency semi-supervised learning for 3D left atrial image segmentation
A network based on complementary consistency training, called CC-Net, has been proposed for semi-supervised left atrium image segmentation.
Lossy Image Compression with Quantized Hierarchical VAEs
Recent research has shown a strong theoretical connection between variational autoencoders (VAEs) and the rate-distortion theory.
Uncertainty Guided Depth Fusion for Spike Camera
In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera.
Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer
Neuromorphic spike data, an upcoming modality with high temporal resolution, has shown promising potential in autonomous driving by mitigating the challenges posed by high-velocity motion blur.
Efficient Meta-Tuning for Content-aware Neural Video Delivery
Our method significantly reduces the computational cost and achieves even better performance, paving the way for applying neural video delivery techniques to practical applications.
Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation
Previous works on morphable models mostly focus on large-scale facial geometry but ignore facial details.
High-Efficiency Lossy Image Coding Through Adaptive Neighborhood Information Aggregation
To this end, Integrated Convolution and Self-Attention (ICSA) unit is first proposed to form a content-adaptive transform to characterize and embed neighborhood information dynamically of any input.
Adaptive Patch Exiting for Scalable Single Image Super-Resolution
Once the incremental capacity is below the threshold, the patch can exit at the specific layer.
Opening the Black Box of Learned Image Coders
End-to-end learned lossy image coders (LICs), as opposed to hand-crafted image codecs, have shown increasing superiority in terms of the rate-distortion performance.
NFANet: A Novel Method for Weakly Supervised Water Extraction from High-Resolution Remote Sensing Imagery
Therefore, we study how to utilize point labels to extract water bodies and propose a novel method called the neighbor feature aggregation network (NFANet).
SamplingAug: On the Importance of Patch Sampling Augmentation for Single Image Super-Resolution
However, existing methods mostly train the DNNs on uniformly sampled LR-HR patch pairs, which makes them fail to fully exploit informative patches within the image.
Transformer-based Image Compression
A Transformer-based Image Compression (TIC) approach is developed which reuses the canonical variational autoencoder (VAE) architecture with paired main and hyper encoder-decoders.
Overfitting the Data: Compact Neural Video Delivery via Content-aware Feature Modulation
Internet video delivery has undergone a tremendous explosion of growth over the past few years.
Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion
We propose a compact and effective framework to fuse multimodal features at multiple layers in a single network.
Ranked #53 on
Semantic Segmentation
on NYU Depth v2
End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation
We further design a one-to-many decoder pipeline to generate multiple predictions from the CSTR, including vector-based resampling, adaptive kernel-based resampling, compensation mode selection maps and texture enhancements, and combines them adaptively to achieve more accurate inter prediction.
NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
Contextual Graph Reasoning Networks
Graph Reasoning has shown great potential recently in modeling long-range dependencies, which are crucial for various computer vision tasks.
Decomposition, Compression, and Synthesis (DCS)-based Video Coding: A Neural Exploration via Resolution-Adaptive Learning
Inspired by the facts that retinal cells actually segregate the visual scene into different attributes (e. g., spatial details, temporal motion) for respective neuronal processing, we propose to first decompose the input video into respective spatial texture frames (STF) at its native spatial resolution that preserve the rich spatial details, and the other temporal motion frames (TMF) at a lower spatial resolution that retain the motion smoothness; then compress them together using any popular video coder; and finally synthesize decoded STFs and TMFs for high-fidelity video reconstruction at the same resolution as its native input.
LID 2020: The Learning from Imperfect Data Challenge Results
The purpose of the Learning from Imperfect Data (LID) workshop is to inspire and facilitate the research in developing novel approaches that would harness the imperfect data and improve the data-efficiency during training.
Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model
Over the past two decades, traditional block-based video coding has made remarkable progress and spawned a series of well-known standards such as MPEG-4, H. 264/AVC and H. 265/HEVC.
Learned Video Compression via Joint Spatial-Temporal Correlation Exploration
Traditional video compression technologies have been developed over decades in pursuit of higher coding efficiency.
A Closed-form Solution to Universal Style Transfer
Although plenty of methods have been proposed, a theoretical analysis of feature transform is still missing.
Learned Quality Enhancement via Multi-Frame Priors for HEVC Compliant Low-Delay Applications
Networked video applications, e. g., video conferencing, often suffer from poor visual quality due to unexpected network fluctuation and limited bandwidth.
Deep Likelihood Network for Image Restoration with Multiple Degradation Levels
Convolutional neural networks have been proven effective in a variety of image restoration tasks.
Decoder Network Over Lightweight Reconstructed Feature for Fast Semantic Style Transfer
Our method decomposes the semantic style transfer problem into feature reconstruction part and feature decoder part.
RON: Reverse Connection with Objectness Prior Networks for Object Detection
To address (a), we design the reverse connection, which enables the network to detect objects on multi-levels of CNNs.
Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation
In this paper, we propose an alternative method to estimate room layouts of cluttered indoor scenes.
