Search Results for author:
Found 81 papers, 25 papers with code
SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting
3D Gaussian Splatting (3DGS) serves as a highly performant and efficient encoding of scene geometry, appearance, and semantics.
HiSin: Efficient High-Resolution Sinogram Inpainting via Resolution-Guided Progressive Inference
High-resolution sinogram inpainting is essential for computed tomography reconstruction, as missing high-frequency projections can lead to visible artifacts and diagnostic errors.
POLARIS: A High-contrast Polarimetric Imaging Benchmark Dataset for Exoplanetary Disk Representation Learning
With over 1, 000, 000 images from more than 10, 000 exposures using state-of-the-art high-contrast imagers (e. g., Gemini Planet Imager, VLT/SPHERE) in the search for exoplanets, can artificial intelligence (AI) serve as a transformative tool in imaging Earth-like exoplanets in the coming decade?
3D Skeleton-Based Action Recognition: A Review
To bridge this gap, our review aims to address these limitations by presenting a comprehensive, task-oriented framework for understanding skeleton-based action recognition.
Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individuals
While virtual try-on (VTON) systems aim to render a garment onto a target person image, this paper tackles the novel task of virtual try-off (VTOFF), which addresses the inverse problem: generating standardized product images of garments from real-world photos of clothed individuals.
Manifold-aware Representation Learning for Degradation-agnostic Image Restoration
In this work, we present MIRAGE, a unified and lightweight framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches, each processed by a specialized module attention for global context, convolution for local textures, and MLP for channel-wise statistics.
Self-Supervised and Generalizable Tokenization for CLIP-Based 3D Understanding
Vision-language models like CLIP can offer a promising foundation for 3D scene understanding when extended with 3D tokenizers.
MLLMs are Deeply Affected by Modality Bias
These findings highlight the need for balanced training strategies and model architectures to better integrate multiple modalities in MLLMs.
Adversarial Robustness for Unified Multi-Modal Encoders via Efficient Calibration
Experiments on six modalities and three Bind-style models show that our method improves adversarial robustness by up to 47. 3 percent at epsilon = 4/255, while preserving or even improving clean zero-shot and retrieval performance with less than 1 percent trainable parameters.
Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation
Restoring any degraded image efficiently via just one model has become increasingly significant and impactful, especially with the proliferation of mobile devices.
The Tenth NTIRE 2025 Image Denoising Challenge Report
This paper presents an overview of the NTIRE 2025 Image Denoising Challenge ({\sigma} = 50), highlighting the proposed methodologies and corresponding results.
NTIRE 2025 Challenge on Cross-Domain Few-Shot Object Detection: Methods and Results
Cross-Domain Few-Shot Object Detection (CD-FSOD) poses significant challenges to existing object detection and few-shot detection models when applied across domains.
Cross-Domain Few-Shot
Cross-Domain Few-Shot Object Detection
+3
The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Report
This paper presents a comprehensive review of the NTIRE 2025 Challenge on Single-Image Efficient Super-Resolution (ESR).
Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook
Retrieval-augmented generation (RAG) has emerged as a pivotal technique in artificial intelligence (AI), particularly in enhancing the capabilities of large language models (LLMs) by enabling access to external, reliable, and up-to-date knowledge sources.
SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretraining
In order to power the proposed methods, we introduce SceneSplat-7K, the first large-scale 3DGS dataset for indoor scenes, comprising of 6868 scenes derived from 7 established datasets like ScanNet, Matterport3D, etc.
Fractal-IR: A Unified Framework for Efficient and Scalable Image Restoration
While vision transformers achieve significant breakthroughs in various image restoration (IR) tasks, it is still challenging to efficiently scale them across multiple types of degradations and resolutions.
CityLoc: 6DoF Pose Distributional Localization for Text Descriptions in Large-Scale Scenes with Gaussian Representation
Localizing textual descriptions within large-scale 3D scenes presents inherent ambiguities, such as identifying all traffic lights in a city.
ROLO-SLAM: Rotation-Optimized LiDAR-Only SLAM in Uneven Terrain with Ground Vehicle
In this article, a LiDAR-based SLAM method is presented to improve the accuracy of pose estimations for ground vehicles in rough terrains, which is termed Rotation-Optimized LiDAR-Only (ROLO) SLAM.
Hierarchical Information Flow for Generalized Efficient Image Restoration
To strike a balance between efficiency and model capacity for a generalized transformer-based IR method, we propose a hierarchical information flow mechanism for image restoration, dubbed Hi-IR, which progressively propagates information among pixels in a bottom-up manner.
Ranked #1 on
Image Deblurring
on HIDE (trained on GOPRO)
Hierarchical Cross-Attention Network for Virtual Try-On
In this paper, we present an innovative solution for the challenges of the virtual try-on task: our novel Hierarchical Cross-Attention Network (HCANet).
PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection
The integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection.
FCDM: A Physics-Guided Bidirectional Frequency Aware Convolution and Diffusion-Based Model for Sinogram Inpainting
Computed tomography (CT) is widely used in industrial and medical imaging, but sparse-view scanning reduces radiation exposure at the cost of incomplete sinograms and challenging reconstruction.
Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalities
By transferring knowledge from teacher to student, the student network can better adapt to complex dynamic scenes with incomplete observations.
ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining
In particular, we show that (1) the distribution of the optimized GS centroids significantly differs from the uniformly sampled point cloud (used for initialization) counterpart; (2) this change in distribution results in degradation in classification but improvement in segmentation tasks when using only the centroids; (3) to leverage additional Gaussian parameters, we propose Gaussian feature grouping in a normalized feature space, along with splats pooling layer, offering a tailored solution to effectively group and embed similar Gaussians, which leads to notable improvement in finetuning tasks.
Restore Anything Model via Efficient Degradation Adaptation
With the proliferation of mobile devices, the need for an efficient model to restore any degraded image has become increasingly significant and impactful.
Ranked #5 on
5-Degradation Blind All-in-One Image Restoration
on 5-Degradation Blind All-in-One Image Restoration
5-Degradation Blind All-in-One Image Restoration
Benchmarking
+2
Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning
To answer this question, we first empirically validate that integrating MAE-based point cloud pre-training with the standard contrastive learning paradigm, even with meticulous design, can lead to a decrease in performance.
Sharing Key Semantics in Transformer Makes Efficient Image Restoration
Additionally, for IR, it is commonly noted that small segments of a degraded image, particularly those closely aligned semantically, provide particularly relevant information to aid in the restoration process, as they contribute essential contextual cues crucial for accurate reconstruction.
SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
Focusing on emerging transformers (specifically the ones with computationally efficient Swin-like architectures) and large models (e. g., Stable Diffusion and LLMs) based on transformers, we observe that layout transformations between the computational operators cause a significant slowdown in these applications.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
Socially Pertinent Robots in Gerontological Healthcare
Despite the many recent achievements in developing and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary.
SoD$^2$: Statically Optimizing Dynamic Deep Neural Network
Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs.
Key-Graph Transformer for Image Restoration
While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution.
Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation
We observe that previous optimization-based methods commonly rely on projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem.
Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)
Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses.
ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models
To address this, we construct scene graphs to capture spatial relationships among objects and apply a graph matching algorithm to these graphs to accurately identify matched objects.
MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards
Sparse rewards pose a significant challenge to achieving high sample efficiency in goal-conditioned reinforcement learning (RL).
Towards High-Quality and Efficient Video Super-Resolution via Spatial-Temporal Data Overfitting
To reconcile such, we propose a novel method for high-quality and efficient video resolution upscaling tasks, which leverages the spatial-temporal information to accurately divide video into chunks, thus keeping the number of chunks as well as the model size to minimum.
Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models
Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains.
Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge
With the ever-increasing popularity of edge devices, it is necessary to implement real-time segmentation on the edge for autonomous driving and many other applications.
Towards Reliable Item Sampling for Recommendation Evaluation
Since Rendle and Krichene argued that commonly used sampling-based evaluation metrics are "inconsistent" with respect to the global metrics (even in expectation), there have been a few studies on the sampling-based recommender system evaluation.
Deep Unsupervised Key Frame Extraction for Efficient Video Classification
The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically.
SparCL: Sparse Continual Learning on the Edge
SparCL achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity.
Survey: Exploiting Data Redundancy for Optimization of Deep Learning
It surveys hundreds of recent papers on the topic, introduces a novel taxonomy to put the various techniques into a single categorization framework, offers a comprehensive description of the main methods used for exploiting data redundancy in improving multiple kinds of DNNs on data, and points out a set of research opportunities for future to explore.
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution
Instead of measuring the speed on mobile devices at each iteration during the search process, a speed model incorporated with compiler optimizations is leveraged to predict the inference latency of the SR block with various width configurations for faster convergence.
PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation
For semantic-guided cross-view image translation, it is crucial to learn where to sample pixels from the source view image and where to reallocate them guided by the target view semantic map, especially when there is little overlap or drastic view difference between the source and target images.
CoCoPIE XGen: A Full-Stack AI-Oriented Optimizing Framework
There is a growing demand for shifting the delivery of AI capability from data centers on the cloud to edge or end devices, exemplified by the fast emerging real-time AI-based apps running on smartphones, AR/VR devices, autonomous vehicles, and various IoT devices.
Real-Time Portrait Stylization on the Edge
In this work we demonstrate real-time portrait stylization, specifically, translating self-portrait into cartoon or anime style on mobile devices.
Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformers
In particular, MJP first shuffles the selected patches via our block-wise random jigsaw puzzle shuffle algorithm, and their corresponding PEs are occluded.
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning
Moreover, our framework can guarantee the identified model to meet resource specifications of mobile devices and FPGA, and even achieve the real-time execution of DeiT-T on mobile platforms.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices.
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Systematical evaluation on accuracy, training speed, and memory footprint are conducted, where the proposed MEST framework consistently outperforms representative SOTA works.
Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation
To ease this problem, we propose a novel two-stage framework with a new Cascaded Cross MLP-Mixer (CrossMLP) sub-network in the first stage and one refined pixel-level loss in the second stage.
Enabling Level-4 Autonomous Driving on a Single $1k Off-the-Shelf Card
Autonomous driving is of great interest in both research and industry.
HFSP: A Hardware-friendly Soft Pruning Framework for Vision Transformers
Recently, Vision Transformer (ViT) has continuously established new milestones in the computer vision field, while the high computation and memory cost makes its propagation in industrial production difficult.
On the regularization landscape for the linear recommendation models
Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders several standard recommendation benchmarks.
DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion
Deep Neural Networks (DNNs) have emerged as the core enabler of many major applications on mobile devices.
GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity
It necessitates the sparse model inference via weight pruning, i. e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy.
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
Though recent years have witnessed remarkable progress in single image super-resolution (SISR) tasks with the prosperous development of deep neural networks (DNNs), the deep learning methods are confronted with the computation and memory consumption issues in practice, especially for resource-limited platforms such as mobile devices.
Achieving Real-Time Object Detection on MobileDevices with Neural Pruning Search
Object detection plays an important role in self-driving cars for security development.
Towards Fast and Accurate Multi-Person Pose Estimation on Mobile Devices
The rapid development of autonomous driving, abnormal behavior detection, and behavior recognition makes an increasing demand for multi-person pose estimation-based applications, especially on mobile platforms.
A Compression-Compilation Framework for On-mobile Real-time BERT Applications
In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices.
Cloth Interactive Transformer for Virtual Try-On
In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask.
Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device
3D object detection is an important task, especially in the autonomous driving application domain.
NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration
With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed.
ClickTrain: Efficient and Accurate End-to-End Deep Learning Training via Fine-Grained Architecture-Preserving Pruning
Moreover, compared with the state-of-the-art pruning-during-training approach, ClickTrain provides significant improvements both accuracy and compression ratio on the tested CNN models and datasets, under similar limited training time.
On Efficient Constructions of Checkpoints
Efficient construction of checkpoints/snapshots is a critical tool for training and diagnosing deep learning models.
Real-Time Execution of Large-scale Language Models on Mobile
Our framework can guarantee the identified model to meet both resource and real-time specifications of mobile devices, thus achieving real-time execution of large transformer-based models like BERT variants.
YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design
In this work, we propose YOLObile framework, a real-time object detection on mobile devices via compression-compilation co-design.
RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices
The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism.
Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization
High-end mobile platforms rapidly serve as primary computing devices for a wide range of Deep Neural Network (DNN) applications.
CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way
Assuming hardware is the major constraint for enabling real-time mobile intelligence, the industry has mainly dedicated their efforts to developing specialized hardware accelerators for machine learning and inference.
A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices.
RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Survey on 3D Skeleton-Based Action Recognition Using Learning Method
To the best of our knowledge, this research represents the first comprehensive discussion of deep learning-based action recognition using 3D skeleton data.
BLK-REW: A Unified Block-based DNN Pruning Framework using Reweighted Regularization Method
Accelerating DNN execution on various resource-limited computing platforms has been a long-standing problem.
An Image Enhancing Pattern-based Sparsity for Real-time Inference on Mobile Devices
Weight pruning has been widely acknowledged as a straightforward and effective method to eliminate redundancy in Deep Neural Networks (DNN), thereby achieving acceleration on various platforms.
Using Data Imputation for Signal Separation in High Contrast Imaging
We apply it to simulated point source and circumstellar disk observations to demonstrate its proper recovery of them.
PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
Weight pruning of DNNs is proposed, but existing schemes represent two extremes in the design space: non-structured pruning is fine-grained, accurate, but not hardware friendly; structured pruning is coarse-grained, hardware-efficient, but with higher accuracy loss.
PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method.
An Exo-Kuiper Belt and An Extended Halo around HD 191089 in Scattered Light
We have obtained Hubble Space Telescope STIS and NICMOS, and Gemini/GPI scattered light images of the HD 191089 debris disk.
Earth and Planetary Astrophysics Solar and Stellar Astrophysics
26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone
With the rapid emergence of a spectrum of high-end mobile devices, many applications that required desktop-level computation capability formerly can now run on these devices without any problem.
