Search Results for author:
Found 95 papers, 34 papers with code
Towards Content-Independent Multi-Reference Super-Resolution: Adaptive Pattern Matching and Feature Aggregation
Recovering realistic textures from a largely down-sampled low resolution (LR) image with complicated patterns is a challenging problem in image super-resolution.
Bridging Video Quality Scoring and Justification via Large Multimodal Models
The automated pipeline eliminates the reliance on expert-written quality descriptions and proprietary systems, ensuring data scalability and generation efficiency.
Recognizing Surgical Phases Anywhere: Few-Shot Test-time Adaptation and Task-graph Guided Refinement
SPA is a lightweight adaptation framework, allowing hospitals to rapidly customize phase recognition models by defining phases in natural language text, annotating a few images with the phase labels, and providing a task graph defining phase transitions.
Efficient Long-Context LLM Inference via KV Cache Clustering
To enable efficient clustering, we divide the sequence into chunks and propose Chunked Soft Matching, which employs an alternating partition strategy within each chunk and identifies clusters based on similarity.
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
Large Language Models (LLMs) have emerged as a pivotal research area, yet the attention module remains a critical bottleneck in LLM inference, even with techniques like KVCache to mitigate redundant computations.
EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy
In endoscopic procedures, autonomous tracking of abnormal regions and following circumferential cutting markers can significantly reduce the cognitive burden on endoscopists.
ORQA: A Benchmark and Foundation Model for Holistic Operating Room Modeling
We show the strong performance of ORQA on our proposed benchmark, and its zero-shot generalization, paving the way for scalable, unified OR modeling and significantly advancing multimodal surgical intelligence.
Diffusion Learning with Partial Agent Participation and Local Updates
Diffusion learning is a framework that endows edge devices with advanced intelligence.
NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Study
In this work, we build the first benchmark dataset for short-form UGC Image Super-resolution in the wild, termed KwaiSR, intending to advance the research on developing image super-resolution algorithms for short-form UGC platforms.
NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Results
This paper presents a review for the NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement.
Can DeepSeek Reason Like a Surgeon? An Empirical Evaluation for Vision-Language Understanding in Robotic-Assisted Surgery
In this study, we investigate the dialogue capabilities of the DeepSeek model in robotic surgery scenarios, focusing on tasks such as Single Phrase QA, Visual QA, and Detailed Description.
Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance
To solve this problem, we convert the multi-objective constraints to a single-objective constraint through a merit function with an easy-to-evaluate gradient, and then, we use a penalty-based reformulation of the bilevel optimization problem.
Optimal Complexity in Byzantine-Robust Distributed Stochastic Optimization with Data Heterogeneity
We establish the lower bounds on the Byzantine error and on the minimum number of queries to a stochastic gradient oracle required to achieve an arbitrarily small optimization error.
KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception
Inspired by the Human Visual System (HVS) that links global quality to the local texture of different regions and their visual saliency, we propose a Kaleidoscope Video Quality Assessment (KVQ) framework, which aims to effectively assess both saliency and local texture, thereby facilitating the assessment of global quality.
MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments
Operating rooms (ORs) are complex, high-stakes environments requiring precise understanding of interactions among medical staff, tools, and equipment for enhancing surgical assistance, situational awareness, and patient safety.
Ranked #1 on
Video Panoptic Segmentation
on 4D-OR
(using extra training data)
A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
In this work, we introduce a Randomized Subspace Optimization framework for pre-training and fine-tuning LLMs.
Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment
Experiments on the IU X-RAY and MIMIC-CXR radiology datasets demonstrate that Adversarial Domain Alignment enables attackers to steal the medical MLLM without any access to medical data.
Visual Autoregressive Modeling for Image Super-Resolution
Image Super-Resolution (ISR) has seen significant progress with the introduction of remarkable generative models.
EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery
Recently, Multimodal Large Language Models (MLLMs) have demonstrated their immense potential in computer-aided diagnosis and decision-making.
Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis
Conclusion: We propose a text-driven adaptation approach that mitigates the modality gap and handles multiple downstream tasks in surgical workflow analysis, with minimal reliance on large annotated datasets.
Breaking Memory Limits: Gradient Wavelet Transform Enhances LLMs Training
Large language models (LLMs) have shown impressive performance across a range of natural language processing tasks.
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Surgical practice involves complex visual interpretation, procedural skills, and advanced medical knowledge, making surgical vision-language pretraining (VLP) particularly challenging due to this complexity and the limited availability of annotated data.
SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
To address these limitations, this paper proposes SPARKLE, a unified Single-loop Primal-dual AlgoRithm frameworK for decentraLized bilEvel optimization.
Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise
This paper investigates the roles of gradient normalization and clipping in ensuring the convergence of Stochastic Gradient Descent (SGD) under heavy-tailed noise.
Subspace Optimization for Large Language Models with Convergence Guarantees
Subspace optimization algorithms, such as GaLore (Zhao et al., 2024), have gained attention for pre-training and fine-tuning large language models (LLMs) due to their memory efficiency.
Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures
However, traditional first-order (FO) fine-tuning algorithms incur substantial memory overhead due to the need to store activation values for back-propagation during gradient computation, particularly in long-context fine-tuning tasks.
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation
Surgical video-language pretraining (VLP) faces unique challenges due to the knowledge domain gap and the scarcity of multi-modal data.
S$^3$Attention: Improving Long Sequence Attention with Smoothed Skeleton Sketching
Attention based models have achieved many remarkable breakthroughs in numerous applications.
QPT V2: Masked Image Modeling Advances Visual Scoring
To this end, we propose Quality- and aesthetics-aware pretraining (QPT V2), the first pretraining framework based on MIM that offers a unified solution to quality and aesthetics assessment.
On the Trade-off between Flatness and Optimization in Distributed Learning
This paper proposes a theoretical framework to evaluate and compare the performance of gradient-descent algorithms for distributed learning in relation to their behavior around local minima in nonconvex environments.
PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild
In this paper, we propose a VQA method named PTM-VQA, which leverages PreTrained Models to transfer knowledge from models pretrained on various pre-tasks, enabling benefits for VQA from different aspects.
HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition
By disentangling embedding spaces of different hierarchical levels, the learned multi-modal representations encode short-term and long-term surgical concepts in the same model.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
NTIRE 2024 Challenge on Image Super-Resolution ($\times$4): Methods and Results
This paper reviews the NTIRE 2024 challenge on image super-resolution ($\times$4), highlighting the solutions proposed and the outcomes obtained.
QNCD: Quantization Noise Correction for Diffusion Models
Diffusion models have revolutionized image synthesis, setting new benchmarks in quality and creativity.
Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model
Based on a large-scale pre-trained Vision Language Model (VLM), our model learns and improves visual, textual, and numerical representations of patient gait videos, through a collective learning across three distinct modalities: gait videos, class-specific descriptions, and numerical gait parameters.
CasSR: Activating Image Power for Real-World Image Super-Resolution
In particular, we develop a cascaded controllable diffusion model that aims to optimize the extraction of information from low-resolution images.
XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution
Diffusion-based methods, endowed with a formidable generative prior, have received increasing attention in Image Super-Resolution (ISR) recently.
KVQ: Kwai Video Quality Assessment for Short-form Videos
Short-form UGC video platforms, like Kwai and TikTok, have been an emerging and irreplaceable mainstream media form, thriving on user-friendly engagement, and kaleidoscope creation, etc.
Asynchronous Diffusion Learning with Agent Subsampling and Local Updates
In this work, we examine a network of agents operating asynchronously, aiming to discover an ideal global model that suits individual local datasets.
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
This provides the first theoretical understanding of how network topology, data heterogeneity, and nested bilevel structures influence decentralized SBO.
Advancing Surgical VQA with Scene Graph Knowledge
We highlight that the primary limitation in the current surgical VQA systems is the lack of scene knowledge to answer complex queries.
Model-free Test Time Adaptation for Out-Of-Distribution Detection
We demonstrate the effectiveness of \abbr through comprehensive experiments on multiple OOD detection benchmarks, extensive empirical studies show that \abbr significantly improves the performance of OOD detection over state-of-the-art methods.
Achieving Linear Speedup with ProxSkip in Distributed Stochastic Optimization
We demonstrate that the leading communication complexity of ProxSkip is $\mathcal{O}(\frac{p\sigma^2}{n\epsilon^2})$ for non-convex and convex settings, and $\mathcal{O}(\frac{p\sigma^2}{n\epsilon})$ for the strongly convex setting, where $n$ represents the number of nodes, $p$ denotes the probability of communication, $\sigma^2$ signifies the level of stochastic noise, and $\epsilon$ denotes the desired accuracy level.
BEVHeight++: Toward Robust Visual Centric 3D Object Detection
In essence, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods.
Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment
Video quality assessment (VQA) has attracted growing attention in recent years.
Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessment
\textit{Second}, the perceptual quality of a video exhibits a multi-distortion distribution, due to the differences in the duration and probability of occurrence for various distortions.
Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures
We then present a novel method, SurgVLP - Surgical Vision Language Pre-training, for multi-modal representation learning.
NTIRE 2023 Quality Assessment of Video Enhancement Challenge
61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions.
Momentum Benefits Non-IID Federated Learning Simply and Provably
When all clients participate in the training process, we demonstrate that incorporating momentum allows FedAvg to converge without relying on the assumption of bounded data heterogeneity even using a constant local learning rate.
DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm
Their communication, governed by the communication topology and gossip weight matrices, facilitates the exchange of model updates.
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
Our results reveal that using independent unbiased compression can reduce the total communication cost by a factor of up to $\Theta(\sqrt{\min\{n, \kappa\}})$ when all local smoothness constants are constrained by a common upper bound, where $n$ is the number of workers and $\kappa$ is the condition number of the functions being minimized.
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression
In this paper, we investigate the performance limit of distributed stochastic optimization algorithms employing communication compression.
AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation
In particular, when the adaptation target is a series of domains, the adaptation accuracy of AdaNPC is 50% higher than advanced TTA methods.
Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessment
Video quality assessment (VQA) aims to simulate the human perception of video quality, which is influenced by factors ranging from low-level color and texture details to high-level semantic content.
BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection
In essence, instead of predicting the pixel-wise depth, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods.
Ranked #3 on
3D Object Detection
on Rope3D
Quality-aware Pre-trained Models for Blind Image Quality Assessment
Blind image quality assessment (BIQA) aims to automatically evaluate the perceived quality of a single image, whose performance has been improved by deep learning-based methods in recent years.
CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection.
Ranked #1 on
Action Triplet Detection
on CholecT50 (Challenge)
Optimal Complexity in Non-Convex Decentralized Learning over Time-Varying Networks
The main difficulties lie in how to gauge the effectiveness when transmitting messages between two nodes via time-varying communications, and how to establish the lower bound when the network size is fixed (which is a prerequisite in stochastic optimization).
Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization
While (Lu and Sa, 2021) have recently provided an optimal rate for non-convex stochastic decentralized optimization with weight matrices defined over linear graphs, the optimal rate with general weight matrices remains unclear.
Communication-Efficient Topologies for Decentralized Learning with $O(1)$ Consensus Rate
In the proposed family, EquiStatic has a degree of $\Theta(\ln(n))$, where $n$ is the network size, and a series of time-dependent one-peer topologies, EquiDyn, has a constant degree of 1.
On the Performance of Gradient Tracking with Local Updates
We study the decentralized optimization problem where a network of $n$ agents seeks to minimize the average of a set of heterogeneous non-convex cost functions distributedly.
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression
We establish a convergence lower bound for algorithms whether using unbiased or contractive compressors in unidirection or bidirection.
Heavy-Tail Phenomenon in Decentralized SGD
To have a more explicit control on the tail exponent, we then consider the case where the loss at each node is a quadratic, and show that the tail-index can be estimated as a function of the step-size, batch-size, and the topological properties of the network of the computational nodes.
ShowFace: Coordinated Face Inpainting with Memory-Disentangled Refinement Networks
Furthermore, to better improve the inter-coordination between the corrupted and non-corrupted regions and enhance the intra-coordination in corrupted regions, we design InCo2 Loss, a pair of similarity based losses to constrain the feature consistency.
CHEX: CHannel EXploration for CNN Model Compression
However, conventional pruning methods have limitations in that: they are restricted to pruning process only, and they require a fully pre-trained large model.
An Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders
In this paper, we will improve the convergence analysis and rates of variance reduction under without-replacement sampling orders for composite finite-sum minimization. Our results are in two-folds.
Self-supervised Correlation Mining Network for Person Image Generation
However, such methods fail to exploit the spatial correlation between the disentangled features.
BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learning
On mainstream DNN training tasks, BlueFog reaches a much higher throughput and achieves an overall $1. 2\times \sim 1. 8\times$ speedup over Horovod, a state-of-the-art distributed deep learning package based on Ring-Allreduce.
Exponential Graph is Provably Efficient for Decentralized Deep Training
Experimental results on a variety of tasks and models demonstrate that decentralized (momentum) SGD over exponential graphs promises both fast and high-quality training.
Communicate Then Adapt: An Effective Decentralized Adaptive Method for Deep Training
Decentralized adaptive gradient methods, in which each node averages only with its neighbors, are critical to save communication and wall-clock training time in deep learning tasks.
Decentralized Composite Optimization with Compression
Decentralized optimization and communication compression have exhibited their great potential in accelerating distributed machine learning by mitigating the communication bottleneck in practice.
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
It addresses the shortcomings of the previous works by repeatedly growing a subset of layers to dense and then pruning them back to sparse after some training.
Accelerating Gossip SGD with Periodic Global Averaging
Communication overhead hinders the scalability of large-scale distributed training.
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
For smooth objective functions, the transient stage (which measures the number of iterations the algorithm has to experience before achieving the linear speedup stage) of D-SGD is on the order of ${\Omega}(n/(1-\beta)^2)$ and $\Omega(n^3/(1-\beta)^4)$ for strongly and generally convex cost functions, respectively, where $1-\beta \in (0, 1)$ is a topology-dependent quantity that approaches $0$ for a large and sparse network.
Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders
In the highly data-heterogeneous scenario, Prox-DFinito with optimal cyclic sampling can attain a sample-size-independent convergence rate, which, to our knowledge, is the first result that can match with uniform-iid-sampling with variance reduction.
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training
Experimental results on a variety of computer vision tasks and models demonstrate that DecentLaM promises both efficient and high-quality training.
Differentiable Network Adaption with Elastic Search Space
In this paper we propose a novel network adaption method called Differentiable Network Adaption (DNA), which can adapt an existing network to a specific computation budget by adjusting the width and depth in a differentiable manner.
Incorporating Convolution Designs into Visual Transformers
Motivated by the success of Transformers in natural language processing (NLP) tasks, there emerge some attempts (e. g., ViT and DeiT) to apply Transformers to the vision domain.
Ranked #1 on
Image Classification
on Oxford-IIIT Pets
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs.
Differentiable Dynamic Wirings for Neural Networks
A standard practice of deploying deep neural networks is to apply the same architecture to all the input instances.
Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks
To facilitate the training, we represent the network connectivity of each sample in an adjacency matrix.
Learning Connectivity of Neural Networks from a Topological Perspective
In this paper, we attempt to optimize the connectivity in neural networks.
Towards Good Practices for Instance Segmentation
Instance Segmentation is an interesting yet challenging task in computer vision.
ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems and GANs
Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood.
Diving into Optimization of Topology in Neural Networks
Seeking effective networks has become one of the most crucial and practical areas in deep learning.
ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems
Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood.
On the Influence of Bias-Correction on Distributed Stochastic Optimization
It is still unknown {\em whether}, {\em when} and {\em why} these bias-correction methods can outperform their traditional counterparts (such as consensus and diffusion) with noisy gradient and constant step-sizes.
Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates
In this scenario, agents collaborate to estimate the value function of a target team policy.
Supervised Learning Under Distributed Features
This work studies the problem of learning under both large datasets and large-dimensional feature space scenarios.
Stochastic Learning under Random Reshuffling with Constant Step-sizes
In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly.
Variance-Reduced Stochastic Learning under Random Reshuffling
First, it resolves this open issue and provides the first theoretical guarantee of linear convergence under random reshuffling for SAGA; the argument is also adaptable to other variance-reduced algorithms.
Variance-Reduced Stochastic Learning by Networked Agents under Random Reshuffling
For such situations, the balanced gradient computation property of AVRG becomes a real advantage in reducing idle time caused by unbalanced local data storage requirements, which is characteristic of other reduced-variance gradient algorithms.
On the Influence of Momentum Acceleration on Online Learning
The article examines in some detail the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime.
Online Dual Coordinate Ascent Learning
The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees.
