Search Results for author:
Found 45 papers, 39 papers with code
Rethinking 3D Dense Caption and Visual Grounding in A Unified Framework through Prompt-based Localization
3D Visual Grounding (3DVG) and 3D Dense Captioning (3DDC) are two crucial tasks in various 3D applications, which require both shared and complementary information in localization and visual-language relationships.
AffineQuant: Affine Transformation Quantization for Large Language Models
Among these techniques, Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its noteworthy compression efficiency and cost-effectiveness in the context of training.
EBFT: Effective and Block-Wise Fine-Tuning for Sparse LLMs
For instance, on the Wikitext2 dataset with LlamaV1-7B at 70% sparsity, our proposed EBFT achieves a perplexity of 16. 88, surpassing the state-of-the-art DSnoT with a perplexity of 75. 14.
Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration
In contrast to traditional image restoration methods, all-in-one image restoration techniques are gaining increased attention for their ability to restore images affected by diverse and unknown corruption types and levels.
Learning Image Demoireing from Unpaired Real Data
The proposed method, referred to as Unpaired Demoireing (UnDeM), synthesizes pseudo moire images from unpaired datasets, generating pairs with clean images for training demoireing models.
Boosting the Cross-Architecture Generalization of Dataset Distillation through an Empirical Study
The poor cross-architecture generalization of dataset distillation greatly weakens its practical significance.
AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration
Therefore, we propose to search the optimal time steps sequence and compressed model architecture in a unified framework to achieve effective image generation for diffusion models without any further training.
A Unified Framework for 3D Point Cloud Visual Grounding
This elaborate design enables 3DRefTR to achieve both well-performing 3DRES and 3DREC capacities with only a 6% additional latency compared to the original 3DREC model.
Spatial Re-parameterization for N:M Sparsity
This paper presents a Spatial Re-parameterization (SpRe) method for the N:M sparsity in CNNs.
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers
Token compression aims to speed up large-scale vision transformers (e. g. ViTs) by pruning (dropping) or merging tokens.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
MultiQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantization
However, in this paper, we investigate existing methods and observe a significant accumulation of quantization errors caused by frequent bit-width switching of weights and activations, leading to limited performance.
Distribution-Flexible Subset Quantization for Post-Quantizing Super-Resolution Networks
Compared to the common iterative exhaustive search algorithm, our strategy avoids the enumeration of all possible combinations in the universal set, reducing the time complexity from exponential to linear.
Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective
In particular, we first formulate the oscillation in PTQ and prove the problem is caused by the difference in module capacity.
Bi-directional Masks for Efficient N:M Sparse Training
We focus on addressing the dense backward propagation issue for training efficiency of N:M fine-grained sparsity that preserves at most N out of M consecutive weights and achieves practical speedups supported by the N:M sparse tensor core.
Real-Time Image Demoireing on Mobile Devices
In this paper, we launch the first study on accelerating demoireing networks and propose a dynamic demoireing acceleration method (DDA) towards a real-time deployment on mobile devices.
Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle
In this paper, we try to solve this problem by introducing a principled and unified framework based on Information Bottleneck (IB) theory, which further guides us to an automatic pruning approach.
Discriminator-Cooperated Feature Map Distillation for GAN Compression
In contrast to conventional pixel-to-pixel match methods in feature map distillation, our DCD utilizes teacher discriminator as a transformation to drive intermediate results of the student generator to be perceptually close to corresponding outputs of the teacher generator.
SMMix: Self-Motivated Image Mixing for Vision Transformers
Due to the subtle designs of the self-motivated paradigm, our SMMix is significant in its smaller training overhead and better performance than other CutMix variants.
Exploring Content Relationships for Distilling Efficient GANs
This paper proposes a content relationship distillation (CRD) to tackle the over-parameterized generative adversarial networks (GANs) for the serviceability in cutting-edge devices.
Shadow Removal by High-Quality Shadow Synthesis
Specifically, the encoder extracts the shadow feature of a region identity which is then paired with another region identity to serve as the generator input to synthesize a pseudo image.
Meta Architecture for Point Cloud Analysis
However, the lack of a unified framework to interpret those networks makes any systematic comparison, contrast, or analysis challenging, and practically limits healthy development of the field.
Ranked #2 on
3D Semantic Segmentation
on OpenTrench3D
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training
In QAT, the contemporary experience is that all quantized weights are updated for an entire training process.
LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal
Finally, the LSA modules are further developed to fully use the prior information in non-shadow regions to cleanse the shadow regions.
Learning Best Combination for Efficient N:M Sparsity
In this paper, we show that the N:M learning can be naturally characterized as a combinatorial problem which searches for the best combination candidate within a finite collection.
Shadow-Aware Dynamic Convolution for Shadow Removal
Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module in a computationally cheap manner and recovers the shadow region with a more complicated convolution module to ensure the quality of image reconstruction.
Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks
However, these methods suffer from severe performance degradation when quantizing the SR models to ultra-low precision (e. g., 2-bit and 3-bit) with the low-cost layer-wise quantizer.
CF-ViT: A General Coarse-to-Fine Method for Vision Transformer
Our proposed CF-ViT is motivated by two important observations in modern ViT models: (1) The coarse-grained patch splitting can locate informative regions of an input image.
OptG: Optimizing Gradient-driven Criteria in Network Sparsity
We prove that supermask training is to accumulate the criteria of gradient-driven sparsity for both removed and preserved weights, and it can partly solve the independence paradox.
Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization
PSNR-oriented models are a critical class of super-resolution models with applications across various fields.
Neural Architecture Search With Representation Mutual Information
Building upon RMI, we further propose a new search algorithm termed RMI-NAS, facilitating with a theorem to guarantee the global optimal of the searched architecture.
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme
In this work, we revisit the role of discriminator in GAN compression and design a novel generator-discriminator cooperative compression scheme for GAN compression, termed GCC.
Fine-grained Data Distribution Alignment for Post-Training Quantization
While post-training quantization receives popularity mostly due to its evasion in accessing the original complete training dataset, its poor performance also stems from scarce images.
Error Controlled Actor-Critic
On error of value function inevitably causes an overestimation phenomenon and has a negative impact on the convergence of the algorithms.
Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion
Each filter in our DCFF is firstly given an inter-similarity distribution with a temperature parameter as a filter proxy, on top of which, a fresh Kullback-Leibler divergence based dynamic-coded criterion is proposed to evaluate the filter importance.
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.
1xN Pattern for Pruning Convolutional Neural Networks
We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.
Towards Compact CNNs via Collaborative Compression
Channel pruning and tensor decomposition have received extensive attention in convolutional neural network compression.
Lottery Jackpots Exist in Pre-trained Models
Existing studies achieve the sparsity of neural networks via time-consuming weight training or complex searching on networks with expanded width, which greatly limits the applications of network pruning.
SiMaN: Sign-to-Magnitude Network Binarization
In this paper, we show that our weight binarization provides an analytical solution by encoding high-magnitude weights into +1s, and 0s otherwise.
Error Controlled Actor-Critic Method to Reinforcement Learning
In the reinforcement learning (RL) algorithms which incorporate function approximation methods, the approximation error of value function inevitably cause overestimation phenomenon and have a negative impact on the convergence of the algorithms.
Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillation
The latter simultaneously distills informative attention maps from both the generator and discriminator of a pre-trained model to the searched generator, effectively stabilizing the adversarial training of our light-weight model.
Task Augmentation by Rotating for Meta-Learning
Data augmentation is one of the most effective approaches for improving the accuracy of modern machine learning models, and it is also indispensable to train a deep model for meta-learning.
Stock Prices Prediction using Deep Learning Models
The results show that predicting stock price through price rate of change is better than predicting absolute prices directly.
Decoder Choice Network for Meta-Learning
This work proposes a method that controls the gradient descent process of the model parameters of a neural network by limiting the model parameters in a low-dimensional latent space.
Gradient Boost with Convolution Neural Network for Stock Forecast
Market economy closely connects aspects to all walks of life.
