Search Results for author:
Found 27 papers, 8 papers with code
DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models
We evaluate our method on various LLMs, including OPT, LLaMA, LLaMA-2, Phi-1. 5, and Phi-2.
Unlocking Memorization in Large Language Models with Dynamic Soft Prompting
Pretrained large language models (LLMs) have revolutionized natural language processing (NLP) tasks such as summarization, question answering, and translation.
MoDeGPT: Modular Decomposition for Large Language Model Compression
Large Language Models (LLMs) have reshaped the landscape of artificial intelligence by demonstrating exceptional performance across various tasks.
Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models
Each architecture code represents a specialized model tailored to the prompts assigned to it, and the number of codes is a hyperparameter.
Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment
We address this challenge by designing a mechanism to model the complex changing dynamics of the reward function and provide a representation of it to the RL agent.
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
Device-Wise Federated Network Pruning
Neural network pruning particularly channel pruning is a widely used technique for compressing deep learning models to enable their deployment on edge devices with limited resources.
BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks
Most existing dynamic or runtime channel pruning methods have to store all weights to achieve efficient inference which brings extra storage costs.
Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold
Thus, they neglect the critical characteristic of GANs: their local density structure over their learned manifold.
Token Fusion: Bridging the Gap between Token Pruning and Token Merging
In this paper, we introduce "Token Fusion" (ToFu), a method that amalgamates the benefits of both token pruning and token merging.
Structural Alignment for Network Pruning through Partial Regularization
To mitigate this gap, we first learn a target sub-network during the model training process, and then we use this sub-network to guide the learning of model weights through partial regularization.
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model.
Enhanced Bilevel Optimization via Bregman Distance
Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) to solve deterministic bilevel problems, which achieves a lower computational complexity than the best known results.
Bregman Gradient Policy Optimization
In the paper, we design a novel Bregman gradient policy optimization framework for reinforcement learning based on Bregman divergences and momentum techniques.
BiAdam: Fast Adaptive Bilevel Optimization Methods
To fill this gap, in the paper, we propose a novel fast adaptive bilevel framework to solve stochastic bilevel optimization problems that the outer problem is possibly nonconvex and the inner problem is strongly convex.
Network Pruning via Performance Maximization
Specifically, we train a stand-alone neural network to predict sub-networks' performance and then maximize the output of the network as a proxy of accuracy to guide pruning.
Adversarial Attack on Deep Cross-Modal Hamming Retrieval
Specifically, given a target model, we first construct its substitute model to exploit cross-modal correlations within hamming space, with which we create adversarial examples by limitedly querying from a target model.
Exploration and Estimation for Model Compression
In this paper, we focus on the discrimination-aware compression of Convolutional Neural Networks (CNNs).
Model Compression via Hyper-Structure Network
In this paper, we propose a novel channel pruning method to solve the problem of compression and acceleration of Convolutional Neural Networks (CNNs).
Gradient Descent Ascent for Minimax Problems on Riemannian Manifolds
At the same time, we present an effective Riemannian stochastic gradient descent ascent (RSGDA) algorithm for the stochastic minimax optimization, which has a sample complexity of $O(\kappa^4\epsilon^{-4})$ for finding an $\epsilon$-stationary solution.
Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
Our Acc-MDA achieves a low gradient complexity of $\tilde{O}(\kappa_y^{4. 5}\epsilon^{-3})$ without requiring large batches for finding an $\epsilon$-stationary point.
Momentum-Based Policy Gradient Methods
In particular, we present a non-adaptive version of IS-MBPG method, i. e., IS-MBPG*, which also reaches the best known sample complexity of $O(\epsilon^{-3})$ without any large batches.
Discrete Model Compression With Resource Constraint for Deep Neural Networks
In this paper, we target to address the problem of compression and acceleration of Convolutional Neural Networks (CNNs).
Cross-Modal Learning with Adversarial Samples
Extensive experiments on two cross-modal benchmark datasets show that the adversarial examples produced by our CMLA are efficient in fooling a target deep cross-modal hashing network.
Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity
Zeroth-order (a. k. a, derivative-free) methods are a class of effective optimization methods for solving complex machine learning problems, where gradients of the objective functions are not available or computationally prohibitive.
Cross Domain Model Compression by Structurally Weight Sharing
Regular model compression methods focus on RGB input.
Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization
In particular, our methods not only reach the best convergence rate $O(1/T)$ for the nonconvex optimization, but also are able to effectively solve many complex machine learning problems with multiple regularized penalties and constraints.
