Search Results for author:
Found 215 papers, 47 papers with code
Adversarial Nonnegative Matrix Factorization
Nonnegative Matrix Factorization (NMF) has become an increasingly important research topic in machine learning.
Binarized Neural Network for Single Image Super Resolution
Lighter model and faster inference are the focus of current single image super-resolution (SISR) research.
Can Stochastic Zeroth-Order Frank-Wolfe Method Converge Faster for Non-Convex Problems?
To address the problem of lacking gradient in many applications, we propose two new stochastic zeroth-order Frank-Wolfe algorithms and theoretically proved that they have a faster convergence rate than existing methods for non-convex problems.
Sparse Shrunk Additive Models
Meanwhile, in these feature selection models, the interactions between features are often ignored or just discussed under prior structure information.
Fast OSCAR and OWL with Safe Screening Rules
Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.
Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models
Recent advances in diffusion generative models have yielded remarkable progress.
AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving
Our findings call for immediate and decisive action to address the trustworthiness of DriveVLMs -- an issue of critical importance to public safety and the welfare of all citizens relying on autonomous transportation systems.
SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models
Recent advances in large-scale text-to-image (T2I) diffusion models have enabled a variety of downstream applications, including style customization, subject-driven personalization, and conditional generation.
Cell as Point: One-Stage Framework for Efficient Cell Tracking
Cellular activities are dynamic and intricate, playing a crucial role in advancing diagnostic and therapeutic techniques, yet they often require substantial resources for accurate tracking.
APDDv2: Aesthetics of Paintings and Drawings Dataset with Artist Labeled Scores and Comments
Datasets play a pivotal role in training visual models, facilitating the development of abstract understandings of visual features through diverse image samples and multidimensional attributes.
Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling
Bilevel Optimization has experienced significant advancements recently with the introduction of new efficient algorithms.
Fast and scalable Wasserstein-1 neural optimal transport solver for single-cell perturbation prediction
Our experiments demonstrate that the proposed $W_1$ neural optimal transport solver can mimic the $W_2$ OT solvers in finding a unique and ``monotonic" map on 2D datasets.
A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution
Authorship attribution aims to identify the origin or author of a document.
Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models
Large language models are probabilistic models, and the process of generating content is essentially sampling from the output distribution of the language model.
A Watermark for Order-Agnostic Language Models
Correspondingly, we propose a statistical pattern-based detection algorithm that recovers the key sequence during detection and conducts statistical tests based on the count of high-frequency patterns.
De-mark: Watermark Removal in Large Language Models
Watermarking techniques offer a promising way to identify machine-generated content via embedding covert information into the contents generated from language models (LMs).
OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities
To address this, OmnixR offers two evaluation variants: (1)synthetic subset: a synthetic dataset generated automatically by translating text into multiple modalities--audio, images, video, and hybrids (Omnify).
Cross-Modal Few-Shot Learning: a Generative Transfer Learning Framework
Most existing studies on few-shot learning focus on unimodal settings, where models are trained to generalize on unseen data using only a small number of labeled examples from the same modality.
LLaVA-Critic: Learning to Evaluate Multimodal Models
We introduce LLaVA-Critic, the first open-source large multimodal model (LMM) designed as a generalist evaluator to assess performance across a wide range of multimodal tasks.
Obtaining Lower Query Complexities through Lightweight Zeroth-Order Proximal Gradient Algorithms
While the random ZO estimator introduces bigger error and makes convergence analysis more challenging compared to coordinated ZO estimator, it requires only $\mathcal{O}(1)$ computation, which is significantly less than $\mathcal{O}(d)$ computation of the coordinated ZO estimator, with $d$ being dimension of the problem space.
Mixture of Efficient Diffusion Experts Through Automatic Interval and Sub-Network Selection
Finally, with a selected configuration, we fine-tune our pruned experts to obtain our mixture of efficient experts.
From Lists to Emojis: How Format Bias Affects Model Alignment
In this work, we extend the study of biases in preference learning beyond the commonly recognized length bias, offering a comprehensive analysis of a wider range of format biases.
Revisiting Adaptive Cellular Recognition Under Domain Shifts: A Contextual Correspondence View
Cellular nuclei recognition serves as a fundamental and essential step in the workflow of digital pathology.
MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modeling
Multi-view hand mesh reconstruction is a critical task for applications in virtual reality and human-computer interaction, but it remains a formidable challenge.
Brain Dialogue Interface (BDI): A User-Friendly fMRI Model for Interactive Brain Decoding
To address this issue, this study introduces a user-friendly decoding model that enables dynamic communication with the brain, as opposed to the static decoding approaches utilized by traditional studies.
Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models
Each architecture code represents a specialized model tailored to the prompts assigned to it, and the number of codes is a hyperparameter.
From Pixels to Prose: A Large Dataset of Dense Image Captions
Training large vision-language models requires extensive, high-quality image-text pairs.
OPTune: Efficient Online Preference Tuning
In this paper, we propose a more efficient data exploration strategy for online preference tuning (OPTune), which does not rely on human-curated or pre-collected teacher responses but dynamically samples informative responses for on-policy preference alignment.
Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion
By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities.
Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions
Experimental results support that the beta-watermark can effectively reduce the distribution bias under key collisions.
Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation
The MRI-derived brain network serves as a pivotal instrument in elucidating both the structural and functional aspects of the brain, encompassing the ramifications of diseases and developmental processes.
Paintings and Drawings Aesthetics Assessment with Rich Attributes for Various Artistic Categories
This dataset solely comprises overall scores for high-quality artistic images.
Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment
We address this challenge by designing a mechanism to model the complex changing dynamics of the reward function and provide a representation of it to the RL agent.
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
A Unified and General Framework for Continual Learning
Extensive experiments on CL benchmarks and theoretical analysis demonstrate the effectiveness of the proposed refresh learning.
Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt
Few-Shot Class-Incremental Learning (FSCIL) models aim to incrementally learn new classes with scarce samples while preserving knowledge of old ones.
class-incremental learning
Few-Shot Class-Incremental Learning
+1
Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning
3D medical image segmentation is a challenging task with crucial implications for disease diagnosis and treatment planning.
Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling
Machine learning holds tremendous promise for transforming the fundamental practice of scientific discovery by virtue of its data-driven nature.
Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection
In the first stage, we devise a scoring network to evaluate the difficulty of training instructions, which is co-trained with the VLM.
ODIN: Disentangled Reward Mitigates Hacking in RLHF
In this work, we study the issue of reward hacking on the response length, a challenge emerging in Reinforcement Learning from Human Feedback (RLHF) on LLMs.
Device-Wise Federated Network Pruning
Neural network pruning particularly channel pruning is a widely used technique for compressing deep learning models to enable their deployment on edge devices with limited resources.
BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks
Most existing dynamic or runtime channel pruning methods have to store all weights to achieve efficient inference which brings extra storage costs.
Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold
Thus, they neglect the critical characteristic of GANs: their local density structure over their learned manifold.
Federated Continual Novel Class Learning
In a privacy-focused era, Federated Learning (FL) has emerged as a promising machine learning technique.
On the Role of Server Momentum in Federated Learning
To our best knowledge, this is the first work that thoroughly analyzes the performances of server momentum with a hyperparameter scheduler and system heterogeneity.
Predicting Scores of Various Aesthetic Attribute Sets by Learning from Overall Score Labels
Then, we define an aesthetic attribute contribution to describe the role of aesthetic attributes throughout an image and use it with the attribute scores and the overall scores to train our F2S model.
Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian
In this paper we consider a nonconvex unconstrained optimization problem minimizing a twice differentiable objective function with H\"older continuous Hessian.
Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
Our work aims to address this concern by introducing a novel approach to detecting adversarial prompts at a token level, leveraging the LLM's capability to predict the next token's probability.
GPT-4 Vision on Medical Image Classification -- A Case Study on COVID-19 Dataset
This technical report delves into the application of GPT-4 Vision (GPT-4V) in the nuanced realm of COVID-19 image classification, leveraging the transformative potential of in-context learning to enhance diagnostic processes.
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation.
A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models
Watermarking techniques offer a promising way to identify machine-generated content via embedding covert information into the contents generated from language models.
Solving a Class of Non-Convex Minimax Optimization in Federated Learning
We propose FL algorithms (FedSGDA+ and FedSGDA-M) and reduce existing complexity results for the most common minimax problems.
Shielding the Unseen: Privacy Protection through Poisoning NeRF with Spatial Deformation
In this paper, we introduce an innovative method of safeguarding user privacy against the generative capabilities of Neural Radiance Fields (NeRF) models.
Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
With the fast growth of LLM-enabled AI applications and democratization of open-souced LLMs, fine-tuning has become possible for non-expert individuals, but intensively performed LLM fine-tuning worldwide could result in significantly high energy consumption and carbon footprint, which may bring large environmental impact.
Deep Prompt Tuning for Graph Transformers
We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning for leveraging large graph transformer models in downstream graph based prediction tasks.
Current and future directions in network biology
Network biology is an interdisciplinary field bridging computational and biological sciences that has proved pivotal in advancing the understanding of cellular functions and diseases across biological systems and scales.
Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining
To address the above challenge, we study the serverless multi-party collaborative AUPRC maximization problem since serverless multi-party collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC.
Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation
The success of automated medical image analysis depends on large-scale and expert-annotated training sets.
AlpaGasus: Training A Better Alpaca with Fewer Data
Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data.
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Through this comprehensive survey, we aspire to uncover potential solutions by drawing upon ideas and approaches from various fields that have dealt with forgetting.
Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets
We identify a phenomenon named player domination in the bargaining game, namely that the existing max-based approaches, such as MAX and MSD, do not converge.
InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models
Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden.
Prediction of Post-Operative Renal and Pulmonary Complications Using Transformers
This work highlights the potential of deep learning techniques, specifically transformer-based models, in revolutionizing the healthcare industry's approach to postoperative care.
Incomplete Multimodal Learning for Complex Brain Disorders Prediction
Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages.
Prompting Language-Informed Distribution for Compositional Zero-Shot Learning
Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts, e. g., sliced tomatoes, where the model is learned only from the seen compositions, e. g., sliced potatoes and red tomatoes.
PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer
To address this critical problem, we first investigate and find that the loss landscape of vanilla prompt tuning is precipitous when it is visualized, where a slight change of input data can cause a big fluctuation in the loss landscape.
Backdoor Learning on Sequence to Sequence Models
Backdoor learning has become an emerging research area towards building a trustworthy machine learning system.
Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction
Direct optimization of interpolated features on multi-resolution voxel grids has emerged as a more efficient alternative to MLP-like modules.
When do you need Chain-of-Thought Prompting for ChatGPT?
However, it is not clear whether CoT is still effective on more recent instruction finetuned (IFT) LLMs such as ChatGPT.
Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
In this work, we investigate Federated Bilevel Optimization problems and propose a communication-efficient algorithm, named FedBiOAcc.
FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging
This matches the best known rate for first-order FL algorithms and \textbf{FedDA-MVR} is the first adaptive FL algorithm that achieves this rate.
Decentralized Riemannian Algorithm for Nonconvex Minimax Problems
The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold).
A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
In this paper we consider finding an approximate second-order stationary point (SOSP) of general nonconvex conic optimization that minimizes a twice differentiable function subject to nonlinear equality constraints and also a convex conic constraint.
Learning with Diversity: Self-Expanded Equalization for Better Generalized Deep Metric Learning
Most existing DML methods focus on improving the model robustness against category shift to keep the performance on unseen categories.
Structural Alignment for Network Pruning through Partial Regularization
To mitigate this gap, we first learn a target sub-network during the model training process, and then we use this sub-network to guide the learning of model weights through partial regularization.
Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks
In this paper, we extensively study this phenomenon for graph data.
Faster Adaptive Federated Learning
Federated learning has attracted increasing attention with the emergence of distributed data.
Towards Robust Dataset Learning
In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust.
BI AVAN: Brain inspired Adversarial Visual Attention Network
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks.
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings
Therefore, Federated Recommender (FedRec) systems are proposed to mitigate privacy concerns to non-distributed recommender systems.
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining
In our theoretical analysis, we prove that our new algorithm achieves a fast convergence rate of $O(\frac{1}{\sqrt{nT}} + \frac{1}{(k/d)^2 T})$ with the communication cost of $O(k \log(d))$ at each iteration.
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model.
An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification
To address this challenge, we propose a novel accelerated doubly stochastic gradient descent (ADSGD) method for sparsity regularized loss minimization problems, which can reduce the number of block iterations by eliminating inactive coefficients during the optimization process and eventually achieve faster explicit model identification and improve the algorithm efficiency.
Aesthetic Language Guidance Generation of Images Using Attribute Comparison
The differences of the three attributes between the input images and the photography templates or the guidance images are described in natural language, which is aesthetic natural language guidance (ALG).
Aesthetic Attributes Assessment of Images with AMANv2 and DPC-CaptionsV2
In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i. e., to assess the aesthetic attributes such as composition, lighting usage and color arrangement.
Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model
Here, we propose an interpretable hierarchical signed graph representation learning model to extract graph-level representations from brain functional networks, which can be used for different prediction tasks.
Balanced Self-Paced Learning for AUC Maximization
Self-paced learning is an effective method for handling noisy data.
RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrieval
The challenge is to design a provably robust algorithm that takes into consideration the 1-NN search and the high-dimensional nature of the embedding space.
Communication-Efficient Robust Federated Learning with Noisy Labels
Bilevel optimization problem is a type of optimization problem with two levels of entangled problems.
Functional2Structural: Cross-Modality Brain Networks Representation Learning
Since brain networks derived from functional and structural MRI describe the brain topology from different perspectives, exploring a representation that combines these cross-modality brain networks is non-trivial.
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction
Specifically, we first propose the FedBiO, a deterministic gradient-based algorithm and we show it requires $O(\epsilon^{-2})$ number of iterations to reach an $\epsilon$-stationary point.
Distributed Dynamic Safe Screening Algorithms for Sparse Regularization
To the best of our knowledge, this is the first work of distributed safe dynamic screening method.
Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation
In this work, we propose a novel training framework FedSM to avoid the client drift issue and successfully close the generalization gap compared with the centralized training for medical image segmentation tasks for the first time.
Towards Bi-directional Skip Connections in Encoder-Decoder Architectures and Beyond
With the backward skip connections, we propose a U-Net based network family, namely Bi-directional O-shape networks, which set new benchmarks on multiple public medical imaging segmentation datasets.
A Law of Robustness beyond Isoperimetry
We prove a Lipschitzness lower bound $\Omega(\sqrt{n/p})$ of the interpolating neural network with $p$ parameters on arbitrary data distributions.
HNF-Netv2 for Brain Tumor Segmentation using multi-modal MR Imaging
In our previous work, $i. e.$, HNF-Net, high-resolution feature representation and light-weight non-local self-attention mechanism are exploited for brain tumor segmentation using multi-modal MR imaging.
Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement
Recent advances in unsupervised domain adaptation (UDA) techniques have witnessed great success in cross-domain computer vision tasks, enhancing the generalization ability of data-driven deep learning architectures by bridging the domain distribution gaps.
Noise Is Also Useful: Negative Correlation-Steered Latent Contrastive Learning
While in metric space, we utilize weakly-supervised contrastive learning to excavate these negative correlations hidden in noisy data.
A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse
Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.
Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.
Optimal Underdamped Langevin MCMC Method
In the paper, we study the underdamped Langevin diffusion (ULD) with strongly-convex potential consisting of finite summation of $N$ smooth components, and propose an efficient discretization method, which requires $O(N+d^\frac{1}{3}N^\frac{2}{3}/\varepsilon^\frac{2}{3})$ gradient evaluations to achieve $\varepsilon$-error (in $\sqrt{\mathbb{E}{\lVert{\cdot}\rVert_2^2}}$ distance) for approximating $d$-dimensional ULD.
Efficient Mirror Descent Ascent Methods for Nonsmooth Minimax Problems
For our stochastic algorithms, we first prove that the mini-batch stochastic mirror descent ascent (SMDA) method obtains a sample complexity of $O(\kappa^3\epsilon^{-4})$ for finding an $\epsilon$-stationary point, where $\kappa$ denotes the condition number.
A Faster Decentralized Algorithm for Nonconvex Minimax Problems
We prove that our DM-HSGD algorithm achieves stochastic first-order oracle (SFO) complexity of $O(\kappa^3 \epsilon^{-3})$ for decentralized stochastic nonconvex-strongly-concave problem to search an $\epsilon$-stationary point, which improves the exiting best theoretical results.
Adaptive Hierarchical Similarity Metric Learning with Noisy Labels
Since these noisy labels often cause severe performance degradation, it is crucial to enhance the robustness and generalization ability of DML.
Efficient Semi-Supervised Adversarial Training without Guessing Labels
To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.
Efficient Bi-level Optimization for Non-smooth Optimization
Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.
Understanding Metric Learning on Unit Hypersphere and Generating Better Examples for Adversarial Training
In this paper, we boost the performance of deep metric learning (DML) models with adversarial examples generated by attacking two new objective functions: \textit{intra-class alignment} and \textit{hyperspherical uniformity}.
Adversarial Fairness Network
To achieve this goal, we reformulate the data input by eliminating the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature.
AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization
To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.
An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization
The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.
DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology Image Analysis
With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label.
Boundary-aware Graph Reasoning for Semantic Segmentation
In this paper, we propose a Boundary-aware Graph Reasoning (BGR) module to learn long-range contextual features for semantic segmentation.
PSGR: Pixel-wise Sparse Graph Reasoning for COVID-19 Pneumonia Segmentation in CT Images
In the PSGR module, a graph is first constructed by projecting each pixel on a node based on the features produced by the segmentation backbone, and then converted into a sparsely-connected graph by keeping only K strongest connections to each uncertain pixel.
Enhanced Bilevel Optimization via Bregman Distance
Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) to solve deterministic bilevel problems, which achieves a lower computational complexity than the best known results.
Fedlearn-Algo: A flexible open-source privacy-preserving machine learning platform
We use this platform to demonstrate our research and development results on privacy preserving machine learning algorithms.
BiX-NAS: Searching Efficient Bi-directional Architecture for Medical Image Segmentation
We evaluate BiX-NAS on two segmentation tasks using three different medical image datasets, and the experimental results show that our BiX-NAS searched architecture achieves the state-of-the-art performance with significantly lower computational cost.
Bregman Gradient Policy Optimization
In the paper, we design a novel Bregman gradient policy optimization framework for reinforcement learning based on Bregman divergences and momentum techniques.
Network Pruning via Performance Maximization
Specifically, we train a stand-alone neural network to predict sub-networks' performance and then maximize the output of the network as a proxy of accuracy to guide pruning.
Nearest Neighbor Matching for Deep Clustering
Specifically, for the local level, we match the nearest neighbors based on batch embedded features, as for the global one, we match neighbors from overall embedded features.
Unsupervised Hyperbolic Metric Learning
Learning feature embedding directly from images without any human supervision is a very challenging and essential task in the field of computer vision and machine learning.
SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients
To fill this gap, we propose a faster and universal framework of adaptive gradients (i. e., SUPER-ADAM) by introducing a universal adaptive matrix that includes most existing adaptive gradient forms.
Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.
Learning Sampling Policy for Faster Derivative Free Optimization
To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.
Doubly Contrastive Deep Clustering
In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results.
Data Augmentation for Object Detection via Differentiable Neural Rendering
We propose a new offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.
Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm
Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.
A New Framework for Variance-Reduced Hamiltonian Monte Carlo
Moreover, our HMC methods with biased gradient estimators, such as SARAH and SARGE, require $\tilde{O}(N+\sqrt{N} \kappa^2 d^{\frac{1}{2}} \varepsilon^{-1})$ gradient complexity, which has the same dependency on condition number $\kappa$ and dimension $d$ as full gradient method, but improves the dependency of sample size $N$ for a factor of $N^\frac{1}{2}$.
Coordinating Momenta for Cross-silo Federated Learning
In this work, we propose a new method to improve the training performance in cross-silo FL via maintaining double momentum buffers.
A Bayesian Federated Learning Framework with Online Laplace Approximation
On the client side, a prior loss that uses the global posterior probabilistic parameters delivered from the server is designed to guide the local training.
Model Compression via Hyper-Structure Network
In this paper, we propose a novel channel pruning method to solve the problem of compression and acceleration of Convolutional Neural Networks (CNNs).
Exploration and Estimation for Model Compression
In this paper, we focus on the discrimination-aware compression of Convolutional Neural Networks (CNNs).
Adversarial Attack on Deep Cross-Modal Hamming Retrieval
Specifically, given a target model, we first construct its substitute model to exploit cross-modal correlations within hamming space, with which we create adversarial examples by limitedly querying from a target model.
Learning Better Visual Data Similarities via New Grouplet Non-Euclidean Embedding
However, pair-wise methods involve expensive training costs, while proxy-based methods are less accurate in characterizing the relationships between data points.
Delay-Tolerant Local SGD for Efficient Distributed Training
The heavy communication for model synchronization is a major bottleneck for scaling up the distributed deep neural network training to many workers.
H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task
In this paper, we propose a Hybrid High-resolution and Non-local Feature Network (H2NF-Net) to segment brain tumor in multimodal MR images.
CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning
In this paper, we propose a new interpretable graph pooling framework - CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process.
A Deep Drift-Diffusion Model for Image Aesthetic Score Distribution Prediction
In this paper, we propose a Deep Drift-Diffusion (DDD) model inspired by psychologists to predict aesthetic score distribution from images.
PDAM: A Panoptic-Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images
In this work, we present an unsupervised domain adaptation (UDA) method, named Panoptic Domain Adaptive Mask R-CNN (PDAM), for unsupervised instance segmentation in microscopy images.
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee
In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.
Adaptive Serverless Learning
To the best of our knowledge, this is the first adaptive decentralized training approach.
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training
The condition for achieving the linear speedup is also provided for this variant.
Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
Our Acc-MDA achieves a low gradient complexity of $\tilde{O}(\kappa_y^{4. 5}\epsilon^{-3})$ without requiring large batches for finding an $\epsilon$-stationary point.
Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.
Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient
Both our theoretical and empirical results show that our new methods can handle the "gradient mismatch" problem.
Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
Our theoretical analysis shows that the online SPIDER-ADMM has the IFO complexity of $\mathcal{O}(\epsilon^{-\frac{3}{2}})$, which improves the existing best results by a factor of $\mathcal{O}(\epsilon^{-\frac{1}{2}})$.
Momentum-Based Policy Gradient Methods
In particular, we present a non-adaptive version of IS-MBPG method, i. e., IS-MBPG*, which also reaches the best known sample complexity of $O(\epsilon^{-3})$ without any large batches.
BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture
U-Net has become one of the state-of-the-art deep learning-based approaches for modern computer vision tasks such as semantic segmentation, super resolution, image denoising, and inpainting.
Fast OSCAR and OWL Regression via Safe Screening Rules
Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.
Faster Secure Data Mining via Distributed Homomorphic Encryption
Due to the rising privacy demand in data mining, Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field.
Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting
More specifically, we first propose a nuclei inpainting mechanism to remove the auxiliary generated objects in the synthesized images.
Detached Error Feedback for Distributed SGD with Random Sparsification
To tackle this important issue, we improve the communication-efficient distributed SGD from a novel aspect, that is, the trade-off between the variance and second moment of the gradient.
Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training
The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications.
Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images
Specifically, our proposed PFFNet contains a residual attention feature fusion mechanism to incorporate the instance prediction with the semantic features, in order to facilitate the semantic contextual information learning in the instance branch.
Large Batch Training Does Not Need Warmup
Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.
Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization
Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances r