Search Results for author:
Found 185 papers, 37 papers with code
Binarized Neural Network for Single Image Super Resolution
Lighter model and faster inference are the focus of current single image super-resolution (SISR) research.
Sparse Shrunk Additive Models
Meanwhile, in these feature selection models, the interactions between features are often ignored or just discussed under prior structure information.
Can Stochastic Zeroth-Order Frank-Wolfe Method Converge Faster for Non-Convex Problems?
To address the problem of lacking gradient in many applications, we propose two new stochastic zeroth-order Frank-Wolfe algorithms and theoretically proved that they have a faster convergence rate than existing methods for non-convex problems.
Adversarial Nonnegative Matrix Factorization
Nonnegative Matrix Factorization (NMF) has become an increasingly important research topic in machine learning.
Fast OSCAR and OWL with Safe Screening Rules
Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.
Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment
We address this challenge by designing a mechanism to model the complex changing dynamics of the reward function and provide a representation of it to the RL agent.
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
A Unified and General Framework for Continual Learning
Extensive experiments on CL benchmarks and theoretical analysis demonstrate the effectiveness of the proposed refresh learning.
Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt
Few-Shot Class-Incremental Learning (FSCIL) models aim to incrementally learn new classes with scarce samples while preserving knowledge of old ones.
Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning
3D medical image segmentation is a challenging task with crucial implications for disease diagnosis and treatment planning.
Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling
Machine learning holds tremendous promise for transforming the fundamental practice of scientific discovery by virtue of its data-driven nature.
Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection
In the first stage, we devise a scoring network to evaluate the difficulty of training instructions, which is co-trained with the VLM.
ODIN: Disentangled Reward Mitigates Hacking in RLHF
In this work, we study the issue of reward hacking on the response length, a challenge emerging in Reinforcement Learning from Human Feedback (RLHF) on LLMs.
Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold
Thus, they neglect the critical characteristic of GANs: their local density structure over their learned manifold.
Federated Continual Novel Class Learning
In a privacy-focused era, Federated Learning (FL) has emerged as a promising machine learning technique.
On the Role of Server Momentum in Federated Learning
To our best knowledge, this is the first work that thoroughly analyzes the performances of server momentum with a hyperparameter scheduler and system heterogeneity.
Predicting Scores of Various Aesthetic Attribute Sets by Learning from Overall Score Labels
Then, we define an aesthetic attribute contribution to describe the role of aesthetic attributes throughout an image and use it with the attribute scores and the overall scores to train our F2S model.
Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
Our work aims to address this concern by introducing a novel approach to detecting adversarial prompts at a token level, leveraging the LLM's capability to predict the next token's probability.
GPT-4 Vision on Medical Image Classification -- A Case Study on COVID-19 Dataset
This technical report delves into the application of GPT-4 Vision (GPT-4V) in the nuanced realm of COVID-19 image classification, leveraging the transformative potential of in-context learning to enhance diagnostic processes.
Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation.
DiPmark: A Stealthy, Efficient and Resilient Watermark for Large Language Models
Watermarking techniques offer a promising way to secure data via embedding covert information into the data.
Solving a Class of Non-Convex Minimax Optimization in Federated Learning
We propose FL algorithms (FedSGDA+ and FedSGDA-M) and reduce existing complexity results for the most common minimax problems.
Shielding the Unseen: Privacy Protection through Poisoning NeRF with Spatial Deformation
In this paper, we introduce an innovative method of safeguarding user privacy against the generative capabilities of Neural Radiance Fields (NeRF) models.
Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagation
With the fast growth of LLM-enabled AI applications and democratization of open-souced LLMs, fine-tuning has become possible for non-expert individuals, but intensively performed LLM fine-tuning worldwide could result in significantly high energy consumption and carbon footprint, which may bring large environmental impact.
Deep Prompt Tuning for Graph Transformers
We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning for leveraging large graph transformer models in downstream graph based prediction tasks.
Current and future directions in network biology
As such, it is expected to help shape short- and long-term vision for future computational and algorithmic research in network biology.
Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining
To address the above challenge, we study the serverless multi-party collaborative AUPRC maximization problem since serverless multi-party collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC.
Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation
The success of automated medical image analysis depends on large-scale and expert-annotated training sets.
AlpaGasus: Training A Better Alpaca with Fewer Data
Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data.
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Through this comprehensive survey, we aspire to uncover potential solutions by drawing upon ideas and approaches from various fields that have dealt with forgetting.
Cooperation or Competition: Avoiding Player Domination for Multi-Target Robustness via Adaptive Budgets
We identify a phenomenon named player domination in the bargaining game, namely that the existing max-based approaches, such as MAX and MSD, do not converge.
InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models
Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden.
Prediction of Post-Operative Renal and Pulmonary Complications Using Transformers
This work highlights the potential of deep learning techniques, specifically transformer-based models, in revolutionizing the healthcare industry's approach to postoperative care.
Incomplete Multimodal Learning for Complex Brain Disorders Prediction
Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages.
Prompting Language-Informed Distribution for Compositional Zero-Shot Learning
Orthogonal to the existing literature of soft, hard, or distributional prompts, our method advocates prompting the LLM-supported class distribution that leads to a better zero-shot generalization.
PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer
To address this critical problem, we first investigate and find that the loss landscape of vanilla prompt tuning is precipitous when it is visualized, where a slight change of input data can cause a big fluctuation in the loss landscape.
Backdoor Learning on Sequence to Sequence Models
Backdoor learning has become an emerging research area towards building a trustworthy machine learning system.
Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction
Direct optimization of interpolated features on multi-resolution voxel grids has emerged as a more efficient alternative to MLP-like modules.
When do you need Chain-of-Thought Prompting for ChatGPT?
However, it is not clear whether CoT is still effective on more recent instruction finetuned (IFT) LLMs such as ChatGPT.
Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
In this work, we investigate Federated Bilevel Optimization problems and propose a communication-efficient algorithm, named FedBiOAcc.
FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging
This matches the best known rate for first-order FL algorithms and \textbf{FedDA-MVR} is the first adaptive FL algorithm that achieves this rate.
Decentralized Riemannian Algorithm for Nonconvex Minimax Problems
The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold).
A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
In this paper we consider finding an approximate second-order stationary point (SOSP) of general nonconvex conic optimization that minimizes a twice differentiable function subject to nonlinear equality constraints and also a convex conic constraint.
Learning with Diversity: Self-Expanded Equalization for Better Generalized Deep Metric Learning
Most existing DML methods focus on improving the model robustness against category shift to keep the performance on unseen categories.
Structural Alignment for Network Pruning through Partial Regularization
To mitigate this gap, we first learn a target sub-network during the model training process, and then we use this sub-network to guide the learning of model weights through partial regularization.
Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks
In this paper, we extensively study this phenomenon for graph data.
Faster Adaptive Federated Learning
Federated learning has attracted increasing attention with the emergence of distributed data.
Towards Robust Dataset Learning
In this paper, we study the problem of learning a robust dataset such that any classifier naturally trained on the dataset is adversarially robust.
BI AVAN: Brain inspired Adversarial Visual Attention Network
Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks.
FedGRec: Federated Graph Recommender System with Lazy Update of Latent Embeddings
Therefore, Federated Recommender (FedRec) systems are proposed to mitigate privacy concerns to non-distributed recommender systems.
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining
In our theoretical analysis, we prove that our new algorithm achieves a fast convergence rate of $O(\frac{1}{\sqrt{nT}} + \frac{1}{(k/d)^2 T})$ with the communication cost of $O(k \log(d))$ at each iteration.
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model.
An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification
To address this challenge, we propose a novel accelerated doubly stochastic gradient descent (ADSGD) method for sparsity regularized loss minimization problems, which can reduce the number of block iterations by eliminating inactive coefficients during the optimization process and eventually achieve faster explicit model identification and improve the algorithm efficiency.
Aesthetic Attributes Assessment of Images with AMANv2 and DPC-CaptionsV2
In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i. e., to assess the aesthetic attributes such as composition, lighting usage and color arrangement.
Aesthetic Language Guidance Generation of Images Using Attribute Comparison
The differences of the three attributes between the input images and the photography templates or the guidance images are described in natural language, which is aesthetic natural language guidance (ALG).
Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model
Here, we propose an interpretable hierarchical signed graph representation learning model to extract graph-level representations from brain functional networks, which can be used for different prediction tasks.
Balanced Self-Paced Learning for AUC Maximization
Self-paced learning is an effective method for handling noisy data.
RetrievalGuard: Provably Robust 1-Nearest Neighbor Image Retrieval
The challenge is to design a provably robust algorithm that takes into consideration the 1-NN search and the high-dimensional nature of the embedding space.
Communication-Efficient Robust Federated Learning with Noisy Labels
Bilevel optimization problem is a type of optimization problem with two levels of entangled problems.
Functional2Structural: Cross-Modality Brain Networks Representation Learning
Since brain networks derived from functional and structural MRI describe the brain topology from different perspectives, exploring a representation that combines these cross-modality brain networks is non-trivial.
Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction
Specifically, we first propose the FedBiO, a deterministic gradient-based algorithm and we show it requires $O(\epsilon^{-2})$ number of iterations to reach an $\epsilon$-stationary point.
Distributed Dynamic Safe Screening Algorithms for Sparse Regularization
To the best of our knowledge, this is the first work of distributed safe dynamic screening method.
Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation
In this work, we propose a novel training framework FedSM to avoid the client drift issue and successfully close the generalization gap compared with the centralized training for medical image segmentation tasks for the first time.
Towards Bi-directional Skip Connections in Encoder-Decoder Architectures and Beyond
With the backward skip connections, we propose a U-Net based network family, namely Bi-directional O-shape networks, which set new benchmarks on multiple public medical imaging segmentation datasets.
A Law of Robustness beyond Isoperimetry
We prove a Lipschitzness lower bound $\Omega(\sqrt{n/p})$ of the interpolating neural network with $p$ parameters on arbitrary data distributions.
HNF-Netv2 for Brain Tumor Segmentation using multi-modal MR Imaging
In our previous work, $i. e.$, HNF-Net, high-resolution feature representation and light-weight non-local self-attention mechanism are exploited for brain tumor segmentation using multi-modal MR imaging.
Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement
Recent advances in unsupervised domain adaptation (UDA) techniques have witnessed great success in cross-domain computer vision tasks, enhancing the generalization ability of data-driven deep learning architectures by bridging the domain distribution gaps.
Noise Is Also Useful: Negative Correlation-Steered Latent Contrastive Learning
While in metric space, we utilize weakly-supervised contrastive learning to excavate these negative correlations hidden in noisy data.
A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse
Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.
Optimal Underdamped Langevin MCMC Method
In the paper, we study the underdamped Langevin diffusion (ULD) with strongly-convex potential consisting of finite summation of $N$ smooth components, and propose an efficient discretization method, which requires $O(N+d^\frac{1}{3}N^\frac{2}{3}/\varepsilon^\frac{2}{3})$ gradient evaluations to achieve $\varepsilon$-error (in $\sqrt{\mathbb{E}{\lVert{\cdot}\rVert_2^2}}$ distance) for approximating $d$-dimensional ULD.
Efficient Mirror Descent Ascent Methods for Nonsmooth Minimax Problems
For our stochastic algorithms, we first prove that the mini-batch stochastic mirror descent ascent (SMDA) method obtains a sample complexity of $O(\kappa^3\epsilon^{-4})$ for finding an $\epsilon$-stationary point, where $\kappa$ denotes the condition number.
A Faster Decentralized Algorithm for Nonconvex Minimax Problems
We prove that our DM-HSGD algorithm achieves stochastic first-order oracle (SFO) complexity of $O(\kappa^3 \epsilon^{-3})$ for decentralized stochastic nonconvex-strongly-concave problem to search an $\epsilon$-stationary point, which improves the exiting best theoretical results.
Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.
Adaptive Hierarchical Similarity Metric Learning with Noisy Labels
Since these noisy labels often cause severe performance degradation, it is crucial to enhance the robustness and generalization ability of DML.
Efficient Bi-level Optimization for Non-smooth Optimization
Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.
Understanding Metric Learning on Unit Hypersphere and Generating Better Examples for Adversarial Training
In this paper, we boost the performance of deep metric learning (DML) models with adversarial examples generated by attacking two new objective functions: \textit{intra-class alignment} and \textit{hyperspherical uniformity}.
Adversarial Fairness Network
To achieve this goal, we reformulate the data input by eliminating the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature.
Efficient Semi-Supervised Adversarial Training without Guessing Labels
To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.
AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization
To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.
An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization
The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.
DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology Image Analysis
With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label.
Boundary-aware Graph Reasoning for Semantic Segmentation
In this paper, we propose a Boundary-aware Graph Reasoning (BGR) module to learn long-range contextual features for semantic segmentation.
PSGR: Pixel-wise Sparse Graph Reasoning for COVID-19 Pneumonia Segmentation in CT Images
In the PSGR module, a graph is first constructed by projecting each pixel on a node based on the features produced by the segmentation backbone, and then converted into a sparsely-connected graph by keeping only K strongest connections to each uncertain pixel.
Enhanced Bilevel Optimization via Bregman Distance
Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) to solve deterministic bilevel problems, which achieves a lower computational complexity than the best known results.
Fedlearn-Algo: A flexible open-source privacy-preserving machine learning platform
We use this platform to demonstrate our research and development results on privacy preserving machine learning algorithms.
BiX-NAS: Searching Efficient Bi-directional Architecture for Medical Image Segmentation
We evaluate BiX-NAS on two segmentation tasks using three different medical image datasets, and the experimental results show that our BiX-NAS searched architecture achieves the state-of-the-art performance with significantly lower computational cost.
Bregman Gradient Policy Optimization
In the paper, we design a novel Bregman gradient policy optimization framework for reinforcement learning based on Bregman divergences and momentum techniques.
Unsupervised Hyperbolic Metric Learning
Learning feature embedding directly from images without any human supervision is a very challenging and essential task in the field of computer vision and machine learning.
Network Pruning via Performance Maximization
Specifically, we train a stand-alone neural network to predict sub-networks' performance and then maximize the output of the network as a proxy of accuracy to guide pruning.
Nearest Neighbor Matching for Deep Clustering
Specifically, for the local level, we match the nearest neighbors based on batch embedded features, as for the global one, we match neighbors from overall embedded features.
SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients
To fill this gap, we propose a faster and universal framework of adaptive gradients (i. e., SUPER-ADAM) by introducing a universal adaptive matrix that includes most existing adaptive gradient forms.
Fast Training Method for Stochastic Compositional Optimization Problems
The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.
Learning Sampling Policy for Faster Derivative Free Optimization
To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.
Doubly Contrastive Deep Clustering
In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results.
Data Augmentation for Object Detection via Differentiable Neural Rendering
We propose a new offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.
Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm
Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.
A New Framework for Variance-Reduced Hamiltonian Monte Carlo
Moreover, our HMC methods with biased gradient estimators, such as SARAH and SARGE, require $\tilde{O}(N+\sqrt{N} \kappa^2 d^{\frac{1}{2}} \varepsilon^{-1})$ gradient complexity, which has the same dependency on condition number $\kappa$ and dimension $d$ as full gradient method, but improves the dependency of sample size $N$ for a factor of $N^\frac{1}{2}$.
Coordinating Momenta for Cross-silo Federated Learning
In this work, we propose a new method to improve the training performance in cross-silo FL via maintaining double momentum buffers.
A Bayesian Federated Learning Framework with Online Laplace Approximation
On the client side, a prior loss that uses the global posterior probabilistic parameters delivered from the server is designed to guide the local training.
Exploration and Estimation for Model Compression
In this paper, we focus on the discrimination-aware compression of Convolutional Neural Networks (CNNs).
Adversarial Attack on Deep Cross-Modal Hamming Retrieval
Specifically, given a target model, we first construct its substitute model to exploit cross-modal correlations within hamming space, with which we create adversarial examples by limitedly querying from a target model.
Learning Better Visual Data Similarities via New Grouplet Non-Euclidean Embedding
However, pair-wise methods involve expensive training costs, while proxy-based methods are less accurate in characterizing the relationships between data points.
Delay-Tolerant Local SGD for Efficient Distributed Training
The heavy communication for model synchronization is a major bottleneck for scaling up the distributed deep neural network training to many workers.
Model Compression via Hyper-Structure Network
In this paper, we propose a novel channel pruning method to solve the problem of compression and acceleration of Convolutional Neural Networks (CNNs).
H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task
In this paper, we propose a Hybrid High-resolution and Non-local Feature Network (H2NF-Net) to segment brain tumor in multimodal MR images.
CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning
In this paper, we propose a new interpretable graph pooling framework - CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process.
A Deep Drift-Diffusion Model for Image Aesthetic Score Distribution Prediction
In this paper, we propose a Deep Drift-Diffusion (DDD) model inspired by psychologists to predict aesthetic score distribution from images.
PDAM: A Panoptic-Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images
In this work, we present an unsupervised domain adaptation (UDA) method, named Panoptic Domain Adaptive Mask R-CNN (PDAM), for unsupervised instance segmentation in microscopy images.
Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee
In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.
Adaptive Serverless Learning
To the best of our knowledge, this is the first adaptive decentralized training approach.
Periodic Stochastic Gradient Descent with Momentum for Decentralized Training
The condition for achieving the linear speedup is also provided for this variant.
Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
Our Acc-MDA achieves a low gradient complexity of $\tilde{O}(\kappa_y^{4. 5}\epsilon^{-3})$ without requiring large batches for finding an $\epsilon$-stationary point.
Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.
Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
Step-Ahead Error Feedback for Distributed Training with Compressed Gradient
Both our theoretical and empirical results show that our new methods can handle the "gradient mismatch" problem.
Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
Our theoretical analysis shows that the online SPIDER-ADMM has the IFO complexity of $\mathcal{O}(\epsilon^{-\frac{3}{2}})$, which improves the existing best results by a factor of $\mathcal{O}(\epsilon^{-\frac{1}{2}})$.
Momentum-Based Policy Gradient Methods
In particular, we present a non-adaptive version of IS-MBPG method, i. e., IS-MBPG*, which also reaches the best known sample complexity of $O(\epsilon^{-3})$ without any large batches.
BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture
U-Net has become one of the state-of-the-art deep learning-based approaches for modern computer vision tasks such as semantic segmentation, super resolution, image denoising, and inpainting.
Fast OSCAR and OWL Regression via Safe Screening Rules
Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.
Faster Secure Data Mining via Distributed Homomorphic Encryption
Due to the rising privacy demand in data mining, Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field.
Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting
More specifically, we first propose a nuclei inpainting mechanism to remove the auxiliary generated objects in the synthesized images.
Detached Error Feedback for Distributed SGD with Random Sparsification
To tackle this important issue, we improve the communication-efficient distributed SGD from a novel aspect, that is, the trade-off between the variance and second moment of the gradient.
Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training
The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications.
Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images
Specifically, our proposed PFFNet contains a residual attention feature fusion mechanism to incorporate the instance prediction with the semantic features, in order to facilitate the semantic contextual information learning in the instance branch.
Large Batch Training Does Not Need Warmup
Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.
Safe Sample Screening for Robust Support Vector Machine
To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP).
Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization
Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled.
Region and Object based Panoptic Image Synthesis through Conditional GANs
Our contribution consists of the proposal of a significant task worth investigating and a naive baseline of solving it.
Curvilinear Distance Metric Learning
To address this issue, we first reveal that the traditional linear distance metric is equivalent to the cumulative arc length between the data pair's nearest points on the learned straight measurer lines.
Straggler-Agnostic and Communication-Efficient Distributed Primal-Dual Algorithm for High-Dimensional Data Mining
Recently, reducing communication time between machines becomes the main focus of distributed data mining.
Deep Relational Factorization Machines
Factorization Machines (FMs) is an important supervised learning approach due to its unique ability to capture feature interactions when dealing with high-dimensional sparse data.
Ouroboros: On Accelerating Training of Transformer-Based Language Models
Model parallelism is required if a model is too large to fit in a single computing device.
Approaching Machine Learning Fairness through Adversarial Network
In order to achieve this goal, we reformulate the data input by removing the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature.
On the Acceleration of Deep Learning Model Parallelism with Staleness
Training the deep convolutional neural network for computer vision problems is slow and inefficient, especially when it is large and distributed across multiple devices.
Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity
Zeroth-order (a. k. a, derivative-free) methods are a class of effective optimization methods for solving complex machine learning problems, where gradients of the objective functions are not available or computationally prohibitive.
Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization
To address this problem, in this paper, we propose a novel scalable quadruply stochastic gradient algorithm (QSG-S2AUC) for nonlinear semi-supervised AUC optimization.
Scalable Semi-Supervised SVM via Triply Stochastic Gradients
Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient.
An Iteratively Re-weighted Method for Problems with Sparsity-Inducing Norms
This work aims at solving the problems with intractable sparsity-inducing norms that are often encountered in various machine learning tasks, such as multi-task learning, subspace clustering, feature selection, robust principal component analysis, and so on.
Robust Linear Discriminant Analysis Using Ratio Minimization of L1,2-Norms
In this paper, we propose a novel robust linear discriminant analysis method based on the L1, 2-norm ratio minimization.
Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization
In particular, our methods not only reach the best convergence rate $O(1/T)$ for the nonconvex optimization, but also are able to effectively solve many complex machine learning problems with multiple regularized penalties and constraints.
LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking
To the best of our knowledge, this is the first paper to propose an online human pose tracking framework in a top-down fashion.
Ranked #3 on
Pose Tracking
on PoseTrack2017
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering
In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention.
Ranked #27 on
Visual Question Answering (VQA)
on MSRVTT-QA
Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization
Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.
Bilevel Distance Metric Learning for Robust Image Recognition
Metric learning, aiming to learn a discriminative Mahalanobis distance matrix M that can effectively reflect the similarity between data samples, has been widely studied in various image recognition problems.
Panoptic Segmentation with an End-to-End Cell R-CNN for Pathology Image Analysis
The morphological clues of various cancer cells are essential for pathologists to determine the stages of cancers.
Ranked #1 on
Nuclear Segmentation
on Cell17
3D Global Convolutional Adversarial Network\\ for Prostate MR Volume Segmentation
In this paper, we propose a 3D Global Convolutional Adversarial Network (3D GCA-Net) to address efficient prostate MR volume segmentation.
Training Neural Networks Using Features Replay
Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.
Faster Derivative-Free Stochastic Algorithm for Shared Memory Machines
Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.
Direct Shape Regression Networks for End-to-End Face Alignment
In this paper, we propose the direct shape regression network (DSRN) for end-to-end face alignment by jointly handling the aforementioned challenges in a unified framework.
Ranked #16 on
Face Alignment
on AFLW-19
Unsupervised Deep Generative Adversarial Hashing Network
HashGAN consists of three networks, a generator, a discriminator and an encoder.
Decoupled Parallel Backpropagation with Convergence Guarantee
The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.
Group Sparse Additive Machine
Among them, learning models with grouped variables have shown competitive performance for prediction and variable selection.
Regularized Modal Regression with Applications in Cognitive Impairment Prediction
Linear regression models have been successfully used to function estimation and model selection in high-dimensional data analysis.
Learning A Structured Optimal Bipartite Graph for Co-Clustering
In graph based co-clustering methods, a bipartite graph is constructed to depict the relation between features and samples.
Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization
To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.
Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes
Image analysis using more than one modality (i. e. multi-modal) has been increasingly applied in the field of biomedical imaging.
Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics
In recent years, analyzing task-based fMRI (tfMRI) data has become an essential tool for understanding brain function and networks.
Locally-Transferred Fisher Vectors for Texture Classification
Texture classification has been extensively studied in computer vision.
Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization
We define a clustering objective function using relative entropy (KL divergence) minimization, regularized by a prior for the frequency of cluster assignments.
Ranked #1 on
Image Clustering
on FRGC
Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization
The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.
Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction
The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).
Error Analysis of Generalized Nyström Kernel Regression
Nystr\"{o}m method has been used successfully to improve the computational efficiency of kernel ridge regression (KRR).
Asynchronous Stochastic Block Coordinate Descent with Variance Reduction
In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.
A Closed Form Solution to Multi-View Low-Rank Regression
Real life data often includes information from different channels.
A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification
In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix.
Decoupled Asynchronous Proximal Stochastic Gradient Descent with Variance Reduction
In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).
Distributed Asynchronous Dual Free Stochastic Dual Coordinate Ascent
Our method does not need the dual formulation of the target problem in the optimization.
Asynchronous Stochastic Gradient Descent with Variance Reduction for Non-Convex Optimization
We provide the first theoretical analysis on the convergence rate of the asynchronous stochastic variance reduced gradient (SVRG) descent algorithm on non-convex optimization.
Enhancing Sentence Relation Modeling with Auxiliary Character-level Embedding
Neural network based approaches for sentence relation modeling automatically generate hidden matching features from raw sentence pairs.
Clinical Information Extraction via Convolutional Neural Network
We report an implementation of a clinical information extraction tool that leverages deep neural network to annotate event spans and their attributes from raw clinical notes and pathology reports.
Non-Greedy L21-Norm Maximization for Principal Component Analysis
In this paper, we propose to maximize the L21-norm based robust PCA objective, which is theoretically connected to the minimization of reconstruction error.
Multi-View Subspace Clustering
In this paper, we propose a novel multi-view subspace clustering method.
Theoretic Analysis and Extremely Easy Algorithms for Domain Adaptive Feature Learning
Domain adaptation problems arise in a variety of applications, where a training dataset from the \textit{source} domain and a test dataset from the \textit{target} domain typically follow different distributions.
Fusing Subcategory Probabilities for Texture Classification
Texture, as a fundamental characteristic of objects, has attracted much attention in computer vision research.
Improved Spectral Clustering via Embedded Label Propagation
Our algorithm is built upon two advancements of the state of the art:1) label propagation, which propagates a node\'s labels to neighboring nodes according to their proximity; and 2) manifold learning, which has been widely used in its capacity to leverage the manifold structure of data points.
A Convex Sparse PCA for Feature Analysis
In addition, based on the sparse model used in CSPCA, an optimal weight is assigned to each of the original feature, which in turn provides the output with good interpretability.
Video Motion Segmentation Using New Adaptive Manifold Denoising Model
After that, we employ an embedded manifold denoising approach with the adaptive kernel to segment the motion of rigid and non-rigid objects.
Heterogeneous Visual Features Fusion via Sparse Multimodal Machine
We applied our SMML method to five broadly used object categorization and scene understanding image data sets for both singlelabel and multi-label image classification tasks.
Forging The Graphs: A Low Rank and Positive Semidefinite Graph Learning Approach
In many graph-based machine learning and data mining approaches, the quality of the graph is critical.
High-Order Multi-Task Feature Learning to Identify Longitudinal Phenotypic Markers for Alzheimer's Disease Progression Prediction
Alzheimer disease (AD) is a neurodegenerative disorder characterized by progressive impairment of memory and other cognitive functions.
Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization
The ℓ2, 1-norm based loss function is robust to outliers in data points and the ℓ2, 1-norm regularization selects features across all data points with joint sparsity.
