Search Results for author:
Found 15 papers, 9 papers with code
Accelerating Deep Learning with Millions of Classes
In our experiment, we demonstrate that the proposed frame-work is able to train deep learning models with millions of classes and achieve above 10×speedup compared to existing approaches.
CLoVe: Encoding Compositional Language in Contrastive Vision-Language Models
Recent years have witnessed a significant increase in the performance of Vision and Language tasks.
LibAUC: A Deep Learning Library for X-Risk Optimization
This paper introduces the award-winning deep learning (DL) library called LibAUC for implementing state-of-the-art algorithms towards optimizing a family of risk functions named X-risks.
Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization
In this paper, we aim to optimize a contrastive loss with individualized temperatures in a principled and systematic manner for self-supervised learning.
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance
In this paper, we study contrastive learning from an optimization perspective, aiming to analyze and address a fundamental issue of existing contrastive learning methods that either rely on a large batch size or a large dictionary of feature vectors.
Compositional Training for End-to-End Deep AUC Maximization
The key idea of compositional training is to minimize a compositional objective function, where the outer function corresponds to an AUC loss and the inner function represents a gradient descent step for minimizing a traditional loss, e. g., the cross-entropy (CE) loss.
Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
The proposed algorithms require sampling a constant number of tasks and data samples per iteration, making them suitable for the continual learning scenario.
Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity
Deep AUC (area under the ROC curve) Maximization (DAM) has attracted much attention recently due to its great potential for imbalanced data classification.
Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification
Our studies demonstrate that the proposed DAM method improves the performance of optimizing cross-entropy loss by a large margin, and also achieves better performance than optimizing the existing AUC square loss on these medical image classification tasks.
Ranked #2 on
Multi-Label Classification
on CheXpert
Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery
Here we develop a suite of comprehensive machine learning methods and tools spanning different computational models, molecular representations, and loss functions for molecular property prediction and drug discovery.
Fast Objective & Duality Gap Convergence for Non-Convex Strongly-Concave Min-Max Problems with PL Condition
However, most of the existing algorithms are slow in practice, and their analysis revolves around the convergence to a nearly stationary point. We consider leveraging the Polyak-Lojasiewicz (PL) condition to design faster stochastic algorithms with stronger convergence guarantee.
Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks
In this paper, we study distributed algorithms for large-scale AUC maximization with a deep neural network as a predictive model.
Stochastic AUC Maximization with Deep Neural Networks
In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model.
Stagewise Training Accelerates Convergence of Testing Error Over SGD
For convex loss functions and two classes of "nice-behaviored" non-convex objectives that are close to a convex function, we establish faster convergence of stagewise training than the vanilla SGD under the PL condition on both training error and testing error.
Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions
For example, there is still a lack of theories of convergence for SGD and its variants that use stagewise step size and return an averaged solution in practice.
