Search Results for author:
Found 98 papers, 31 papers with code
Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search
It is able to find top 0. 16\% and 0. 29\% architectures on average on two search spaces under the budget of only 50 models.
ManiTrans: Entity-Level Text-Guided Image Manipulation via Token-wise Semantic Alignment and Generation
Existing text-guided image manipulation methods aim to modify the appearance of the image or to edit a few objects in a virtual or simple scenario, which is far from practical application.
Generalizing Few-Shot NAS with Gradient Matching
Efficient performance estimation of architectures drawn from large search spaces is essential to Neural Architecture Search.
On Redundancy and Diversity in Cell-based Neural Architecture Search
Searching for the architecture cells is a dominant paradigm in NAS.
CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving
One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases.
Revisiting Over-smoothing in BERT from the Perspective of Graph
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields.
Memory Replay with Data Compression for Continual Learning
In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.
DropNAS: Grouped Operation Dropout for Differentiable Architecture Search
Neural architecture search (NAS) has shown encouraging results in automating the architecture design.
Long-tail Recognition via Compositional Knowledge Transfer
In this work, we introduce a novel strategy for long-tail recognition that addresses the tail classes' few-shot problem via training-free knowledge transfer.
Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks
Gradient-based methods for the distributed training of residual networks (ResNets) typically require a forward pass of the input data, followed by back-propagating the error gradient to update model parameters, which becomes time-consuming as the network goes deeper.
Understanding Square Loss in Training Overparametrized Neural Network Classifiers
Comparing to cross-entropy, square loss has comparable generalization error but noticeable advantages in robustness and model calibration.
On Effective Scheduling of Model-based Reinforcement Learning
Model-based reinforcement learning has attracted wide attention due to its superior sample efficiency.
MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps
First, we theoretically show the transferability of robustness from an adversarially trained teacher model to a student model with the help of mixup augmentation.
FILIP: Fine-grained Interactive Language-Image Pre-Training
In this paper, we introduce a large-scale Fine-grained Interactive Language-Image Pre-training (FILIP) to achieve finer-level alignment through a cross-modal late interaction mechanism, which uses a token-wise maximum similarity between visual and textual tokens to guide the contrastive objective.
AIM: Automatic Interaction Machine for Click-Through Rate Prediction
To address these three issues mentioned above, we propose Automatic Interaction Machine (AIM) with three core components, namely, Feature Interaction Search (FIS), Interaction Function Search (IFS) and Embedding Dimension Search (EDS), to select significant feature interactions, appropriate interaction functions and necessary embedding dimensions automatically in a unified framework.
OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression
To eliminate the requirement of saving separate models for different target datasets, we propose a novel setting that starts from a pretrained deep generative model and compresses the data batches while adapting the model with a dynamical system for only one epoch.
iFlow: Numerically Invertible Flows for Efficient Lossless Compression via a Uniform Coder
In this paper, we discuss lossless compression using normalizing flows which have demonstrated a great capacity for achieving high compression ratios.
Nonlinear ICA Using Volume-Preserving Transformations
Nonlinear ICA is a fundamental problem in machine learning, aiming to identify the underlying independent components (sources) from data which is assumed to be a nonlinear function (mixing function) of these sources.
Rethinking Adversarial Transferability from a Data Distribution Perspective
Adversarial transferability enables attackers to generate adversarial examples from the source model to attack the target model, which has raised security concerns about the deployment of DNNs in practice.
How Well Does Self-Supervised Pre-Training Perform with Streaming ImageNet?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
Layer-Parallel Training of Residual Networks with Auxiliary Variables
Backpropagation algorithm is indispensable for training modern residual networks (ResNets) and usually tends to be time-consuming due to its inherent algorithmic lockings.
DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture
In view of these, we propose DHA, which achieves joint optimization of Data augmentation policy, Hyper-parameter and Architecture.
NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization
In this work, we propose robust Neural Architecture Search for OoD generalization (NAS-OoD), which optimizes the architecture with respect to its performance on generated OoD data by gradient descent.
Ranked #1 on
Domain Generalization
on NICO Animal
Adversarial Robustness for Unsupervised Domain Adaptation
Extensive Unsupervised Domain Adaptation (UDA) studies have shown great success in practice by learning transferable representations across a labeled source domain and an unlabeled target domain with deep models.
MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving
By pre-training on SODA10M, a large-scale autonomous driving dataset, MultiSiam exceeds the ImageNet pre-trained MoCo-v2, demonstrating the potential of domain-specific pre-training.
Towards Understanding the Generative Capability of Adversarially Robust Classifiers
We find that adversarial training contributes to obtaining an energy function that is flat and has low energy around the real data, which is the key for generative capability.
G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation
In this paper, we investigate the knowledge distillation (KD) strategy for object detection and propose an effective framework applicable to both homogeneous and heterogeneous student-teacher pairs.
NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models
Fine-tuning from pre-trained ImageNet models has been a simple, effective, and popular approach for various computer vision tasks.
AutoBERT-Zero: Evolving BERT Backbone from Scratch
Transformer-based pre-trained language models like BERT and its variants have recently achieved promising performance in various natural language processing (NLP) tasks.
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving
Experiments show that SODA10M can serve as a promising pre-training dataset for different self-supervised learning methods, which gives superior performance when fine-tuning with different downstream tasks (i. e., detection, semantic/instance segmentation) in autonomous driving domain.
One Million Scenes for Autonomous Driving: ONCE Dataset
To facilitate future research on exploiting unlabeled data for 3D detection, we additionally provide a benchmark in which we reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset.
Transformation Invariant Few-Shot Object Detection
To this end, we propose a simple yet effective Transformation Invariant Principle (TIP) that can be flexibly applied to various meta-learning models for boosting the detection performance on novel class objects.
Adversarial Invariant Learning
To the best of our knowledge, this is one of the first to adopt differentiable environment splitting method to enable stable predictions across environments without environment index information, which achieves the state-of-the-art performance on datasets with strong spurious correlation, such as Colored MNIST.
Contextualizing Multiple Tasks via Learning to Decompose
One single instance could possess multiple portraits and reveal diverse relationships with others according to different contexts.
Towards a Theoretical Framework of Out-of-Distribution Generalization
We also introduce a new concept of expansion function, which characterizes to what extent the variance is amplified in the test domains over the training domains, and therefore give a quantitative meaning of invariant features.
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation
For student morphism, weight inheritance strategy is adopted, allowing the student to flexibly update its architecture while fully utilize the predecessor's weights, which considerably accelerates the search; To facilitate dynamic distillation, an elastic teacher pool is trained via integrated progressive shrinking strategy, from which teacher detectors can be sampled without additional cost in subsequent searches.
TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on a single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whitening
To address this problem, we develop a probability-based pruning algorithm, called batch whitening channel pruning (BWCP), which can stochastically discard unimportant channels by modeling the probability of a channel being activated.
How Well Does Self-Supervised Pre-Training Perform with Streaming Data?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression
We also propose a lossless compression algorithm based on iVPF.
Deeply Unsupervised Patch Re-Identification for Pre-training Object Detectors
Unsupervised pre-training aims at learning transferable features that are beneficial for downstream tasks.
SparseBERT: Rethinking the Importance Analysis in Self-attention
A surprising result is that diagonal elements in the attention map are the least important compared with other attention positions.
Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search
For object detection, the well-established classification and regression loss functions have been carefully designed by considering diverse learning challenges.
DetCo: Unsupervised Contrastive Learning for Object Detection
Unlike most recent methods that focused on improving accuracy of image classification, we present a novel contrastive learning approach, named DetCo, which fully explores the contrasts between global image and local image patches to learn discriminative representations for object detection.
Out-of-Distribution Generalization Analysis via Influence Function
One of the definitions of OOD accuracy is worst-domain accuracy.
Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptation
Semi-supervised domain adaptation (SSDA), which aims to learn models in a partially labeled target domain with the assistance of the fully labeled source domain, attracts increasing attention in recent years.
ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning
Continual learning usually assumes the incoming data are fully labeled, which might not be applicable in real applications.
Optimal Designs of Gaussian Processes with Budgets for Hyperparameter Optimization
Moreover, comparisons between different initial designs with the same model show the advantage of the proposed optimal design.
Fewmatch: Dynamic Prototype Refinement for Semi-Supervised Few-Shot Learning
Semi-Supervised Few-shot Learning (SS-FSL) investigates the benefit of incorporating unlabelled data in few-shot settings.
SAD: Saliency Adversarial Defense without Adversarial Training
We justify the algorithm with a linear model that the added saliency maps pull data away from its closest decision boundary.
TransNAS-Bench-101: Improving Transferrability and Generalizability of Cross-Task Neural Architecture Search
While existing NAS methods mostly design architectures on one single task, algorithms that look beyond single-task search are surging to pursue a more efficient and universal solution across various tasks.
DiffAutoML: Differentiable Joint Optimization for Efficient End-to-End Automated Machine Learning
Our method performs co-optimization of the neural architectures, training hyper-parameters and data augmentation policies in an end-to-end fashion without the need of model retraining.
Exploring Geometry-Aware Contrast and Clustering Harmonization for Self-Supervised 3D Object Detection
Here we present a novel self-supervised 3D Object detection framework that seamlessly integrates the geometry-aware contrast and clustering harmonization to lift the unsupervised 3D representation learning, named GCC-3D.
NASOA: Towards Faster Task-oriented Online Fine-tuning
The resulting model zoo is more training efficient than SOTA NAS models, e. g. 6x faster than RegNetY-16GF, and 1. 7x faster than EfficientNetB3.
MetaAugment: Sample-Aware Data Augmentation Policy Learning
Automated data augmentation has shown superior performance in image recognition.
DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation
To address that, we propose DecAug, a novel decomposed feature representation and semantic augmentation approach for OoD generalization.
An Embedding Learning Framework for Numerical Features in CTR Prediction
In this paper, we propose a novel embedding learning framework for numerical features in CTR prediction (AutoDis) with high model capacity, end-to-end training and unique representation properties preserved.
Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
Panoptic segmentation that unifies instance segmentation and semantic segmentation has recently attracted increasing attention.
Ranked #14 on
Panoptic Segmentation
on COCO test-dev
Batch Group Normalization
Among previous normalization methods, Batch Normalization (BN) performs well at medium and large batch sizes and is with good generalizability to multiple vision tasks, while its performance degrades significantly at small batch sizes.
MOFA: Modular Factorial Design for Hyperparameter Optimization
MOFA pursues several rounds of HPO, where each round alternates between exploration of hyperparameter space by factorial design and exploitation of evaluation results by factorial analysis.
VEGA: Towards an End-to-End Configurable AutoML Pipeline
Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models.
A Practical Layer-Parallel Training Algorithm for Residual Networks
Gradient-based algorithms for training ResNets typically require a forward pass of the input data, followed by back-propagating the objective gradient to update parameters, which are time-consuming for deep ResNets.
CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending
In this paper, we propose a novel lane-sensitive architecture search framework named CurveLane-NAS to automatically capture both long-ranged coherent and accurate short-range curve information while unifying both architecture search and post-processing on curve lane predictions via point blending.
Ranked #4 on
Lane Detection
on CurveLanes
AABO: Adaptive Anchor Box Optimization for Object Detection via Bayesian Sub-sampling
By carefully analyzing the existing bounding box patterns on the feature hierarchy, we design a flexible and tight hyper-parameter space for anchor configurations.
CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search
In spite of its remarkable progress, many algorithms are restricted to particular search spaces.
Ranked #4 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (val) metric)
An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization
The evaluation of hyperparameters, neural architectures, or data augmentation policies becomes a critical model selection problem in advanced deep learning with a large hyperparameter search space.
Decoder-free Robustness Disentanglement without (Additional) Supervision
Adversarial Training (AT) is proposed to alleviate the adversarial vulnerability of machine learning models by extracting only robust features from the input, which, however, inevitably leads to severe accuracy reduction as it discards the non-robust yet useful features.
New Interpretations of Normalization Methods in Deep Learning
In recent years, a variety of normalization methods have been proposed to help train neural networks, such as batch normalization (BN), layer normalization (LN), weight normalization (WN), group normalization (GN), etc.
Risk Variance Penalization
The key of the out-of-distribution (OOD) generalization is to generalize invariance from training domains to target domains.
Locally Differentially Private (Contextual) Bandits Learning
We study locally differentially private (LDP) bandits learning in this paper.
Boosting Few-Shot Learning With Adaptive Margin Loss
Few-shot learning (FSL) has attracted increasing attention in recent years but remains challenging, due to the intrinsic difficulty in learning to generalize from a few examples.
Ranked #1 on
Few-Shot Image Classification
on ImageNet (1-shot)
Rethinking Performance Estimation in Neural Architecture Search
In this paper, we provide a novel yet systematic rethinking of PE in a resource constrained regime, termed budgeted PE (BPE), which precisely and effectively estimates the performance of an architecture sampled from an architecture space.
AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction
By implementing a regularized optimizer over the architecture parameters, the model can automatically identify and remove the redundant feature interactions during the training process of the model.
Ranked #15 on
Click-Through Rate Prediction
on Criteo
EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement
In this paper, we study the hybrid-supervised object detection problem, aiming to train a high quality detector with only a limited amount of fullyannotated data and fully exploiting cheap data with imagelevel labels.
Universal-RCNN: Universal Object Detector via Transferable Graph R-CNN
Finally, an InterDomain Transfer Module is proposed to exploit diverse transfer dependencies across all domains and enhance the regional feature representation by attending and transferring semantic contexts globally.
Multi-objective Neural Architecture Search via Non-stationary Policy Gradient
Despite recent progress, the problem of approximating the full Pareto front accurately and efficiently remains challenging.
MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
Recommender systems often face heterogeneous datasets containing highly personalized historical data of users, where no single model could give the best recommendation for every user.
Meta-Learning PAC-Bayes Priors in Model Averaging
In this paper, we mainly consider the scenario in which we have a common model set used for model averaging instead of selecting a single final model via a model selection procedure to account for this model's uncertainty to improve reliability and accuracy of inferences.
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection
In this paper, we present a two-stage coarse-to-fine searching strategy named Structural-to-Modular NAS (SM-NAS) for searching a GPU-friendly design of both an efficient combination of modules and better modular-level architecture for object detection.
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously.
Hierarchical Neural Architecture Search via Operator Clustering
Recently, the efficiency of automatic neural architecture design has been significantly improved by gradient-based search methods such as DARTS.
Multi-objective Neural Architecture Search via Predictive Network Performance Optimization
Inspired by the nature of the graph structure of a neural network, we propose BOGCN-NAS, a NAS algorithm using Bayesian Optimization with Graph Convolutional Network (GCN) predictor.
DARTS+: Improved Differentiable Architecture Search with Early Stopping
Therefore, we propose a simple and effective algorithm, named "DARTS+", to avoid the collapse and improve the original DARTS, by "early stopping" the search procedure when meeting a certain criterion.
Meta Reinforcement Learning with Task Embedding and Shared Policy
Despite significant progress, deep reinforcement learning (RL) suffers from data-inefficiency and limited generalization.
Formulating Camera-Adaptive Color Constancy as a Few-shot Meta-Learning Problem
In this work, we propose a new approach that affords fast adaptation to previously unseen cameras, and robustness to changes in capture device by leveraging annotated samples across different cameras and datasets.
DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
In this paper, we study two instances of DeepFM where its "deep" component is DNN and PNN respectively, for which we denote as DeepFM-D and DeepFM-P. Comprehensive experiments are conducted to demonstrate the effectiveness of DeepFM-D and DeepFM-P over the existing models for CTR prediction, on both benchmark data and commercial data.
Federated Meta-Learning with Fast Convergence and Efficient Communication
Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning.
Deep Meta-Learning: Learning to Learn in the Concept Space
Few-shot learning remains challenging for meta-learning that learns a learning algorithm (meta-learner) from many related tasks.
Graph Edge Partitioning via Neighborhood Heuristic
We provide a worst-case upper bound of replication factor for our heuristic on general graphs.
Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial.
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
Learning sophisticated feature interactions behind user behaviors is critical in maximizing CTR for recommender systems.
Ranked #1 on
Click-Through Rate Prediction
on Company*
New Insights Into Laplacian Similarity Search
Graph-based computer vision applications rely critically on similarity metrics which compute the pairwise similarity between any pair of vertices on graphs.
Locally Linear Hashing for Extracting Non-Linear Manifolds
Previous efforts in hashing intend to preserve data variance or pairwise affinity, but neither is adequate in capturing the manifold structures hidden in most visual data.
Analyzing the Harmonic Structure in Graph-Based Learning
We show that either explicitly or implicitly, various well-known graph-based models exhibit a common significant \emph{harmonic} structure in its target function -- the value of a vertex is approximately the weighted average of the values of its adjacent neighbors.
A Bayesian Approach to Multimodal Visual Dictionary Learning
nary learning methods rely on image descriptors alone or together with class labels.
Learning with Partially Absorbing Random Walks
We prove that under proper absorption rates, a random walk starting from a set $\mathcal{S}$ of low conductance will be mostly absorbed in $\mathcal{S}$.
Fast Graph Laplacian Regularized Kernel Learning via Semidefinite–Quadratic–Linear Programming
In this paper, we show that a large class of kernel learning problems can be reformulated as semidefinite-quadratic-linear programs (SQLPs), which only contain a simple positive semidefinite constraint, a second-order cone constraint and a number of linear constraints.
