Search Results for author: Lei Huang

Found 57 papers, 27 papers with code

WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks

1 code implementation16 Feb 2023 Shaowu Chen, Weize Sun, Lei Huang

Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks.

Classification Network Pruning +1

Scale-Semantic Joint Decoupling Network for Image-text Retrieval in Remote Sensing

no code implementations12 Dec 2022 Chengyu Zheng, Ning Song, Ruoyu Zhang, Lei Huang, Zhiqiang Wei, Jie Nie

To address these issues, we propose a novel Scale-Semantic Joint Decoupling Network (SSJDN) for remote sensing image-text retrieval.

Cross-Modal Retrieval Retrieval +1

FIXED: Frustratingly Easy Domain Generalization with Mixup

1 code implementation7 Nov 2022 Wang Lu, Jindong Wang, Han Yu, Lei Huang, Xiang Zhang, Yiqiang Chen, Xing Xie

Firstly, Mixup cannot effectively identify the domain and class information that can be used for learning invariant representations.

Domain Generalization Image Classification +1

Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning

no code implementations22 Oct 2022 Xuhua Li, Weize Sun, Lei Huang, Shaowu Chen

Filter pruning is a common method to achieve model compression and acceleration in deep neural networks (DNNs). Some research regarded filter pruning as a combinatorial optimization problem and thus used evolutionary algorithms (EA) to prune filters of DNNs.

Combinatorial Optimization Model Compression

Understanding the Failure of Batch Normalization for Transformers in NLP

1 code implementation11 Oct 2022 Jiaxi Wang, Ji Wu, Lei Huang

Batch Normalization (BN) is a core and prevalent technique in accelerating the training of deep neural networks and improving the generalization on Computer Vision (CV) tasks.

Image Classification Language Modelling +3

Towards Better Understanding with Uniformity and Explicit Regularization of Embeddings in Embedding-based Neural Topic Models

no code implementations16 Jun 2022 Wei Shao, Lei Huang, Shuqi Liu, Shihua Ma, Linqi Song

In this paper, we propose an embedding regularized neural topic model, which applies the specially designed training constraints on word embedding and topic embedding to reduce the optimization space of parameters.

Topic Models

Bi-level Doubly Variational Learning for Energy-based Latent Variable Models

no code implementations CVPR 2022 Ge Kan, Jinhu Lü, Tian Wang, Baochang Zhang, Aichun Zhu, Lei Huang, Guodong Guo, Hichem Snoussi

In this paper, we propose Bi-level doubly variational learning (BiDVL), which is based on a new bi-level optimization framework and two tractable variational distributions to facilitate learning EBLVMs.

Image Generation Image Reconstruction +1

Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective

1 code implementation CVPR 2022 Jiawei Zhang, Xiang Wang, Xiao Bai, Chen Wang, Lei Huang, Yimin Chen, Lin Gu, Jun Zhou, Tatsuya Harada, Edwin R. Hancock

The stereo contrastive feature loss function explicitly constrains the consistency between learned features of matching pixel pairs which are observations of the same 3D points.

Contrastive Learning Stereo Matching

Delving into the Estimation Shift of Batch Normalization in a Network

1 code implementation CVPR 2022 Lei Huang, Yi Zhou, Tian Wang, Jie Luo, Xianglong Liu

We define the estimation shift magnitude of BN to quantitatively measure the difference between its estimated population statistics and expected ones.

Towards Defending Multiple $\ell_p$-Norm Bounded Adversarial Perturbations via Gated Batch Normalization

no code implementations29 Sep 2021 Aishan Liu, Shiyu Tang, Xianglong Liu, Xinyun Chen, Lei Huang, Haotong Qin, Dawn Song, DaCheng Tao

We observe that different $\ell_p$ bounded adversarial perturbations induce different statistical properties that can be separated and characterized by the statistics of Batch Normalization (BN).

Joint Matrix Decomposition for Deep Convolutional Neural Networks Compression

1 code implementation9 Jul 2021 Shaowu Chen, Jiahao Zhou, Weize Sun, Lei Huang

To overcome this problem, we propose to compress CNNs and alleviate performance degradation via joint matrix decomposition, which is different from existing works that compressed layers separately.

Efficient Neural Network Matrix Factorization / Decomposition +1

KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks

2 code implementations4 Jul 2021 J. Gregory Pauloski, Qi Huang, Lei Huang, Shivaram Venkataraman, Kyle Chard, Ian Foster, Zhao Zhang

Kronecker-factored Approximate Curvature (K-FAC) has recently been shown to converge faster in deep neural network (DNN) training than stochastic gradient descent (SGD); however, K-FAC's larger memory footprint hinders its applicability to large models.

Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification

no code implementations26 Feb 2021 Yi Zhou, Lei Huang, Tianfei Zhou, Ling Shao

For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming.

S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration

1 code implementation CVPR 2021 Zhiqiang Shen, Zechun Liu, Jie Qin, Lei Huang, Kwang-Ting Cheng, Marios Savvides

In this paper, we focus on this more difficult scenario: learning networks where both weights and activations are binary, meanwhile, without any human annotated labels.

Contrastive Learning Self-Supervised Learning

EGFI: Drug-Drug Interaction Extraction and Generation with Fusion of Enriched Entity and Sentence Information

1 code implementation25 Jan 2021 Lei Huang, Jiecong Lin, Xiangtao Li, Linqi Song, Ka-Chun Wong

To address such a problem, we propose EGFI for extracting and consolidating drug interactions from large-scale medical literature text data.

Classification Drug–drug Interaction Extraction +2

Visual-Textual Attentive Semantic Consistency for Medical Report Generation

no code implementations ICCV 2021 Yi Zhou, Lei Huang, Tao Zhou, Huazhu Fu, Ling Shao

Second, the progressive report decoder consists of a sentence decoder and a word decoder, where we propose image-sentence matching and description accuracy losses to constrain the visual-textual semantic consistency.

Medical Report Generation Word Embeddings

Slimmable Generative Adversarial Networks

1 code implementation10 Dec 2020 Liang Hou, Zehuan Yuan, Lei Huang, HuaWei Shen, Xueqi Cheng, Changhu Wang

In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power.

Towards Defending Multiple Adversarial Perturbations via Gated Batch Normalization

no code implementations3 Dec 2020 Aishan Liu, Shiyu Tang, Xianglong Liu, Xinyun Chen, Lei Huang, Zhuozhuo Tu, Dawn Song, DaCheng Tao

To better understand this phenomenon, we propose the \emph{multi-domain} hypothesis, stating that different types of adversarial perturbations are drawn from different domains.

Group Whitening: Balancing Learning Efficiency and Representational Capacity

1 code implementation CVPR 2021 Lei Huang, Yi Zhou, Li Liu, Fan Zhu, Ling Shao

Results show that GW consistently improves the performance of different architectures, with absolute gains of $1. 02\%$ $\sim$ $1. 49\%$ in top-1 accuracy on ImageNet and $1. 82\%$ $\sim$ $3. 21\%$ in bounding box AP on COCO.

Normalization Techniques in Training DNNs: Methodology, Analysis and Application

no code implementations27 Sep 2020 Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

Normalization techniques are essential for accelerating the training and improving the generalization of deep neural networks (DNNs), and have successfully been used in various applications.

A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability

no code implementations22 Aug 2020 Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, Ling Shao

This dataset has 1, 842 images with pixel-level DR-related lesion annotations, and 1, 000 images with image-level labels graded by six board-certified ophthalmologists with intra-rater consistency.

Lesion Segmentation Transfer Learning

Invertible Zero-Shot Recognition Flows

1 code implementation ECCV 2020 Yuming Shen, Jie Qin, Lei Huang

Deep generative models have been successfully applied to Zero-Shot Learning (ZSL) recently.

Zero-Shot Learning

Convolutional Neural Network Training with Distributed K-FAC

3 code implementations1 Jul 2020 J. Gregory Pauloski, Zhao Zhang, Lei Huang, Weijia Xu, Ian T. Foster

Training neural networks with many processors can reduce time-to-solution; however, it is challenging to maintain convergence and efficiency at large scales.

Topic Detection and Summarization of User Reviews

no code implementations30 May 2020 Pengyuan Li, Lei Huang, Guang-jie Ren

As the sentiments are typically short, we combine sentiments talking about the same aspect into a single document and apply topic modeling method to identify hidden topics among customer reviews and summaries.

Sentiment Analysis

Controllable Orthogonalization in Training DNNs

1 code implementation CVPR 2020 Lei Huang, Li Liu, Fan Zhu, Diwen Wan, Zehuan Yuan, Bo Li, Ling Shao

Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1 and reduce redundancy in representation.

Image Classification

An Efficient Agreement Mechanism in CapsNets By Pairwise Product

1 code implementation1 Apr 2020 Lei Zhao, Xiaohui Wang, Lei Huang

Capsule networks (CapsNets) are capable of modeling visual hierarchical relationships, which is achieved by the "routing-by-agreement" mechanism.

An Investigation into the Stochasticity of Batch Whitening

1 code implementation CVPR 2020 Lei Huang, Lei Zhao, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

Our work originates from the observation that while various whitening transformations equivalently improve the conditioning, they show significantly different behaviors in discriminative scenarios and training Generative Adversarial Networks (GANs).

Layer-wise Conditioning Analysis in Exploring the Learning Dynamics of DNNs

no code implementations ECCV 2020 Lei Huang, Jie Qin, Li Liu, Fan Zhu, Ling Shao

To this end, we propose layer-wise conditioning analysis, which explores the optimization landscape with respect to each layer independently.

Deformable Tube Network for Action Detection in Videos

no code implementations3 Jul 2019 Wei Li, Zehuan Yuan, Dashan Guo, Lei Huang, Xiangzhong Fang, Changhu Wang

To perform action detection, we design a 3D convolution network with skip connections for tube classification and regression.

Action Detection Action Recognition

Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search

no code implementations18 Apr 2019 Xianglong Liu, Lei Huang, Cheng Deng, Bo Lang, DaCheng Tao

For each hash table, a query-adaptive bitwise weighting is introduced to alleviate the quantization loss by simultaneously exploiting the quality of hash functions and their complement for nearest neighbor search.

Image Retrieval Quantization +1

Iterative Normalization: Beyond Standardization towards Efficient Whitening

5 code implementations CVPR 2019 Lei Huang, Yi Zhou, Fan Zhu, Li Liu, Ling Shao

With the support of SND, we provide natural explanations to several phenomena from the perspective of optimization, e. g., why group-wise whitening of DBN generally outperforms full-whitening and why the accuracy of BN degenerates with reduced batch sizes.

Analyzing Periodicity and Saliency for Adult Video Detection

no code implementations11 Jan 2019 Yizhi Liu, Xiaoyan Gu, Lei Huang, Junlin Ouyang, Miao Liao, Liangran Wu

Content-based adult video detection plays an important role in preventing pornography.

Utilizing Complex-valued Network for Learning to Compare Image Patches

no code implementations29 Nov 2018 Siwen Jiang, Wenxuan Wei, Shihao Guo, Hongguang Fu, Lei Huang

At present, the great achievements of convolutional neural network(CNN) in feature and metric learning have attracted many researchers.

Metric Learning

FanStore: Enabling Efficient and Scalable I/O for Distributed Deep Learning

1 code implementation27 Sep 2018 Zhao Zhang, Lei Huang, Uri Manor, Linjing Fang, Gabriele Merlo, Craig Michoski, John Cazes, Niall Gaffney

Our experiments with benchmarks and real applications show that FanStore can scale DL training to 512 compute nodes with over 90\% scaling efficiency.

Distributed, Parallel, and Cluster Computing

Data-driven Analytics for Business Architectures: Proposed Use of Graph Theory

no code implementations5 Jun 2018 Lei Huang, Guang-jie Ren, Shun Jiang, Raphael Arar, Eric Young Liu

Business Architecture (BA) plays a significant role in helping organizations understand enterprise structures and processes, and align them with strategic objectives.

Decorrelated Batch Normalization

6 code implementations CVPR 2018 Lei Huang, Dawei Yang, Bo Lang, Jia Deng

Batch Normalization (BN) is capable of accelerating the training of deep models by centering and scaling activations within mini-batches.

Orthogonal Weight Normalization: Solution to Optimization overMultiple Dependent Stiefel Manifolds in Deep Neural Networks

1 code implementation The Thirty-Second AAAI Conferenceon Artificial Intelligence 2018 Lei Huang, Xianglong Liu, Bo Lang, Adams Wei Yu, Yongliang Wang, Bo Li

In this paper, we generalize such square orthogonal matrix to orthogonal rectangular matrix and formulating this problem in feed-forward Neural Networks (FNNs) as Optimization over Multiple Dependent Stiefel Manifolds (OMDSM).

DGCNN: Disordered Graph Convolutional Neural Network Based on the Gaussian Mixture Model

no code implementations10 Dec 2017 Bo Wu, Yang Liu, Bo Lang, Lei Huang

Convolutional neural networks (CNNs) can be applied to graph similarity matching, in which case they are called graph CNNs.

Graph Classification Graph Similarity +1

Projection Based Weight Normalization for Deep Neural Networks

1 code implementation6 Oct 2017 Lei Huang, Xianglong Liu, Bo Lang, Bo Li

We conduct comprehensive experiments on several widely-used image datasets including CIFAR-10, CIFAR-100, SVHN and ImageNet for supervised learning over the state-of-the-art convolutional neural networks, such as Inception, VGG and residual networks.

Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks

1 code implementation16 Sep 2017 Lei Huang, Xianglong Liu, Bo Lang, Adams Wei Yu, Yongliang Wang, Bo Li

In this paper, we generalize such square orthogonal matrix to orthogonal rectangular matrix and formulating this problem in feed-forward Neural Networks (FNNs) as Optimization over Multiple Dependent Stiefel Manifolds (OMDSM).

Image Classification

Block-Normalized Gradient Method: An Empirical Study for Training Deep Neural Network

2 code implementations ICLR 2018 Adams Wei Yu, Lei Huang, Qihang Lin, Ruslan Salakhutdinov, Jaime Carbonell

In this paper, we propose a generic and simple strategy for utilizing stochastic gradient information in optimization.

Micro Fourier Transform Profilometry ($μ$FTP): 3D shape measurement at 10,000 frames per second

no code implementations31 May 2017 Chao Zuo, Tianyang Tao, Shijie Feng, Lei Huang, Anand Asundi, Qian Chen

Recent advances in imaging sensors and digital light projection technology have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces of complex-shaped objects to be captured with improved resolution and accuracy.

Varying-smoother models for functional responses

no code implementations2 Dec 2014 Philip T. Reiss, Lei Huang, Huaihou Chen, Stan Colcombe

We discuss three approaches to estimating varying-smoother models: (a) methods that employ a tensor product penalty; (b) an approach based on smoothed functional principal component scores; and (c) two-step methods consisting of an initial smooth with respect to $t$ at each $s$, followed by a postprocessing step.


Cannot find the paper you are looking for? You can Submit a new open access paper.