Search Results for author: Wei Wen

Found 38 papers, 18 papers with code

Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale

no code implementations14 Nov 2023 Wei Wen, Kuang-Hung Liu, Igor Fedorov, Xin Zhang, Hang Yin, Weiwei Chu, Kaveh Hassani, Mengying Sun, Jiang Liu, Xu Wang, Lin Jiang, Yuxin Chen, Buyun Zhang, Xi Liu, Dehua Cheng, Zhengxing Chen, Guang Zhao, Fangqiu Han, Jiyan Yang, Yuchen Hao, Liang Xiong, Wen-Yen Chen

In industry system, such as ranking system in Meta, it is unclear whether NAS algorithms from the literature can outperform production baselines because of: (1) scale - Meta ranking systems serve billions of users, (2) strong baselines - the baselines are production models optimized by hundreds to thousands of world-class engineers for years since the rise of deep learning, (3) dynamic baselines - engineers may have established new and stronger baselines during NAS search, and (4) efficiency - the search pipeline must yield results quickly in alignment with the productionization life cycle.

Neural Architecture Search

DistDNAS: Search Efficient Feature Interactions within 2 Hours

no code implementations1 Nov 2023 Tunhou Zhang, Wei Wen, Igor Fedorov, Xi Liu, Buyun Zhang, Fangqiu Han, Wen-Yen Chen, Yiping Han, Feng Yan, Hai Li, Yiran Chen

To optimize search efficiency, DistDNAS distributes the search and aggregates the choice of optimal interaction modules on varying data dates, achieving over 25x speed-up and reducing search cost from 2 days to 2 hours.

Recommendation Systems

Farthest Greedy Path Sampling for Two-shot Recommender Search

no code implementations31 Oct 2023 Yufan Cao, Tunhou Zhang, Wei Wen, Feng Yan, Hai Li, Yiran Chen

FGPS enhances path diversity to facilitate more comprehensive supernet exploration, while emphasizing path quality to ensure the effective identification and utilization of promising architectures.

Click-Through Rate Prediction Neural Architecture Search

D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation

1 code implementation ICCV 2023 Hanjun Li, Xiujun Shu, Sunan He, Ruizhi Qiao, Wei Wen, Taian Guo, Bei Gan, Xing Sun

Under this setup, we propose a Dynamic Gaussian prior based Grounding framework with Glance annotation (D3G), which consists of a Semantic Alignment Group Contrastive Learning module (SA-GCL) and a Dynamic Gaussian prior Adjustment module (DGA).

Contrastive Learning Sentence +1

Sex Differences in 6-Year Progression of White Matter Hyperintensities in Non-Demented Older Adults: Sydney Memory and Ageing Study

no code implementations17 Jul 2023 Abdullah Alqarni, Wei Wen, Ben C. P. Lam, Nicole Kochan, Henry Brodaty, Perminder S. Sachdev, Jiyang Jiang

Conclusion: The findings highlighted sex differences in the associations between WMH progression and cognition decline over time, suggesting sex-specific strategies in managing WMH accumulations in ageing.

See Finer, See More: Implicit Modality Alignment for Text-based Person Retrieval

1 code implementation18 Aug 2022 Xiujun Shu, Wei Wen, Haoqian Wu, Keyu Chen, Yiran Song, Ruizhi Qiao, Bo Ren, Xiao Wang

To explore the fine-grained alignment, we further propose two implicit semantic alignment paradigms: multi-level alignment (MLA) and bidirectional mask modeling (BMM).

Person Retrieval Retrieval +3

Exploiting Feature Diversity for Make-up Temporal Video Grounding

no code implementations12 Aug 2022 Xiujun Shu, Wei Wen, Taian Guo, Sunan He, Chen Wu, Ruizhi Qiao

This technical report presents the 3rd winning solution for MTVG, a new task introduced in the 4-th Person in Context (PIC) Challenge at ACM MM 2022.

Video Grounding

NASRec: Weight Sharing Neural Architecture Search for Recommender Systems

2 code implementations14 Jul 2022 Tunhou Zhang, Dehua Cheng, Yuchen He, Zhengxing Chen, Xiaoliang Dai, Liang Xiong, Feng Yan, Hai Li, Yiran Chen, Wei Wen

To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i. e., search space) to search the full architectures.

Click-Through Rate Prediction Neural Architecture Search +1

Hormonal Factors Moderate the Associations Between Vascular Risk Factors and White Matter Hyperintensities

no code implementations12 May 2022 Abdullah Alqarni, Wei Wen, Ben C. P. Lam, John D. Crawford, Perminder S. Sachdev, Jiyang Jiang

Generalised linear models were applied to examine 1) the main effects of vascular (body mass index, hip to waist ratio, pulse wave velocity, hypercholesterolemia, diabetes, hypertension, smoking status) and hormonal (testosterone levels, contraceptive pill, hormone replacement therapy, menopause) factors on WMH, and 2) the moderation effects of hormonal factors on the relationship between vascular risk factors and WMH volumes.


Computer-Aided Extraction of Select MRI Markers of Cerebral Small Vessel Disease: A Systematic Review

no code implementations4 Apr 2022 Jiyang Jiang, Dadong Wang, Yang song, Perminder S. Sachdev, Wei Wen

Cerebral small vessel disease (CSVD) is a major vascular contributor to cognitive impairment in ageing, including dementias.

Transfer Learning

Network resilience in the aging brain

no code implementations3 Feb 2022 Tao Liu, Shu Guo, Hao liu, Rui Kang, Mingyang Bai, Jiyang Jiang, Wei Wen, Xing Pan, Jun Tai, JianXin Li, Jian Cheng, Jing Jing, Zhenzhou Wu, Haijun Niu, Haogang Zhu, Zixiao Li, Yongjun Wang, Henry Brodaty, Perminder Sachdev, Daqing Li

Degeneration and adaptation are two competing sides of the same coin called resilience in the progressive processes of brain aging or diseases.

Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss

1 code implementation6 Jun 2021 Jian Cheng, Ziyang Liu, Hao Guan, Zhenzhou Wu, Haogang Zhu, Jiyang Jiang, Wei Wen, DaCheng Tao, Tao Liu

In this paper, a novel 3D convolutional network, called two-stage-age-network (TSAN), is proposed to estimate brain age from T1-weighted MRI data.

Age Estimation

TRP: Trained Rank Pruning for Efficient Deep Neural Networks

1 code implementation30 Apr 2020 Yuhui Xu, Yuxi Li, Shuai Zhang, Wei Wen, Botao Wang, Yingyong Qi, Yiran Chen, Weiyao Lin, Hongkai Xiong

The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss, thus eliminating the fine-tuning process after low rank decomposition.

Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsification

1 code implementation20 Apr 2020 Huanrui Yang, Minxue Tang, Wei Wen, Feng Yan, Daniel Hu, Ang Li, Hai Li, Yiran Chen

In this work, we propose SVD training, the first method to explicitly achieve low-rank DNNs during training without applying SVD on every step.

Neural Predictor for Neural Architecture Search

2 code implementations ECCV 2020 Wei Wen, Hanxiao Liu, Hai Li, Yiran Chen, Gabriel Bender, Pieter-Jan Kindermans

First we train N random architectures to generate N (architecture, validation accuracy) pairs and use them to train a regression model that predicts accuracy based on the architecture.

Neural Architecture Search regression

Trained Rank Pruning for Efficient Deep Neural Networks

1 code implementation9 Oct 2019 Yuhui Xu, Yuxi Li, Shuai Zhang, Wei Wen, Botao Wang, Wenrui Dai, Yingyong Qi, Yiran Chen, Weiyao Lin, Hongkai Xiong

To accelerate DNNs inference, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations.

Conditional Transferring Features: Scaling GANs to Thousands of Classes with 30% Less High-quality Data for Training

no code implementations25 Sep 2019 Chunpeng Wu, Wei Wen, Yiran Chen, Hai Li

As such, training our GAN architecture requires much fewer high-quality images with a small number of additional low-quality images.

Generative Adversarial Network Image Generation

DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures

1 code implementation ICLR 2020 Huanrui Yang, Wei Wen, Hai Li

Inspired by the Hoyer measure (the ratio between L1 and L2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant.

Efficient Neural Network

Joint Regularization on Activations and Weights for Efficient Neural Network Pruning

no code implementations19 Jun 2019 Qing Yang, Wei Wen, Zuoguan Wang, Hai Li

With the rapid scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for improving deployment efficiency.

Efficient Neural Network Model Compression +1

Integral Pruning on Activations and Weights for Efficient Neural Networks

no code implementations ICLR 2019 Qing Yang, Wei Wen, Zuoguan Wang, Yiran Chen, Hai Li

With the rapidly scaling up of deep neural networks (DNNs), extensive research studies on network model compression such as weight pruning have been performed for efficient deployment.

Model Compression

PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration

1 code implementation26 Jan 2019 Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez

State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights.

Trained Rank Pruning for Efficient Deep Neural Networks

1 code implementation6 Dec 2018 Yuhui Xu, Yuxi Li, Shuai Zhang, Wei Wen, Botao Wang, Yingyong Qi, Yiran Chen, Weiyao Lin, Hongkai Xiong

We propose Trained Rank Pruning (TRP), which iterates low rank approximation and training.


Mini-batch Serialization: CNN Training with Inter-layer Data Reuse

1 code implementation30 Sep 2018 Sangkug Lym, Armand Behroozi, Wei Wen, Ge Li, Yongkee Kwon, Mattan Erez

Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth.

Learning Intrinsic Sparse Structures within Long Short-Term Memory

no code implementations ICLR 2018 Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, Hai Li

This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs.

Language Modelling Model Compression +1

Coordinating Filters for Faster Deep Neural Networks

5 code implementations ICCV 2017 Wei Wen, Cong Xu, Chunpeng Wu, Yandan Wang, Yiran Chen, Hai Li

Moreover, Force Regularization better initializes the low-rank DNNs such that the fine-tuning can converge faster toward higher accuracy.

Group Scissor: Scaling Neuromorphic Computing Design to Large Neural Networks

no code implementations11 Feb 2017 Yandan Wang, Wei Wen, Beiye Liu, Donald Chiarulli, Hai Li

Following rank clipping, group connection deletion further reduces the routing area of LeNet and ConvNet to 8. 1\% and 52. 06\%, respectively.

Classification Accuracy Improvement for Neuromorphic Computing Systems with One-level Precision Synapses

no code implementations7 Jan 2017 Yandan Wang, Wei Wen, Linghao Song, Hai Li

Brain inspired neuromorphic computing has demonstrated remarkable advantages over traditional von Neumann architecture for its high energy efficiency and parallel data processing.

General Classification Image Classification +1

Learning Structured Sparsity in Deep Neural Networks

3 code implementations NeurIPS 2016 Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, Hai Li

SSL can: (1) learn a compact structure from a bigger DNN to reduce computation cost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently accelerate the DNNs evaluation.

Faster CNNs with Direct Sparse Convolutions and Guided Pruning

1 code implementation4 Aug 2016 Jongsoo Park, Sheng Li, Wei Wen, Ping Tak Peter Tang, Hai Li, Yiran Chen, Pradeep Dubey

Pruning CNNs in a way that increase inference speed often imposes specific sparsity structures, thus limiting the achievable sparsity levels.

Cannot find the paper you are looking for? You can Submit a new open access paper.