Search Results for author: Haoyi Zhou

Found 12 papers, 3 papers with code

Building Flexible Machine Learning Models for Scientific Computing at Scale

no code implementations25 Feb 2024 Tianyu Chen, Haoyi Zhou, Ying Li, Hao Wang, Chonghan Gao, Shanghang Zhang, JianXin Li

Foundation models have revolutionized knowledge acquisition across domains, and our study introduces OmniArch, a paradigm-shifting approach designed for building foundation models in multi-physics scientific computing.

Zero-Shot Learning

PhoGAD: Graph-based Anomaly Behavior Detection with Persistent Homology Optimization

no code implementations19 Jan 2024 Ziqi Yuan, Haoyi Zhou, Tianyu Chen, JianXin Li

The analysis of persistent homology demonstrates its effectiveness in capturing the topological structure formed by normal edge features.

Anomaly Detection

Learning Music Sequence Representation from Text Supervision

no code implementations31 May 2023 Tianyu Chen, Yuan Xie, Shuai Zhang, Shaohan Huang, Haoyi Zhou, JianXin Li

Music representation learning is notoriously difficult for its complex human-related concepts contained in the sequence of numerical signals.

Contrastive Learning Representation Learning

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption

no code implementations Findings (ACL) 2022 Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, JianXin Li, Furu Wei

As more and more pre-trained language models adopt on-cloud deployment, the privacy issues grow quickly, mainly for the exposure of plain-text user data (e. g., search history, medical record, bank account).

Privacy Preserving

Task-Specific Expert Pruning for Sparse Mixture-of-Experts

no code implementations1 Jun 2022 Tianyu Chen, Shaohan Huang, Yuan Xie, Binxing Jiao, Daxin Jiang, Haoyi Zhou, JianXin Li, Furu Wei

The sparse Mixture-of-Experts (MoE) model is powerful for large-scale pre-training and has achieved promising results due to its model capacity.

MTTrans: Cross-Domain Object Detection with Mean-Teacher Transformer

1 code implementation3 May 2022 Jinze Yu, Jiaming Liu, Xiaobao Wei, Haoyi Zhou, Yohei Nakata, Denis Gudovskiy, Tomoyuki Okuno, JianXin Li, Kurt Keutzer, Shanghang Zhang

To solve this problem, we propose an end-to-end cross-domain detection Transformer based on the mean teacher framework, MTTrans, which can fully exploit unlabeled target domain data in object detection training and transfer knowledge between domains via pseudo labels.

Domain Adaptation Object +3

POLLA: Enhancing the Local Structure Awareness in Long Sequence Spatial-temporal Modeling

1 code implementation TIST 2021 2021 Haoyi Zhou, Hao Peng, Jieqi Peng, Shuai Zhang, JianXin Li

Extensive experiments are conducted on five large-scale datasets, which demonstrate that our method achieves state-of-the-art performance and validates the effectiveness brought by local structure information.

Decoder

Gradient Broadcast Adaptation: Defending against the backdoor attack in pre-trained models

no code implementations29 Sep 2021 Tianyu Chen, Haoyi Zhou, He Mingrui, JianXin Li

Pre-trained language models (e. g, BERT, GPT-3) have revolutionized the NLP research and fine-tuning becomes the indispensable step of downstream adaptation.

Backdoor Attack text-classification +1

RoSearch: Search for Robust Student Architectures When Distilling Pre-trained Language Models

no code implementations7 Jun 2021 Xin Guo, Jianlei Yang, Haoyi Zhou, Xucheng Ye, JianXin Li

In order to overcome these security problems, RoSearch is proposed as a comprehensive framework to search the student models with better adversarial robustness when performing knowledge distillation.

Adversarial Robustness Knowledge Distillation +1

Differentially-private Federated Neural Architecture Search

no code implementations16 Jun 2020 Ishika Singh, Haoyi Zhou, Kunlin Yang, Meng Ding, Bill Lin, Pengtao Xie

To address this problem, we propose federated neural architecture search (FNAS), where different parties collectively search for a differentiable architecture by exchanging gradients of architecture variables without exposing their data to other parties.

Neural Architecture Search

Stacked Kernel Network

no code implementations25 Nov 2017 Shuai Zhang, Jian-Xin Li, Pengtao Xie, Yingchun Zhang, Minglai Shao, Haoyi Zhou, Mengyi Yan

Similar to DNNs, a SKN is composed of multiple layers of hidden units, but each parameterized by a RKHS function rather than a finite-dimensional vector.

Cannot find the paper you are looking for? You can Submit a new open access paper.