Search Results for author: Zonghan Yang

Found 20 papers, 9 papers with code

Interpolation between CNNs and ResNets

no code implementations ICML 2020 Zonghan Yang, Yang Liu, Chenglong Bao, Zuoqiang Shi

Although ordinary differential equations (ODEs) provide insights for designing networks architectures, its relationship with the non-residual convolutional neural networks (CNNs) is still unclear.

Adversarial Attack Image Classification

PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs

no code implementations20 Feb 2024 An Liu, Zonghan Yang, Zhenhe Zhang, Qingyuan Hu, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu

While Large language models (LLMs) have demonstrated considerable capabilities across various natural language tasks, they often fall short of the performance achieved by domain-specific state-of-the-art models.

text-classification Text Classification

Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models

1 code implementation19 Feb 2024 Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, Yang Liu

State-of-the-art Large Multi-Modal Models (LMMs) have demonstrated exceptional capabilities in vision-language tasks.

Visual Prompting

OneBit: Towards Extremely Low-bit Large Language Models

no code implementations17 Feb 2024 Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, Wanxiang Che

Model quantification uses low bit-width values to represent the weight matrices of models, which is a promising approach to reduce both storage and computational overheads of deploying highly anticipated LLMs.

Quantization

Towards Unified Alignment Between Agents, Humans, and Environment

no code implementations12 Feb 2024 Zonghan Yang, An Liu, Zijun Liu, Kaiming Liu, Fangzhou Xiong, Yile Wang, Zeyuan Yang, Qingyuan Hu, Xinrui Chen, Zhenhe Zhang, Fuwen Luo, Zhicheng Guo, Peng Li, Yang Liu

We also conduct proof-of-concept studies by introducing realistic features to WebShop, including user profiles to demonstrate intentions, personalized reranking for complex environmental dynamics, and runtime cost statistics to reflect self-constraints.

Decision Making

Adversarial Robust Memory-Based Continual Learner

no code implementations29 Nov 2023 Xiaoyue Mi, Fan Tang, Zonghan Yang, Danding Wang, Juan Cao, Peng Li, Yang Liu

Despite the remarkable advances that have been made in continual learning, the adversarial vulnerability of such methods has not been fully discussed.

Adversarial Robustness Continual Learning

Bridging the Gap between Decision and Logits in Decision-based Knowledge Distillation for Pre-trained Language Models

1 code implementation15 Jun 2023 Qinhong Zhou, Zonghan Yang, Peng Li, Yang Liu

By combining the theoretical and empirical estimations of the decision distributions together, the estimation of logits can be successfully reduced to a simple root-finding problem.

Data Augmentation Knowledge Distillation +2

Exploring the Impact of Model Scaling on Parameter-Efficient Tuning

1 code implementation4 Jun 2023 Yusheng Su, Chi-Min Chan, Jiali Cheng, Yujia Qin, Yankai Lin, Shengding Hu, Zonghan Yang, Ning Ding, Xingzhi Sun, Guotong Xie, Zhiyuan Liu, Maosong Sun

Our investigations reveal that model scaling (1) mitigates the effects of the positions of tunable parameters on performance, and (2) enables tuning methods to achieve performance comparable to full-parameter fine-tuning by optimizing fewer tunable parameters.

A Closer Look at the Adversarial Robustness of Deep Equilibrium Models

1 code implementation2 Jun 2023 Zonghan Yang, Tianyu Pang, Yang Liu

Deep equilibrium models (DEQs) refrain from the traditional layer-stacking paradigm and turn to find the fixed point of a single layer.

Adversarial Defense Adversarial Robustness

Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics

1 code implementation2 Jun 2023 Zonghan Yang, Peng Li, Tianyu Pang, Yang Liu

To this end, we interpret DEQs through the lens of neural dynamics and find that AT under-regulates intermediate states.

Adversarial Robustness

Restricted Orthogonal Gradient Projection for Continual Learning

no code implementations28 Jan 2023 Zeyuan Yang, Zonghan Yang, Peng Li, Yang Liu

The basic idea is to adopt a restricted orthogonal constraint allowing parameters optimized in the direction oblique to the whole frozen space to facilitate forward knowledge transfer while consolidating previous knowledge.

Continual Learning Transfer Learning

On Robust Prefix-Tuning for Text Classification

1 code implementation ICLR 2022 Zonghan Yang, Yang Liu

Recently, prefix-tuning has gained increasing attention as a parameter-efficient finetuning method for large-scale pretrained language models.

Language Modelling text-classification +1

Layer-wise Adversarial Defense: An ODE Perspective

no code implementations1 Jan 2021 Zonghan Yang, Yang Liu, Chenglong Bao, Zuoqiang Shi

Deep neural networks are observed to be fragile against adversarial attacks, which have dramatically limited their practical applicability.

Adversarial Defense

Neural Machine Translation: A Review of Methods, Resources, and Tools

no code implementations31 Dec 2020 Zhixing Tan, Shuo Wang, Zonghan Yang, Gang Chen, Xuancheng Huang, Maosong Sun, Yang Liu

Machine translation (MT) is an important sub-field of natural language processing that aims to translate natural languages using computers.

Data Augmentation Machine Translation +2

Interpolation between Residual and Non-Residual Networks

1 code implementation10 Jun 2020 Zonghan Yang, Yang Liu, Chenglong Bao, Zuoqiang Shi

Although ordinary differential equations (ODEs) provide insights for designing network architectures, its relationship with the non-residual convolutional neural networks (CNNs) is still unclear.

Adversarial Attack Image Classification

Chinese Poetry Generation with a Working Memory Model

1 code implementation12 Sep 2018 Xiaoyuan Yi, Maosong Sun, Ruoyu Li, Zonghan Yang

Different from previous methods, our model explicitly maintains topics and informative limited history in a neural memory.

Cultural Vocal Bursts Intensity Prediction

Cannot find the paper you are looking for? You can Submit a new open access paper.