Search Results for author: Xinyu Yang

Found 68 papers, 29 papers with code

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity

no code implementations5 Jun 2024 Wentao Guo, Jikai Long, Yimeng Zeng, Zirui Liu, Xinyu Yang, Yide Ran, Jacob R. Gardner, Osbert Bastani, Christopher De Sa, Xiaodong Yu, Beidi Chen, Zhaozhuo Xu

Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes.


FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research

1 code implementation22 May 2024 Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, Zhicheng Dou

With the advent of Large Language Models (LLMs), the potential of Retrieval Augmented Generation (RAG) techniques have garnered considerable research attention.


Quantifying Emergence in Large Language Models

1 code implementation21 May 2024 Hang Chen, Xinyu Yang, Jiaying Zhu, Wenya Wang

Empirical results show that (1) our method gives consistent measurements which align with existing observations based on performance metrics, validating the effectiveness of our emergence quantification; (2) our proposed metric uncovers novel emergence patterns such as the correlations between the variance of our metric and the number of ``shots'' in ICL, which further suggests a new way of interpreting hallucinations in LLMs; (3) we offer a potential solution towards estimating the emergence of larger and closed-resource LMs via smaller LMs like GPT-2.

In-Context Learning

Bridging the Gap: Protocol Towards Fair and Consistent Affect Analysis

1 code implementation10 May 2024 Guanyu Hu, Eleni Papadopoulou, Dimitrios Kollias, Paraskevi Tzouveli, Jie Wei, Xinyu Yang

Automatic affect analysis, at the intersection of physiology, psychology, and machine learning, has seen significant development.

Decision Making Fairness

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

1 code implementation7 May 2024 DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, JianZhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Size Zheng, T. Wang, Tian Pei, Tian Yuan, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Liu, Xin Xie, Xingkai Yu, Xinnan Song, Xinyi Zhou, Xinyu Yang, Xuan Lu, Xuecheng Su, Y. Wu, Y. K. Li, Y. X. Wei, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Zheng, Yichao Zhang, Yiliang Xiong, Yilong Zhao, Ying He, Ying Tang, Yishi Piao, Yixin Dong, Yixuan Tan, Yiyuan Liu, Yongji Wang, Yongqiang Guo, Yuchen Zhu, Yuduan Wang, Yuheng Zou, Yukun Zha, Yunxian Ma, Yuting Yan, Yuxiang You, Yuxuan Liu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhewen Hao, Zhihong Shao, Zhiniu Wen, Zhipeng Xu, Zhongyu Zhang, Zhuoshu Li, Zihan Wang, Zihui Gu, Zilin Li, Ziwei Xie

MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.

Language Modelling Reinforcement Learning (RL)

Separate in the Speech Chain: Cross-Modal Conditional Audio-Visual Target Speech Extraction

no code implementations19 Apr 2024 Zhaoxi Mu, Xinyu Yang

In audio-visual target speech extraction tasks, the audio modality tends to dominate, potentially overshadowing the importance of visual guidance.

Speech Extraction

TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

1 code implementation18 Apr 2024 Hanshi Sun, Zhuoming Chen, Xinyu Yang, Yuandong Tian, Beidi Chen

However, key-value (KV) cache, which is stored to avoid re-computation, has emerged as a critical bottleneck by growing linearly in size with the sequence length.

What's documented in AI? Systematic Analysis of 32K AI Model Cards

1 code implementation7 Feb 2024 Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, James Zou

To evaluate the impact of model cards, we conducted an intervention study by adding detailed model cards to 42 popular models which had no or sparse model cards previously.


Research about the Ability of LLM in the Tamper-Detection Area

no code implementations24 Jan 2024 Xinyu Yang, Jizhe Zhou

In recent years, particularly since the early 2020s, Large Language Models (LLMs) have emerged as the most powerful AI tools in addressing a diverse range of challenges, from natural language processing to complex problem-solving in various domains.

Navigating Dataset Documentations in AI: A Large-Scale Analysis of Dataset Cards on Hugging Face

1 code implementation24 Jan 2024 Xinyu Yang, Weixin Liang, James Zou

By analyzing all 7, 433 dataset documentation on Hugging Face, our investigation provides an overview of the Hugging Face dataset ecosystem and insights into dataset documentation practices, yielding 5 main findings: (1) The dataset card completion rate shows marked heterogeneity correlated with dataset popularity.

Towards Causal Relationship in Indefinite Data: Baseline Model and New Datasets

1 code implementation16 Jan 2024 Hang Chen, Xinyu Yang, Keqing Du

These highpoints make the probabilistic model capable of overcoming challenges brought by the coexistence of multi-structure data and multi-value representations and pave the way for the extension of latent confounders.

Causal Discovery Disentanglement

High-Fidelity Face Swapping with Style Blending

no code implementations17 Dec 2023 Xinyu Yang, Hongbo Bo

Face swapping has gained significant traction, driven by the plethora of human face synthesis facilitated by deep learning methods.

Decoder Face Generation +1

Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction

no code implementations16 Dec 2023 Zhaoxi Mu, Xinyu Yang, Sining Sun, Qing Yang

However, in the task of target speech extraction, certain elements of global and local semantic information in the reference speech, which are irrelevant to speaker identity, can lead to speaker confusion within the speech extraction network.

Disentanglement Speech Extraction

MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion

1 code implementation16 Dec 2023 Shulei Ji, Xinyu Yang

However, prior research on deep learning-based emotional music generation has rarely explored the contribution of different musical elements to emotions, let alone the deliberate manipulation of these elements to alter the emotion of music, which is not conducive to fine-grained element-level control over emotions.

Music Generation

CASR: Refining Action Segmentation via Marginalizing Frame-levle Causal Relationships

no code implementations21 Nov 2023 Keqing Du, Xinyu Yang, Hang Chen

CASR works out by reducing the difference in the causal adjacency matrix between we constructed and pre-segmentation results of backbone models.

Action Segmentation Causal Discovery +1

Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges

1 code implementation6 Nov 2023 Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, Huaxiu Yao

To bridge this gap, we introduce a new benchmark, namely, the Bias and Interference Challenges in Visual Language Models (Bingo).


A Review and Roadmap of Deep Causal Model from Different Causal Structures and Representations

no code implementations2 Nov 2023 Hang Chen, Keqing Du, Chenguang Li, Xinyu Yang

The fusion of causal models with deep learning introducing increasingly intricate data sets, such as the causal associations within images or between textual components, has surfaced as a focal research area.

Time Series

SSL Framework for Causal Inconsistency between Structures and Representations

no code implementations28 Oct 2023 Hang Chen, Xinyu Yang, Keqing Du

The cross-pollination of deep learning and causal discovery has catalyzed a burgeoning field of research seeking to elucidate causal relationships within non-statistical data forms like images, videos, and text.

Causal Discovery Philosophy +1

Can large language models provide useful feedback on research papers? A large-scale empirical analysis

1 code implementation3 Oct 2023 Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, Daniel McFarland, James Zou

We first quantitatively compared GPT-4's generated feedback with human peer reviewer feedback in 15 Nature family journals (3, 096 papers in total) and the ICLR machine learning conference (1, 709 papers).

MemDA: Forecasting Urban Time Series with Memory-based Drift Adaptation

1 code implementation25 Sep 2023 Zekun Cai, Renhe Jiang, Xinyu Yang, Zhaonan Wang, Diansheng Guo, Hiroki Kobayashi, Xuan Song, Ryosuke Shibasaki

Urban time series data forecasting featuring significant contributions to sustainable development is widely studied as an essential task of the smart city.

Multivariate Time Series Forecasting Time Series +2

Emotion-Conditioned Melody Harmonization with Hierarchical Variational Autoencoder

no code implementations6 Jun 2023 Shulei Ji, Xinyu Yang

To solve these problems, we propose a novel LSTM-based Hierarchical Variational Auto-Encoder (LHVAE) to investigate the influence of emotional conditions on melody harmonization, while improving the quality of generated harmonies and capturing the abundant variability of chord progressions.

Learning a Structural Causal Model for Intuition Reasoning in Conversation

1 code implementation28 May 2023 Hang Chen, Bingyu Liao, Jing Luo, Wenjing Zhu, Xinyu Yang

Reasoning, a crucial aspect of NLP research, has not been adequately addressed by prevailing models including Large Language Model.

Causal Discovery Language Modelling +2

Towards Causal Representation Learning and Deconfounding from Indefinite Data

no code implementations4 May 2023 Hang Chen, Xinyu Yang, Qing Yang

We implement the above designs as a dynamic variational inference model, tailored to learn causal representation from indefinite data under latent confounding.

Causal Discovery Disentanglement +1

Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulations

1 code implementation4 May 2023 Weixin Liang, Yining Mao, Yongchan Kwon, Xinyu Yang, James Zou

Our work highlights the importance of understanding the nonlinear effects of model improvement on performance in different subpopulations, and has the potential to inform the development of more equitable and responsible machine learning models.


Improving the Transferability of Adversarial Examples via Direction Tuning

2 code implementations27 Mar 2023 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

Although considerable efforts have been developed on improving the transferability of adversarial examples generated by transfer-based adversarial attacks, our investigation found that, the big deviation between the actual and steepest update directions of the current transfer-based adversarial attacks is caused by the large update step length, resulting in the generated adversarial examples can not converge well.

Network Pruning

Fuzziness-tuned: Improving the Transferability of Adversarial Examples

no code implementations17 Mar 2023 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

In this paper, we first systematically investigated this issue and found that the enormous difference of attack success rates between the surrogate model and victim model is caused by the existence of a special area (known as fuzzy domain in our paper), in which the adversarial examples in the area are classified wrongly by the surrogate model while correctly by the victim model.

A Multi-Stage Triple-Path Method for Speech Separation in Noisy and Reverberant Environments

no code implementations7 Mar 2023 Zhaoxi Mu, Xinyu Yang, Xiangyuan Yang, Wenjing Zhu

In noisy and reverberant environments, the performance of deep learning-based speech separation methods drops dramatically because previous methods are not designed and optimized for such situations.

Denoising Speech Denoising +1

Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning

no code implementations7 Mar 2023 Zhaoxi Mu, Xinyu Yang, Wenjing Zhu

Specifically, we design a new network SE-Conformer that can model audio sequences in multiple dimensions and scales, and apply it to the dual-path speech separation framework.

Speech Separation

Improving Domain Generalization with Domain Relations

no code implementations6 Feb 2023 Huaxiu Yao, Xinyu Yang, Xinyi Pan, Shengchao Liu, Pang Wei Koh, Chelsea Finn

Distribution shift presents a significant challenge in machine learning, where models often underperform during the test stage when faced with a different distribution than the one they were trained on.

Domain Generalization

Zebra: Deeply Integrating System-Level Provenance Search and Tracking for Efficient Attack Investigation

no code implementations10 Nov 2022 Xinyu Yang, Haoyuan Liu, Ziyu Wang, Peng Gao

System auditing has emerged as a key approach for monitoring system call events and investigating sophisticated attacks.

Aircraft Ground Taxiing Deduction and Conflict Early Warning Method Based on Control Command Information

no code implementations4 Nov 2022 Jingchang Zhuge, Huiyuan Liang, Yiming Zhang, Shichao Li, Xinyu Yang, Jun Wu

Aircraft taxiing conflict is a threat to the safety of airport operations, mainly due to the human error in control command infor-mation.

Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations

1 code implementation25 Oct 2022 Xinyu Yang, Huaxiu Yao, Allan Zhou, Chelsea Finn

We study this multi-domain long-tailed learning problem and aim to produce a model that generalizes well across all classes and domains.

Data Augmentation Disentanglement +1

A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms

no code implementations14 Sep 2022 Hang Chen, Keqing Du, Xinyu Yang, Chenguang Li

Understanding causality helps to structure interventions to achieve specific goals and enables predictions under interventions.

Causal Discovery

Learning a General Clause-to-Clause Relationships for Enhancing Emotion-Cause Pair Extraction

no code implementations29 Aug 2022 Hang Chen, Xinyu Yang, Xiang Li

To learn it applicably, we propose a general clause-level encoding model named EA-GAT comprising E-GAT and Activation Sort.

Emotion-Cause Pair Extraction

Video-TransUNet: Temporally Blended Vision Transformer for CT VFSS Instance Segmentation

2 code implementations17 Aug 2022 Chengxi Zeng, Xinyu Yang, Majid Mirmehdi, Alberto M Gambaruto, Tilo Burghardt

Our findings suggest that the proposed model can indeed enhance the TransUNet architecture via exploiting temporal information and improving segmentation performance by a significant margin.

Instance Segmentation Segmentation +1

Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour

no code implementations7 Aug 2022 Feixiang Zhou, Xinyu Yang, Fang Chen, Long Chen, Zheheng Jiang, Hui Zhu, Reiko Heckel, Haikuan Wang, Minrui Fei, Huiyu Zhou

Furthermore, we design a novel Interaction-Aware Transformer (IAT) to dynamically learn the graph-level representation of social behaviours and update the node-level representation, guided by our proposed interaction-aware self-attention mechanism.

Representation Learning Self-Supervised Learning

FACM: Intermediate Layer Still Retain Effective Features against Adversarial Examples

no code implementations2 Jun 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

To enhance the robustness of the classifier, in our paper, a \textbf{F}eature \textbf{A}nalysis and \textbf{C}onditional \textbf{M}atching prediction distribution (FACM) model is proposed to utilize the features of intermediate layers to correct the classification.

Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction

no code implementations2 Jun 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

The empirical and theoretical analysis demonstrates that the MDL loss improves the robustness and generalization of the model simultaneously for natural training.

Gradient Aligned Attacks via a Few Queries

no code implementations19 May 2022 Xiangyuan Yang, Jie Lin, HANLIN ZHANG, Xinyu Yang, Peng Zhao

Specifically, we propose a gradient aligned mechanism to ensure that the derivatives of the loss function with respect to the logit vector have the same weight coefficients between the surrogate and victim models.

Dynamic Curriculum Learning for Great Ape Detection in the Wild

1 code implementation30 Apr 2022 Xinyu Yang, Tilo Burghardt, Majid Mirmehdi

We propose a novel end-to-end curriculum learning approach for sparsely labelled animal datasets leveraging large volumes of unlabelled data to improve supervised species detectors.

object-detection Object Detection

Mixed Strategies for Security Games with General Defending Requirements

no code implementations26 Apr 2022 Rufan Bai, Haoxing Lin, Xinyu Yang, Xiaowei Wu, Minming Li, Weijia Jia

In this work, we initiate the study of mixed strategies for the security games in which the targets can have different defending requirements.

Adversarial Attack

ACE: Towards Application-Centric Edge-Cloud Collaborative Intelligence

1 code implementation24 Mar 2022 Luhui Wang, Cong Zhao, Shusen Yang, Xinyu Yang, Julie McCann

Intelligent applications based on machine learning are impacting many parts of our lives.


DTWSSE: Data Augmentation with a Siamese Encoder for Time Series

no code implementations23 Aug 2021 Xinyu Yang, Xinlan Zhang, Zhenguo Zhang, Yahui Zhao, Rongyi Cui

In order to reasonably measure the distance of the time series, DTW, which has been verified to be an effective method forts, is employed as the distance metric.

Data Augmentation Decoder +2

Review of end-to-end speech synthesis technology based on deep learning

no code implementations20 Apr 2021 Zhaoxi Mu, Xinyu Yang, Yizhuo Dong

As an indispensable part of modern human-computer interaction system, speech synthesis technology helps users get the output of intelligent machine more easily and intuitively, thus has attracted more and more attention.

Speech Synthesis

WakaVT: A Sequential Variational Transformer for Waka Generation

no code implementations1 Apr 2021 Yuka Takeishi, Mingxuan Niu, Jing Luo, Zhong Jin, Xinyu Yang

To further explore the creative potential of natural language generation systems in Japanese poetry creation, we propose a novel Waka generation model, WakaVT, which automatically produces Waka poems given user-specified keywords.

Text Generation

A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions

no code implementations13 Nov 2020 Shulei Ji, Jing Luo, Xinyu Yang

This paper attempts to provide an overview of various composition tasks under different music generation levels, covering most of the currently popular music generation tasks using deep learning.

Audio Generation Music Generation

Back to the Future: Cycle Encoding Prediction for Self-supervised Contrastive Video Representation Learning

1 code implementation14 Oct 2020 Xinyu Yang, Majid Mirmehdi, Tilo Burghardt

In this paper we show that learning video feature spaces in which temporal cycles are maximally predictable benefits action classification.

Action Classification Action Recognition +1

Latent Dirichlet Allocation Model Training with Differential Privacy

no code implementations9 Oct 2020 Fangyuan Zhao, Xuebin Ren, Shusen Yang, Qing Han, Peng Zhao, Xinyu Yang

To address the privacy issue in LDA, we systematically investigate the privacy protection of the main-stream LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose several differentially private LDA algorithms for typical training scenarios.

Privacy Preserving

Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction

1 code implementation16 Jun 2020 Haoxing Lin, Rufan Bai, Weijia Jia, Xinyu Yang, Yongjian You

To filter out irrelevant noises and alleviate the error propagation, DSAN dynamically extracts valuable information by applying self-attention over the noisy input and bridges each output directly to the purified inputs via implementing a switch-attention mechanism.

Future prediction

OL4EL: Online Learning for Edge-cloud Collaborative Learning on Heterogeneous Edges with Resource Constraints

no code implementations22 Apr 2020 Qing Han, Shusen Yang, Xuebin Ren, Cong Zhao, Jingqi Zhang, Xinyu Yang

However, heterogeneous and limited computation and communication resources on edge servers (or edges) pose great challenges on distributed ML and formulate a new paradigm of Edge Learning (i. e. edge-cloud collaborative machine learning).

BIG-bench Machine Learning

Reviewing and Improving the Gaussian Mechanism for Differential Privacy

no code implementations27 Nov 2019 Jun Zhao, Teng Wang, Tao Bai, Kwok-Yan Lam, Zhiying Xu, Shuyu Shi, Xuebin Ren, Xinyu Yang, Yang Liu, Han Yu

Although both classical Gaussian mechanisms [1, 2] assume $0 < \epsilon \leq 1$, our review finds that many studies in the literature have used the classical Gaussian mechanisms under values of $\epsilon$ and $\delta$ where the added noise amounts of [1, 2] do not achieve $(\epsilon,\delta)$-DP.

MG-VAE: Deep Chinese Folk Songs Generation with Specific Regional Style

no code implementations29 Sep 2019 Jing Luo, Xinyu Yang, Shulei Ji, Juan Li

In this paper, we propose MG-VAE, a music generative model based on VAE (Variational Auto-Encoder) that is capable of capturing specific music style and generating novel tunes for Chinese folk songs (Min Ge) in a manipulatable way.

Music Generation Multimedia Sound Audio and Speech Processing

Impact of Prior Knowledge and Data Correlation on Privacy Leakage: A Unified Analysis

no code implementations5 Jun 2019 Yanan Li, Xuebin Ren, Shusen Yang, Xinyu Yang

Considering general correlations, a closed-form expression of privacy leakage is derived for continuous data, and a chain rule is presented for discrete data.


Privacy-preserving Crowd-guided AI Decision-making in Ethical Dilemmas

no code implementations4 Jun 2019 Teng Wang, Jun Zhao, Han Yu, Jinyan Liu, Xinyu Yang, Xuebin Ren, Shuyu Shi

To investigate such ethical dilemmas, recent studies have adopted preference aggregation, in which each voter expresses her/his preferences over decisions for the possible ethical dilemma scenarios, and a centralized system aggregates these preferences to obtain the winning decision.

Autonomous Vehicles Decision Making +1

On Privacy Protection of Latent Dirichlet Allocation Model Training

no code implementations4 Jun 2019 Fangyuan Zhao, Xuebin Ren, Shusen Yang, Xinyu Yang

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for discovery of hidden semantic architecture of text datasets, and plays a fundamental role in many machine learning applications.

BIG-bench Machine Learning Privacy Preserving

Cannot find the paper you are looking for? You can Submit a new open access paper.