Search Results for author: Zhiyuan Zhang

Found 57 papers, 25 papers with code

SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing

no code implementations15 Oct 2024 Zhiyuan Zhang, Dongdong Chen, Jing Liao

Scene graphs offer a structured, hierarchical representation of images, with nodes and edges symbolizing objects and the relationships among them.

Language Modelling Large Language Model +1

RISurConv: Rotation Invariant Surface Attention-Augmented Convolutions for 3D Point Cloud Classification and Segmentation

1 code implementation12 Aug 2024 Zhiyuan Zhang, Licheng Yang, Zhiyu Xiang

We achieve an overall accuracy of 96. 0% (+4. 7%) on ModelNet40, 93. 1% (+12. 8%) on ScanObjectNN, and class accuracies of 91. 5% (+3. 6%), 82. 7% (+5. 1%), and 78. 5% (+9. 2%) on the three categories of the FG3D dataset for the fine-grained classification task.

3D Point Cloud Classification Point Cloud Classification

Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving

1 code implementation6 Jun 2024 Xiaosong Jia, Zhenjie Yang, QiFeng Li, Zhiyuan Zhang, Junchi Yan

In an era marked by the rapid scaling of foundation models, autonomous driving technologies are approaching a transformative threshold where end-to-end autonomous driving (E2E-AD) emerges due to its potential of scaling up in the data-driven manner.

Autonomous Driving Benchmarking

Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion

1 code implementation18 May 2024 Zeyu Zhang, Yiran Wang, Biao Wu, Shuo Chen, Zhiyuan Zhang, Shiya Huang, Wenbo Zhang, Meng Fang, Ling Chen, Yang Zhao

Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries.

Motion Generation

Closed-Loop Open-Vocabulary Mobile Manipulation with GPT-4V

no code implementations16 Apr 2024 Peiyuan Zhi, Zhiyuan Zhang, Muzhi Han, Zeyu Zhang, Zhitian Li, Ziyuan Jiao, Baoxiong Jia, Siyuan Huang

Autonomous robot navigation and manipulation in open environments require reasoning and replanning with closed-loop feedback.

Instruction Following Multimodal Reasoning +1

Scaling Up Dynamic Human-Scene Interaction Modeling

no code implementations CVPR 2024 Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Siyuan Huang

Confronting the challenges of data scarcity and advanced motion synthesis in human-scene interaction modeling, we introduce the TRUMANS dataset alongside a novel HSI motion synthesis method.

Motion Synthesis

MetaScript: Few-Shot Handwritten Chinese Content Generation via Generative Adversarial Networks

1 code implementation25 Dec 2023 Xiangyuan Xue, Kailing Wang, Jiazi Bu, Qirui Li, Zhiyuan Zhang

In this work, we propose MetaScript, a novel Chinese content generation system designed to address the diminishing presence of personal handwriting styles in the digital representation of Chinese characters.

Few-Shot Learning

Test-Time Augmentation for 3D Point Cloud Classification and Segmentation

no code implementations22 Nov 2023 Tuan-Anh Vu, Srinjay Sarkar, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

We are inspired by the recent revolution of learning implicit representation and point cloud upsampling, which can produce high-quality 3D surface reconstruction and proximity-to-surface, respectively.

3D Point Cloud Classification Data Augmentation +3

GelFlow: Self-supervised Learning of Optical Flow for Vision-Based Tactile Sensor Displacement Measurement

no code implementations13 Sep 2023 Zhiyuan Zhang, Hua Yang, Zhouping Yin

However, these methods need to be more precise for accurately measuring the displacement of markers during large elastic deformation of the gel, as this can significantly impact the accuracy of downstream tasks.

Optical Flow Estimation Self-Supervised Learning

Incorporating Pre-trained Model Prompting in Multimodal Stock Volume Movement Prediction

1 code implementation11 Sep 2023 Ruibo Chen, Zhiyuan Zhang, Yi Liu, Ruihan Bao, Keiko Harimoto, Xu sun

Existing multimodal works that train models from scratch face the problem of lacking universal knowledge when modeling financial news.

Time Series

Continuous Layout Editing of Single Images with Diffusion Models

no code implementations22 Jun 2023 Zhiyuan Zhang, Zhitong Huang, Jing Liao

However, none of these methods have been able to edit the layout of single existing images.

TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest

2 code implementations31 May 2023 Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, Andrew Zhai

This paper (1) presents Pinterest's ranking architecture for Homefeed, our personalized recommendation product and the largest engagement surface; (2) proposes TransAct, a sequential model that extracts users' short-term preferences from their realtime activities; (3) describes our hybrid approach to ranking, which combines end-to-end sequential modeling via TransAct with batch-generated user embeddings.

Sequential Recommendation

Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias

no code implementations8 May 2023 Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie zhou, Xu sun

To settle this issue, we propose the Fine-purifying approach, which utilizes the diffusion theory to study the dynamic process of fine-tuning for finding potentially poisonous dimensions.

Full-Body Articulated Human-Object Interaction

1 code implementation ICCV 2023 Nan Jiang, Tengyu Liu, Zhexuan Cao, Jieming Cui, Zhiyuan Zhang, Yixin Chen, He Wang, Yixin Zhu, Siyuan Huang

By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation to tackle the estimation of articulated object poses and shapes during whole-body interactions.

Action Recognition Human-Object Interaction Detection +3

Learning a Task-specific Descriptor for Robust Matching of 3D Point Clouds

no code implementations26 Oct 2022 Zhiyuan Zhang, Yuchao Dai, Bin Fan, Jiadai Sun, Mingyi He

In this paper, we propose to learn a robust task-specific feature descriptor to consistently describe the correct point correspondence under interference.

Searching Dense Point Correspondences via Permutation Matrix Learning

no code implementations26 Oct 2022 Zhiyuan Zhang, Jiadai Sun, Yuchao Dai, Bin Fan, Qi Liu

In response, this paper presents a novel end-to-end learning-based method to estimate the dense correspondence of 3D point clouds, in which the problem of point matching is formulated as a zero-one assignment problem to achieve a permutation matching matrix to implement the one-to-one principle fundamentally.

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

1 code implementation18 Oct 2022 Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, Xu sun

In this work, we take the first step to exploit the pre-trained (unfine-tuned) weights to mitigate backdoors in fine-tuned language models.

Language Modelling Sentence +4

Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks

1 code implementation14 Oct 2022 Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, Xu sun

In this work, we take the first step to investigate the unconcealment of textual poisoned samples at the intermediate-feature level and propose a feature-based efficient online defense method.

backdoor defense Sentiment Analysis

Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation

no code implementations13 Oct 2022 Zhiyuan Zhang, Qi Su, Xu sun

NLP attacks tend to have small relative backdoor strengths, which may result in the failure of robust federated aggregation methods for NLP attacks.

Federated Learning

Stock Trading Volume Prediction with Dual-Process Meta-Learning

1 code implementation11 Oct 2022 Ruibo Chen, Wei Li, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu sun

Our method can model the common pattern behind different stocks with a meta-learner, while modeling the specific pattern for each stock across time spans with stock-dependent parameters.

Algorithmic Trading Meta-Learning

Distributional Correlation--Aware Knowledge Distillation for Stock Trading Volume Prediction

1 code implementation4 Aug 2022 Lei LI, Zhiyuan Zhang, Ruihan Bao, Keiko Harimoto, Xu sun

Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced by teacher models, which are not available in regression problems like stock trading volume prediction.

Knowledge Distillation regression

Context-Aware Video Reconstruction for Rolling Shutter Cameras

1 code implementation CVPR 2022 Bin Fan, Yuchao Dai, Zhiyuan Zhang, Qi Liu, Mingyi He

Then, a refinement scheme is proposed to guide the GS frame synthesis along with bilateral occlusion masks to produce high-fidelity GS video frames at arbitrary times.

Motion Compensation Video Reconstruction

VRNet: Learning the Rectified Virtual Corresponding Points for 3D Point Cloud Registration

no code implementations24 Mar 2022 Zhiyuan Zhang, Jiadai Sun, Yuchao Dai, Bin Fan, Mingyi He

3D point cloud registration is fragile to outliers, which are labeled as the points without corresponding points.

Point Cloud Registration

A Representation Separation Perspective to Correspondences-free Unsupervised 3D Point Cloud Registration

no code implementations24 Mar 2022 Zhiyuan Zhang, Jiadai Sun, Yuchao Dai, Dingfu Zhou, Xibin Song, Mingyi He

Existing correspondences-free methods generally learn the holistic representation of the entire point cloud, which is fragile for partial and noisy point clouds.

Point Cloud Registration

CVFNet: Real-time 3D Object Detection by Learning Cross View Features

no code implementations13 Mar 2022 Jiaqi Gu, Zhiyu Xiang, Pan Zhao, Tingming Bai, Lingxuan Wang, Xijun Zhao, Zhiyuan Zhang

In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies.

3D geometry 3D Object Detection +2

RIConv++: Effective Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

1 code implementation26 Feb 2022 Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

3D point clouds deep learning is a promising field of research that allows a neural network to learn features of point clouds directly, making it a robust tool for solving 3D scene understanding tasks.

3D Point Cloud Classification Point Cloud Segmentation +2

End-to-end Learning the Partial Permutation Matrix for Robust 3D Point Cloud Registration

no code implementations28 Oct 2021 Zhiyuan Zhang, Jiadai Sun, Yuchao Dai, Dingfu Zhou, Xibin Song, Mingyi He

Even though considerable progress has been made in deep learning-based 3D point cloud processing, how to obtain accurate correspondences for robust registration remains a major challenge because existing hard assignment methods cannot deal with outliers naturally.

Point Cloud Registration

Adversarial Parameter Defense by Multi-Step Risk Minimization

no code implementations7 Sep 2021 Zhiyuan Zhang, Ruixuan Luo, Xuancheng Ren, Qi Su, Liangyou Li, Xu sun

To enhance neural networks, we propose the adversarial parameter defense algorithm that minimizes the average risk of multiple adversarial parameter corruptions.

How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

no code implementations ICLR 2022 Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang, Lichao Sun, Xu sun

In this work, we observe an interesting phenomenon that the variations of parameters are always AWPs when tuning the trained clean model to inject backdoors.

ASAT: Adaptively Scaled Adversarial Training in Time Series

no code implementations20 Aug 2021 Zhiyuan Zhang, Wei Li, Ruihan Bao, Keiko Harimoto, Yunfang Wu, Xu sun

Besides the security concerns of potential adversarial examples, adversarial training can also improve the generalization ability of neural networks, train robust neural networks, and provide interpretability for neural networks.

Adversarial Robustness Time Series +1

Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects

no code implementations NAACL 2021 Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu sun, Bin He

Motivated by neuroscientific evidence and theoretical results, we demonstrate that side effects can be controlled by the number of changed parameters and thus, we propose to conduct \textit{neural network surgery} by only modifying a limited number of parameters.

Rethinking Skip Connection with Layer Normalization in Transformers and ResNets

no code implementations15 May 2021 Fenglin Liu, Xuancheng Ren, Zhiyuan Zhang, Xu sun, Yuexian Zou

In this work, we investigate how the scale factors in the effectiveness of the skip connection and reveal that a trivial adjustment of the scale will lead to spurious gradient exploding or vanishing in line with the deepness of the models, which could be addressed by normalization, in particular, layer normalization, which induces consistent improvements over the plain skip connection.

Image Classification Machine Translation +1

Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models

1 code implementation NAACL 2021 Wenkai Yang, Lei LI, Zhiyuan Zhang, Xuancheng Ren, Xu sun, Bin He

However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one single word embedding vector, with almost no accuracy sacrificed on clean samples.

Backdoor Attack Data Poisoning +4

Spatial Structure Engineering in Enhancing Performance of Mosaic Electrocatalysts

no code implementations19 Feb 2021 Yuting Luo, Sum Wai Chiang, Lei Tang, Zhiyuan Zhang, Fengning Yang, Qiangmin Yu, Baofu Ding, Bilu Liu

Understanding the mechanism and developing strategies toward efficient electrocatalysis at gas-liquidsolid interfaces are important yet challenging.

Applied Physics Materials Science

Rethinking Skip Connection with Layer Normalization

no code implementations COLING 2020 Fenglin Liu, Xuancheng Ren, Zhiyuan Zhang, Xu sun, Yuexian Zou

In this work, we investigate how the scale factors in the effectiveness of the skip connection and reveal that a trivial adjustment of the scale will lead to spurious gradient exploding or vanishing in line with the deepness of the models, which could by addressed by normalization, in particular, layer normalization, which induces consistent improvements over the plain skip connection.

Image Classification Machine Translation +1

RRCN: A Reinforced Random Convolutional Network based Reciprocal Recommendation Approach for Online Dating

no code implementations25 Nov 2020 Linhao Luo, Liqi Yang, Ju Xin, Yixiang Fang, Xiaofeng Zhang, Xiaofei Yang, Kai Chen, Zhiyuan Zhang, Kai Liu

In particular, we technically propose a novel random CNN component that can randomly convolute non-adjacent features to capture their interaction information and learn feature embeddings of key attributes to make the final recommendation.

Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models

no code implementations Findings of the Association for Computational Linguistics 2020 Zhiyuan Zhang, Xiaoqian Liu, Yi Zhang, Qi Su, Xu sun, Bin He

Conventional knowledge graph embedding (KGE) often suffers from limited knowledge representation, leading to performance degradation especially on the low-resource problem.

Knowledge Graph Embedding World Knowledge

Global Context Aware Convolutions for 3D Point Cloud Understanding

no code implementations7 Aug 2020 Zhiyuan Zhang, Binh-Son Hua, Wei Chen, Yibin Tian, Sai-Kit Yeung

We found that a key reason is that compared to point coordinates, rotation-invariant features consumed by point cloud convolution are not as distinctive.

Point Cloud Classification Retrieval +1

Exploring the Vulnerability of Deep Neural Networks: A Study of Parameter Corruption

1 code implementation10 Jun 2020 Xu Sun, Zhiyuan Zhang, Xuancheng Ren, Ruixuan Luo, Liangyou Li

We argue that the vulnerability of model parameters is of crucial value to the study of model robustness and generalization but little research has been devoted to understanding this matter.

Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?

no code implementations10 Jun 2020 Lifu Wang, Bo Shen, Ning Zhao, Zhiyuan Zhang

In this paper, we follow this line to study the topology (sub-level sets) of the loss landscape of deep ReLU neural networks with a skip connection and theoretically prove that the skip connection network inherits the good properties of the two-layer network and skip connections can help to control the connectedness of the sub-level sets, such that any local minima worse than the global minima of some two-layer ReLU network will be very ``shallow".

Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection

2 code implementations25 Dec 2019 Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Qi Su, Xu sun

Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks.

Image Captioning Language Modelling +2

MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning

2 code implementations17 Nov 2019 Guangxiang Zhao, Xu sun, Jingjing Xu, Zhiyuan Zhang, Liangchen Luo

In this work, we explore parallel multi-scale representation learning on sequence data, striving to capture both long-range and short-range language structures.

Machine Translation Representation Learning +1

Understanding and Improving Layer Normalization

2 code implementations NeurIPS 2019 Jingjing Xu, Xu sun, Zhiyuan Zhang, Guangxiang Zhao, Junyang Lin

Unlike them, we find that the derivatives of the mean and variance are more important than forward normalization by re-centering and re-scaling backward gradients.

Machine Translation Translation

Sparse Transformer: Concentrated Attention Through Explicit Selection

no code implementations25 Sep 2019 Guangxiang Zhao, Junyang Lin, Zhiyuan Zhang, Xuancheng Ren, Xu sun

Extensive experimental results on a series of natural language processing tasks, including neural machine translation, image captioning, and language modeling, all demonstrate the advantages of Sparse Transformer in model performance.

Image Captioning Language Modelling +2

ShellNet: Efficient Point Cloud Convolutional Neural Networks using Concentric Shells Statistics

1 code implementation ICCV 2019 Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data.

3D Point Cloud Classification 3D Semantic Segmentation +2

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

1 code implementation17 Aug 2019 Zhiyuan Zhang, Binh-Son Hua, David W. Rosen, Sai-Kit Yeung

Our core idea is to use low-level rotation invariant geometric features such as distances and angles to design a convolution operator for point cloud learning.

Deep Learning Scene Understanding +1

PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation

4 code implementations27 Jun 2019 Ruixuan Luo, Jingjing Xu, Yi Zhang, Zhiyuan Zhang, Xuancheng Ren, Xu sun

Through this method, we generate synthetic data using a large amount of unlabeled data in the target domain and then obtain a word segmentation model for the target domain.

Chinese Word Segmentation Domain Adaptation +3

Memorized Sparse Backpropagation

no code implementations24 May 2019 Zhiyuan Zhang, Pengcheng Yang, Xuancheng Ren, Qi Su, Xu sun

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers.

Primal Meaning Recommendation via On-line Encyclopedia

no code implementations14 Aug 2018 Zhiyuan Zhang, Wei Li, Jingjing Xu, Xu sun

We define the primal meaning of an expression to be a frequently used sense of that expression from which its other frequent senses can be deduced.

Automatic Translating between Ancient Chinese and Contemporary Chinese with Limited Aligned Corpora

no code implementations5 Mar 2018 Zhiyuan Zhang, Wei Li, Qi Su

In this paper, we propose to build an end-to-end neural model to automatically translate between ancient and contemporary Chinese.

Sentence Translation

Building an Ellipsis-aware Chinese Dependency Treebank for Web Text

1 code implementation LREC 2018 Xuancheng Ren, Xu sun, Ji Wen, Bingzhen Wei, Weidong Zhan, Zhiyuan Zhang

Web 2. 0 has brought with it numerous user-produced data revealing one's thoughts, experiences, and knowledge, which are a great source for many tasks, such as information extraction, and knowledge base construction.

Dependency Parsing Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.