Search Results for author: Lei Yang

Found 138 papers, 54 papers with code

面向垂直领域的阅读理解数据增强方法(Method for reading comprehension data enhancement in vertical field)

no code implementations CCL 2020 Zhengwei Lv, Lei Yang, Zhizhong Shi, Xiao Liang, Tao Lei, Duoxing Liu

阅读理解问答系统是利用语义理解等自然语言处理技术, 根据输入问题, 对非结构化文档数据进行分析, 生成一个答案, 具有很高的研究和应用价值。在垂直领域应用过程中, 阅读理解问答数据标注成本高且用户问题表达复杂多样, 使得阅读理解问答系统准确率低、鲁棒性差。针对这一问题, 本文提出一种面向垂直领域的阅读理解问答数据的增强方法, 该方法基于真实用户问题, 构造阅读理解训练数据, 一方面降低标注成本, 另一方面增加训练数据多样性, 提升模型的准确率和鲁棒性。本文用汽车领域数据对该方法进行实验验证, 其结果表明该方法对垂直领域阅读理解模型的准确率和鲁棒性均能有效提升。

Reading Comprehension

Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience

no code implementations15 Mar 2024 Xiaohang Yu, Zhengxian Yang, Shi Pan, Yuqi Han, Haoxiang Wang, Jun Zhang, Shi Yan, Borong Lin, Lei Yang, Tao Yu, Lu Fang

We have built a custom mobile multi-camera large-space dense light field capture system, which provides a series of high-quality and sufficiently dense light field images for various scenarios.

3D Reconstruction 3D Scene Reconstruction +1

Neuromorphic Synergy for Video Binarization

1 code implementation20 Feb 2024 ShiJie Lin, Xiang Zhang, Lei Yang, Lei Yu, Bin Zhou, Xiaowei Luo, Wenping Wang, Jia Pan

We also develop an efficient integration method to propagate this binary image to high frame rate binary video.

Binarization Camera Calibration +1

Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off

no code implementations10 Feb 2024 Yuecheng Li, Tong Wang, Chuan Chen, Jian Lou, Bin Chen, Lei Yang, Zibin Zheng

This implies that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space for different privacy settings and continuous training processes.

Federated Learning

Towards Scenario Generalization for Vision-based Roadside 3D Object Detection

1 code implementation29 Jan 2024 Lei Yang, Xinyu Zhang, Jun Li, Li Wang, Chuang Zhang, Li Ju, Zhiwei Li, Yang shen

Our method surpasses all previous methods by a significant margin in new scenes, including +42. 57% for vehicle, +5. 87% for pedestrian, and +14. 89% for cyclist compared to BEVHeight on the DAIR-V2X-I heterologous benchmark.

3D Object Detection Autonomous Vehicles +1

Robustness-Aware 3D Object Detection in Autonomous Driving: A Review and Outlook

no code implementations12 Jan 2024 Ziying Song, Lin Liu, Feiyang Jia, Yadan Luo, Guoxin Zhang, Lei Yang, Li Wang, Caiyan Jia

In the realm of modern autonomous driving, the perception system is indispensable for accurately assessing the state of the surrounding environment, thereby enabling informed prediction and planning.

3D Object Detection Autonomous Driving +2

RoboFusion: Towards Robust Multi-Modal 3D obiect Detection via SAM

no code implementations8 Jan 2024 Ziying Song, Guoxing Zhang, Lin Liu, Lei Yang, Shaoqing Xu, Caiyan Jia, Feiyang Jia, Li Wang

To align SAM or SAM-AD with multi-modal methods, we then introduce AD-FPN for upsampling the image features extracted by SAM.

3D Object Detection Autonomous Driving +2

A Physics-guided Generative AI Toolkit for Geophysical Monitoring

no code implementations6 Jan 2024 Junhuan Yang, Hanchen Wang, Yi Sheng, Youzuo Lin, Lei Yang

Full-waveform inversion (FWI) plays a vital role in geoscience to explore the subsurface.

SSIM

On Optimal Sampling for Learning SDF Using MLPs Equipped with Positional Encoding

no code implementations2 Jan 2024 Guying Lin, Lei Yang, YuAn Liu, Congyi Zhang, Junhui Hou, Xiaogang Jin, Taku Komura, John Keyser, Wenping Wang

Sampling against this intrinsic frequency following the Nyquist-Sannon sampling theorem allows us to determine an appropriate training sampling rate.

Discrete Distribution Networks

no code implementations29 Dec 2023 Lei Yang

This selected output is then fed back into the network as a condition for the second layer, thereby generating new outputs more similar to the GT.

FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing

1 code implementation NeurIPS 2023 Mingyuan Zhang, Huirong Li, Zhongang Cai, Jiawei Ren, Lei Yang, Ziwei Liu

Notably, FineMoGen further enables zero-shot motion editing capabilities with the aid of modern large language models (LLM), which faithfully manipulates motion sequences with fine-grained instructions.

Motion Synthesis

TrojFair: Trojan Fairness Attacks

no code implementations16 Dec 2023 Mengxin Zheng, Jiaqi Xue, Yi Sheng, Lei Yang, Qian Lou, Lei Jiang

TrojFair is a stealthy Fairness attack that is resilient to existing model fairness audition detectors since the model for clean inputs is fair.

Fairness

Learning Dense Correspondence for NeRF-Based Face Reenactment

no code implementations16 Dec 2023 Songlin Yang, Wei Wang, Yushi Lan, Xiangyu Fan, Bo Peng, Lei Yang, Jing Dong

Therefore, we are inspired to ask: Can we learn the dense correspondence between different NeRF-based face representations without a 3D parametric model prior?

Face Reenactment

Let All be Whitened: Multi-teacher Distillation for Efficient Visual Retrieval

1 code implementation15 Dec 2023 Zhe Ma, Jianfeng Dong, Shouling Ji, Zhenguang Liu, Xuhong Zhang, Zonghui Wang, Sifeng He, Feng Qian, Xiaobo Zhang, Lei Yang

Instead of crafting a new method pursuing further improvement on accuracy, in this paper we propose a multi-teacher distillation framework Whiten-MTD, which is able to transfer knowledge from off-the-shelf pre-trained retrieval models to a lightweight student model for efficient visual retrieval.

Image Retrieval Retrieval +1

Towards Robust and Expressive Whole-body Human Pose and Shape Estimation

1 code implementation NeurIPS 2023 Hui EnPang, Zhongang Cai, Lei Yang, Qingyi Tao, Zhonghua Wu, Tianwei Zhang, Ziwei Liu

Whole-body pose and shape estimation aims to jointly predict different behaviors (e. g., pose, hand gesture, facial expression) of the entire human body from a monocular image.

Speed Up Federated Learning in Heterogeneous Environment: A Dynamic Tiering Approach

1 code implementation9 Dec 2023 Seyed Mahmoud Sajjadi Mohammadabadi, Syed Zawad, Feng Yan, Lei Yang

The dynamic tier scheduler assigns clients to suitable tiers to minimize the overall training time in each round.

 Ranked #1 on Image Classification on CIFAR-10 (training time (s) metric)

Federated Learning Image Classification

Digital Life Project: Autonomous 3D Characters with Social Intelligence

no code implementations7 Dec 2023 Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, Xiangyu Fan, Han Du, Liang Pan, Peng Gao, Zhitao Yang, Yang Gao, Jiaqi Li, Tianxiang Ren, Yukun Wei, Xiaogang Wang, Chen Change Loy, Lei Yang, Ziwei Liu

In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.

Motion Captioning Motion Synthesis

AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing

no code implementations3 Dec 2023 Fan Yang, Tianyi Chen, Xiaosheng He, Zhongang Cai, Lei Yang, Si Wu, Guosheng Lin

We propose AttriHuman-3D, an editable 3D human generation model, which address the aforementioned problems with attribute decomposition and indexing.

Attribute Disentanglement

Machine-Learned Atomic Cluster Expansion Potentials for Fast and Quantum-Accurate Thermal Simulations of Wurtzite AlN

no code implementations20 Nov 2023 Guang Yang, Yuan-Bin Liu, Lei Yang, Bing-Yang Cao

Using the atomic cluster expansion (ACE) framework, we develop a machine learning interatomic potential for fast and accurately modelling the phonon transport properties of wurtzite aluminum nitride.

Community-Aware Efficient Graph Contrastive Learning via Personalized Self-Training

no code implementations18 Nov 2023 Yuecheng Li, YanMing Hu, Lele Fu, Chuan Chen, Lei Yang, Zibin Zheng

However, for unsupervised and structure-related tasks such as community detection, current GCL algorithms face difficulties in acquiring the necessary community-level information, resulting in poor performance.

Community Detection Contrastive Learning +1

Story-to-Motion: Synthesizing Infinite and Controllable Character Animation from Long Text

no code implementations13 Nov 2023 Zhongfei Qing, Zhongang Cai, Zhitao Yang, Lei Yang

Generating natural human motion from a story has the potential to transform the landscape of animation, gaming, and film industries.

Motion Synthesis Position

Contrastive Deep Nonnegative Matrix Factorization for Community Detection

1 code implementation4 Nov 2023 Yuecheng Li, Jialong Chen, Chuan Chen, Lei Yang, Zibin Zheng

Recently, nonnegative matrix factorization (NMF) has been widely adopted for community detection, because of its better interpretability.

Community Detection Contrastive Learning +3

GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection

1 code implementation24 Oct 2023 Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Tong He, Yonghui Li, Wanli Ouyang

It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning.

Monocular 3D Object Detection object-detection

Fuzzy-NMS: Improving 3D Object Detection with Fuzzy Classification in NMS

no code implementations21 Oct 2023 Li Wang, Xinyu Zhang, Fachuan Zhao, Chuze Wu, Yichen Wang, Ziying Song, Lei Yang, Jun Li, Huaping Liu

The proposed Fuzzy-NMS module combines the volume and clustering density of candidate bounding boxes, refining them with a fuzzy classification method and optimizing the appropriate suppression thresholds to reduce uncertainty in the NMS process.

3D Object Detection object-detection

FMRT: Learning Accurate Feature Matching with Reconciliatory Transformer

no code implementations20 Oct 2023 Xinyu Zhang, Li Wang, Zhiqiang Jiang, Kun Dai, Tao Xie, Lei Yang, Wenhao Yu, Yang shen, Jun Li

However, these methods only integrate long-range context information among keypoints with a fixed receptive field, which constrains the network from reconciling the importance of features with different receptive fields to realize complete image perception, hence limiting the matching accuracy.

Homography Estimation Pose Estimation +1

Training and Predicting Visual Error for Real-Time Applications

1 code implementation13 Oct 2023 João Libório Cardoso, Bernhard Kerbl, Lei Yang, Yury Uralsky, Michael Wimmer

Specifically, we train and deploy a neural network to estimate the visual error resulting from reusing shading or using reduced shading rates.

GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection

no code implementations ICCV 2023 Ziying Song, Haiyue Wei, Lin Bai, Lei Yang, Caiyan Jia

Through the projection calibration between the image and point cloud, we project the nearest neighbors of point cloud features onto the image features.

3D Object Detection Autonomous Driving +3

Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving

1 code implementation11 Oct 2023 Xinyu Zhang, Li Wang, Jian Chen, Cheng Fang, Lei Yang, Ziying Song, Guangqi Yang, Yichen Wang, Xiaofei Zhang, Jun Li, Zhiwei Li, Qingshan Yang, Zhenlin Zhang, Shuzhi Sam Ge

Compared with commonly used 3D radars, the latest 4D radars have precise vertical resolution and higher point cloud density, making it a highly promising sensor for autonomous driving in complex environmental perception.

3D Object Detection Autonomous Driving +1

Evaluating Explanation Methods for Vision-and-Language Navigation

no code implementations10 Oct 2023 Guanqi Chen, Lei Yang, Guanhua Chen, Jia Pan

The ability to navigate robots with natural language instructions in an unknown environment is a crucial step for achieving embodied artificial intelligence (AI).

Decision Making Navigate +3

MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings

no code implementations30 Sep 2023 Lei Yang, Jiaxin Yu, Xinyu Zhang, Jun Li, Li Wang, Yi Huang, Chuang Zhang, Hong Wang, Yiming Li

We discover that most existing monocular 3D object detectors rely on the ego-vehicle prior assumption that the optical axis of the camera is parallel to the ground.

Autonomous Driving Monocular 3D Object Detection +1

BEVHeight++: Toward Robust Visual Centric 3D Object Detection

no code implementations28 Sep 2023 Lei Yang, Tao Tang, Jun Li, Peng Chen, Kun Yuan, Li Wang, Yi Huang, Xinyu Zhang, Kaicheng Yu

In essence, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods.

3D Object Detection Autonomous Driving +2

Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrieval

no code implementations20 Sep 2023 Chen Jiang, Kaiming Huang, Sifeng He, Xudong Yang, Wei zhang, Xiaobo Zhang, Yuan Cheng, Lei Yang, Qing Wang, Furong Xu, Tan Pan, Wei Chu

SSAN is based on two newly proposed modules in video retrieval: (1) An efficient Self-supervised Keyframe Extraction (SKE) module to reduce redundant frame features, (2) A robust Similarity Pattern Detection (SPD) module for temporal alignment.

Retrieval Video Retrieval

Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization

1 code implementation13 Sep 2023 Zhenguang Liu, Xinyang Yu, Ruili Wang, Shuai Ye, Zhe Ma, Jianfeng Dong, Sifeng He, Feng Qian, Xiaobo Zhang, Roger Zimmermann, Lei Yang

We theoretically analyzed the mutual information between the label and the disentangled features, arriving at a loss that maximizes the extraction of task-relevant information from the original feature.

Disentanglement

ReliTalk: Relightable Talking Portrait Generation from a Single Video

1 code implementation5 Sep 2023 Haonan Qiu, Zhaoxi Chen, Yuming Jiang, Hang Zhou, Xiangyu Fan, Lei Yang, Wayne Wu, Ziwei Liu

Our key insight is to decompose the portrait's reflectance from implicitly learned audio-driven facial normals and images.

Single-Image Portrait Relighting

ImmersiveNeRF: Hybrid Radiance Fields for Unbounded Immersive Light Field Reconstruction

no code implementations4 Sep 2023 Xiaohang Yu, Haoxiang Wang, Yuqi Han, Lei Yang, Tao Yu, Qionghai Dai

This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation.

Segmentation

PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds

no code implementations28 Aug 2023 Zhongang Cai, Liang Pan, Chen Wei, Wanqi Yin, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu

To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.

3D human pose and shape estimation

IT3D: Improved Text-to-3D Generation with Explicit View Synthesis

1 code implementation22 Aug 2023 YiWen Chen, Chi Zhang, Xiaofeng Yang, Zhongang Cai, Gang Yu, Lei Yang, Guosheng Lin

Recent strides in Text-to-3D techniques have been propelled by distilling knowledge from powerful large text-to-image diffusion models (LDMs).

Text to 3D

HumanLiff: Layer-wise 3D Human Generation with Diffusion Model

no code implementations18 Aug 2023 Shoukang Hu, Fangzhou Hong, Tao Hu, Liang Pan, Haiyi Mei, Weiye Xiao, Lei Yang, Ziwei Liu

In this work, we propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process.

Neural Rendering

ZeRO++: Extremely Efficient Collective Communication for Giant Model Training

1 code implementation16 Jun 2023 Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Connor Holmes, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He

Zero Redundancy Optimizer (ZeRO) has been used to train a wide range of large language models on massive GPUs clusters due to its ease of use, efficiency, and good scalability.

Quantization

NeRF2: Neural Radio-Frequency Radiance Fields

no code implementations10 May 2023 Xiaopeng Zhao, Zhenlin An, Qingrui Pan, Lei Yang

Inspired by the great success of using a neural network to describe the optical field in computer vision, we propose a neural radio-frequency radiance field, NeRF$^\textbf{2}$, which represents a continuous volumetric scene function that makes sense of an RF signal's propagation.

Indoor Localization Position

Boundary-aware Backward-Compatible Representation via Adversarial Learning in Image Retrieval

1 code implementation CVPR 2023 Tan Pan, Furong Xu, Xudong Yang, Sifeng He, Chen Jiang, Qingpei Guo, Feng Qian Xiaobo Zhang, Yuan Cheng, Lei Yang, Wei Chu

For traditional model upgrades, the old model will not be replaced by the new one until the embeddings of all the images in the database are re-computed by the new model, which takes days or weeks for a large amount of data.

Image Retrieval Retrieval

Search-Map-Search: A Frame Selection Paradigm for Action Recognition

no code implementations CVPR 2023 Mingjun Zhao, Yakun Yu, Xiaoli Wang, Lei Yang, Di Niu

To overcome the limitations of existing methods, we propose a Search-Map-Search learning paradigm which combines the advantages of heuristic search and supervised learning to select the best combination of frames from a video as one entity.

Action Recognition Video Understanding

AoI-Delay Tradeoff in Mobile Edge Caching: A Mixed-Order Drift-Plus-Penalty Algorithm

no code implementations18 Apr 2023 Ran Li, Chuan Huang, Xiaoqi Qin, Lei Yang

Mobile edge caching (MEC) is a promising technique to improve the quality of service (QoS) for mobile users (MU) by bringing data to the network edge.

Decision Making Scheduling

Zolly: Zoom Focal Length Correctly for Perspective-Distorted Human Mesh Reconstruction

1 code implementation ICCV 2023 Wenjia Wang, Yongtao Ge, Haiyi Mei, Zhongang Cai, Qingping Sun, Yanjun Wang, Chunhua Shen, Lei Yang, Taku Komura

As it is hard to calibrate single-view RGB images in the wild, existing 3D human mesh reconstruction (3DHMR) methods either use a constant large focal length or estimate one based on the background environment context, which can not tackle the problem of the torso, limb, hand or face distortion caused by perspective camera projection when the camera is close to the human body.

3D Human Pose Estimation 3D Reconstruction

SHERF: Generalizable Human NeRF from a Single Image

1 code implementation ICCV 2023 Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, Ziwei Liu

To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.

3D Human Reconstruction

BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

1 code implementation CVPR 2023 Lei Yang, Kaicheng Yu, Tao Tang, Jun Li, Kun Yuan, Li Wang, Xinyu Zhang, Peng Chen

In essence, instead of predicting the pixel-wise depth, we regress the height to the ground to achieve a distance-agnostic formulation to ease the optimization process of camera-only perception methods.

3D Object Detection Autonomous Driving +1

On-Device Unsupervised Image Segmentation

no code implementations24 Feb 2023 Junhuan Yang, Yi Sheng, Yuzhou Zhang, Weiwen Jiang, Lei Yang

What's more, for a larger size image in the BBBC005 dataset, the existing approach cannot be accommodated to Raspberry PI due to out of memory; on the other hand, SegHDC can obtain segmentation results within 3 minutes while achieving a 0. 9587 IoU score.

Image Segmentation Segmentation +2

Web Photo Source Identification based on Neural Enhanced Camera Fingerprint

1 code implementation18 Feb 2023 Feng Qian, Sifeng He, Honghao Huang, Huanyu Ma, Xiaobo Zhang, Lei Yang

With the growing popularity of smartphone photography in recent years, web photos play an increasingly important role in all walks of life.

Metric Learning

Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production

no code implementations30 Jan 2023 Anyi Rao, Xuekun Jiang, Yuwei Guo, Linning Xu, Lei Yang, Libiao Jin, Dahua Lin, Bo Dai

Amateurs working on mini-films and short-form videos usually spend lots of time and effort on the multi-round complicated process of setting and adjusting scenes, plots, and cameras to deliver satisfying video shots.

Development of A Real-time POCUS Image Quality Assessment and Acquisition Guidance System

no code implementations16 Dec 2022 Zhenge Jia, Yiyu Shi, Jingtong Hu, Lei Yang, Benjamin Nti

Point-of-care ultrasound (POCUS) is one of the most commonly applied tools for cardiac function imaging in the clinical routine of the emergency department and pediatric intensive care unit.

Image Quality Assessment

TransVCL: Attention-enhanced Video Copy Localization Network with Flexible Supervision

2 code implementations23 Nov 2022 Sifeng He, Yue He, Minlong Lu, Chen Jiang, Xudong Yang, Feng Qian, Xiaobo Zhang, Lei Yang, Jiandong Zhang

Previous methods typically start from frame-to-frame similarity matrix generated by cosine similarity between frame-level features of the input video pair, and then detect and refine the boundaries of copied segments on similarity matrix under temporal constraints.

Retrieval Video Retrieval

DCVQE: A Hierarchical Transformer for Video Quality Assessment

no code implementations10 Oct 2022 Zutong Li, Lei Yang

Inspired by our observation on the actions of human annotation, we put forward a Divide and Conquer Video Quality Estimator (DCVQE) for NR-VQA.

Video Quality Assessment Visual Question Answering (VQA)

A Keypoint Based Enhancement Method for Audio Driven Free View Talking Head Synthesis

no code implementations7 Oct 2022 Yichen Han, Ya Li, Yingming Gao, Jinlong Xue, Songpo Wang, Lei Yang

Then we used keypoint decomposition to extract video synthesis controlling parameters from the backend output and the source image.

Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms

1 code implementation21 Sep 2022 Hui En Pang, Zhongang Cai, Lei Yang, Tianwei Zhang, Ziwei Liu

Experiments with 10 backbones, ranging from CNNs to transformers, show the knowledge learnt from a proximity task is readily transferable to human mesh recovery.

3D human pose and shape estimation Benchmarking +1

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

no code implementations6 Sep 2022 Li Wang, Xinyu Zhang, Wenyuan Qin, Xiaoyu Li, Lei Yang, Zhiwei Li, Lei Zhu, Hong Wang, Jun Li, Huaping Liu

As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on the Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection.

3D Multi-Object Tracking Autonomous Driving +2

MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model

2 code implementations31 Aug 2022 Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu

Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.

Denoising Motion Synthesis

Federated Self-Supervised Contrastive Learning and Masked Autoencoder for Dermatological Disease Diagnosis

no code implementations24 Aug 2022 Yawen Wu, Dewen Zeng, Zhepeng Wang, Yi Sheng, Lei Yang, Alaina J. James, Yiyu Shi, Jingtong Hu

Self-supervised learning (SSL) methods, contrastive learning (CL) and masked autoencoders (MAE), can leverage the unlabeled data to pre-train models, followed by fine-tuning with limited labels.

Contrastive Learning Federated Learning +1

Mix-Teaching: A Simple, Unified and Effective Semi-Supervised Learning Framework for Monocular 3D Object Detection

1 code implementation10 Jul 2022 Lei Yang, Xinyu Zhang, Li Wang, Minghan Zhu, Chuang Zhang, Jun Li

Besides, by leveraging full training set and the additional 48K raw images of KITTI, it can further improve the MonoFlex by +4. 65% improvement on AP@0. 7 for car detection, reaching 18. 54% AP@0. 7, which ranks the 1st place among all monocular based methods on KITTI test leaderboard.

Autonomous Driving Model Optimization +2

Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space

1 code implementation7 Jul 2022 Wenqi Shao, Xun Zhao, Yixiao Ge, Zhaoyang Zhang, Lei Yang, Xiaogang Wang, Ying Shan, Ping Luo

It is challenging because the ground-truth model ranking for each task can only be generated by fine-tuning the pre-trained models on the target dataset, which is brute-force and computationally expensive.

Transferability

Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions

no code implementations25 Jun 2022 Weilin Wan, Lei Yang, Lingjie Liu, Zhuoying Zhang, Ruixing Jia, Yi-King Choi, Jia Pan, Christian Theobalt, Taku Komura, Wenping Wang

We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.

Human-Object Interaction Detection motion prediction +1

RT-DNAS: Real-time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation

no code implementations8 Jun 2022 Qing Lu, Xiaowei Xu, Shunjie Dong, Cong Hao, Lei Yang, Cheng Zhuo, Yiyu Shi

Accurately segmenting temporal frames of cine magnetic resonance imaging (MRI) is a crucial step in various real-time MRI guided cardiac interventions.

MRI segmentation Neural Architecture Search

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

1 code implementation17 May 2022 Fangzhou Hong, Mingyuan Zhang, Liang Pan, Zhongang Cai, Lei Yang, Ziwei Liu

Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.

Language Modelling Motion Synthesis +1

DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation

1 code implementation16 Mar 2022 Ailing Zeng, Xuan Ju, Lei Yang, Ruiyuan Gao, Xizhou Zhu, Bo Dai, Qiang Xu

This paper proposes a simple baseline framework for video-based 2D/3D human pose estimation that can achieve 10 times efficiency improvement over existing works without any performance degradation, named DeciWatch.

2D Human Pose Estimation 3D Human Pose Estimation +2

Visual-Tactile Sensing for Real-time Liquid Volume Estimation in Grasping

no code implementations23 Feb 2022 Fan Zhu, Ruixing Jia, Lei Yang, Youcan Yan, Zheng Wang, Jia Pan, Wenping Wang

We propose a deep visuo-tactile model for realtime estimation of the liquid inside a deformable container in a proprioceptive way. We fuse two sensory modalities, i. e., the raw visual inputs from the RGB camera and the tactile cues from our specific tactile sensor without any extra sensor calibrations. The robotic system is well controlled and adjusted based on the estimation model in real time.

Multi-Task Learning

The Larger The Fairer? Small Neural Networks Can Achieve Fairness for Edge Devices

no code implementations23 Feb 2022 Yi Sheng, Junhuan Yang, Yawen Wu, Kevin Mao, Yiyu Shi, Jingtong Hu, Weiwen Jiang, Lei Yang

Results show that FaHaNa can identify a series of neural networks with higher fairness and accuracy on a dermatology dataset.

Face Recognition Fairness +2

Federated Contrastive Learning for Dermatological Disease Diagnosis via On-device Learning

no code implementations14 Feb 2022 Yawen Wu, Dewen Zeng, Zhepeng Wang, Yi Sheng, Lei Yang, Alaina J. James, Yiyu Shi, Jingtong Hu

The recently developed self-supervised learning approach, contrastive learning (CL), can leverage the unlabeled data to pre-train a model, after which the model is fine-tuned on limited labeled data for dermatological disease diagnosis.

Contrastive Learning Federated Learning +1

Automated Architecture Search for Brain-inspired Hyperdimensional Computing

no code implementations11 Feb 2022 Junhuan Yang, Yi Sheng, Sizhe Zhang, Ruixuan Wang, Kenneth Foreman, Mikell Paige, Xun Jiao, Weiwen Jiang, Lei Yang

On the Clintox dataset, which tries to learn features from developed drugs that passed/failed clinical trials for toxicity reasons, the searched HDC architecture obtains the state-of-the-art ROC-AUC scores, which are 0. 80% higher than the manually designed HDC and 9. 75% higher than conventional neural networks.

Drug Discovery

SkiM: Skipping Memory LSTM for Low-Latency Real-Time Continuous Speech Separation

no code implementations26 Jan 2022 Chenda Li, Lei Yang, Weiqin Wang, Yanmin Qian

We adopt the time-domain speech separation method and the recently proposed Graph-PIT to build a super low-latency online speech separation model, which is very important for the real application.

Speech Separation

SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos

2 code implementations27 Dec 2021 Ailing Zeng, Lei Yang, Xuan Ju, Jiefeng Li, Jianyi Wang, Qiang Xu

With a simple yet effective motion-aware fully-connected network, SmoothNet improves the temporal smoothness of existing pose estimators significantly and enhances the estimation accuracy of those challenging frames as a side-effect.

2D Human Pose Estimation 3D Human Pose Estimation +2

SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement

1 code implementation NeurIPS 2021 Heyang Qin, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He

In this paper, we propose a fully automated and lightweight adaptive batching methodology to enable fine-grained batch size adaption (e. g., at a mini-batch level) that can achieve state-of-the-art performance with record breaking batch sizes.

Computational Efficiency

BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML

no code implementations12 Nov 2021 Yuhong Song, Edwin Hsing-Mean Sha, Qingfeng Zhuge, Rui Xu, Yongzhuo Zhang, Bingzhe Li, Lei Yang

Unlike ML on the edge, TinyML with a limited energy supply has higher demands on low-power execution.

Robust Event Classification Using Imperfect Real-world PMU Data

no code implementations19 Oct 2021 Yunchuan Liu, Lei Yang, Amir Ghasemkhani, Hanif Livani, Virgilio A. Centeno, Pin-Yu Chen, Junshan Zhang

Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e. g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers.

Classification Event Detection +1

Playing for 3D Human Recovery

no code implementations14 Oct 2021 Zhongang Cai, Mingyuan Zhang, Jiawei Ren, Chen Wei, Daxuan Ren, Zhengyu Lin, Haiyu Zhao, Lei Yang, Chen Change Loy, Ziwei Liu

Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.

iShape: A First Step Towards Irregular Shape Instance Segmentation

no code implementations30 Sep 2021 Lei Yang, Yan Zi Wei, Yisheng He, Wei Sun, Zhenhang Huang, Haibin Huang, Haoqiang Fan

In this paper, we introduce a brand new dataset to promote the study of instance segmentation for objects with irregular shapes.

Instance Segmentation Segmentation +1

Can Noise on Qubits Be Learned in Quantum Neural Network? A Case Study on QuantumFlow

no code implementations8 Sep 2021 Zhiding Liang, Zhepeng Wang, Junhuan Yang, Lei Yang, JinJun Xiong, Yiyu Shi, Weiwen Jiang

Specifically, this paper targets quantum neural network (QNN), and proposes to learn the errors in the training phase, so that the identified QNN model can be resilient to noise.

Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation

no code implementations ICCV 2021 Ailing Zeng, Xiao Sun, Lei Yang, Nanxuan Zhao, Minhao Liu, Qiang Xu

While the average prediction accuracy has been improved significantly over the years, the performance on hard poses with depth ambiguity, self-occlusion, and complex or rare poses is still far from satisfactory.

3D Human Pose Estimation 3D Pose Estimation +3

DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation

1 code implementation27 Jul 2021 Yilin Wen, Xiangyu Li, Hao Pan, Lei Yang, Zheng Wang, Taku Komura, Wenping Wang

Scalable 6D pose estimation for rigid objects from RGB images aims at handling multiple objects and generalizing to novel objects.

6D Pose Estimation Metric Learning +2

IPS300+: a Challenging Multimodal Dataset for Intersection Perception System

no code implementations5 Jun 2021 Huanan Wang, Xinyu Zhang, Jun Li, Zhiwei Li, Lei Yang, Shuyue Pan, Yongqiang Deng

Through an IPS (Intersection Perception System) installed at the diagonal of the intersection, this paper proposes a high-quality multimodal dataset for the intersection perception task.

Lite-FPN for Keypoint-based Monocular 3D Object Detection

1 code implementation1 May 2021 Lei Yang, Xinyu Zhang, Li Wang, Minghan Zhu, Jun Li

3D object detection with a single image is an essential and challenging task for autonomous driving.

Autonomous Driving Monocular 3D Object Detection +2

The Age of Correlated Features in Supervised Learning based Forecasting

no code implementations27 Feb 2021 MD Kamran Chowdhury Shisher, Heyang Qin, Lei Yang, Feng Yan, Yin Sun

In these applications, a neural network is trained to predict a time-varying target (e. g., solar power), based on multiple correlated features (e. g., temperature, humidity, and cloud coverage).

Edge Computing Assisted Autonomous Flight for UAV: Synergies between Vision and Communications

no code implementations10 Dec 2020 Quan Chen, Hai Zhu, Lei Yang, Xiaoqian Chen, Sofie Pollin, Evgenii Vinogradov

By proposing a framework of Edge Computing Assisted Autonomous Flight (ECAAF), we illustrate that vision and communications can interact with and assist each other with the aid of edge computing and offloading, and further speed up the UAV mission completion.

Edge-computing Trajectory Planning Networking and Internet Architecture Robotics Systems and Control Systems and Control

CoEdge: Cooperative DNN Inference with Adaptive Workload Partitioning over Heterogeneous Edge Devices

no code implementations6 Dec 2020 Liekang Zeng, Xu Chen, Zhi Zhou, Lei Yang, Junshan Zhang

CoEdge utilizes available computation and communication resources at the edge and dynamically partitions the DNN inference workload adaptive to devices' computing capabilities and network conditions.

Event Cause Analysis in Distribution Networks using Synchro Waveform Measurements

no code implementations25 Aug 2020 Iman Niazazari, Hanif Livani, Amir Ghasemkhani, Yunchuan Liu, Lei Yang

This paper presents a machine learning method for event cause analysis to enhance situational awareness in distribution networks.

BIG-bench Machine Learning

E-Tree Learning: A Novel Decentralized Model Learning Framework for Edge AI

no code implementations4 Aug 2020 Lei Yang, Yanyan Lu, Jiannong Cao, Jiaming Huang, Mingjin Zhang

In this paper, we propose a novel decentralized model learning approach, namely E-Tree, which makes use of a well-designed tree structure imposed on the edge devices.

Clustering Federated Learning

Mapping in a cycle: Sinkhorn regularized unsupervised learning for point cloud shapes

1 code implementation ECCV 2020 Lei Yang, Wenxi Liu, Zhiming Cui, Nenglun Chen, Wenping Wang

We propose an unsupervised learning framework with the pretext task of finding dense correspondences between point cloud shapes from the same category based on the cycle-consistency formulation.

Learn to Propagate Reliably on Noisy Affinity Graphs

no code implementations ECCV 2020 Lei Yang, Qingqiu Huang, Huaiyi Huang, Linning Xu, Dahua Lin

Recent works have shown that exploiting unlabeled data through label propagation can substantially reduce the labeling cost, which has been a critical issue in developing visual recognition models.

Open-Ended Question Answering

Standing on the Shoulders of Giants: Hardware and Neural Architecture Co-Search with Hot Start

1 code implementation17 Jul 2020 Weiwen Jiang, Lei Yang, Sakyasingha Dasgupta, Jingtong Hu, Yiyu Shi

To tackle this issue, HotNAS builds a chain of tools to design hardware to support compression, based on which a global optimizer is developed to automatically co-search all the involved search spaces.

Neural Architecture Search

Learning to Cluster Faces via Confidence and Connectivity Estimation

3 code implementations CVPR 2020 Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, Dahua Lin

With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.

Clustering Connectivity Estimation +2

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

no code implementations10 Feb 2020 Lei Yang, Zheyu Yan, Meng Li, Hyoukjun Kwon, Liangzhen Lai, Tushar Krishna, Vikas Chandra, Weiwen Jiang, Yiyu Shi

Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs).

Neural Architecture Search

Deep Human Answer Understanding for Natural Reverse QA

no code implementations1 Dec 2019 Rujing Yao, Linlin Hou, Lei Yang, Jie Gui, Qing Yin, Ou wu

This study focuses on a reverse question answering (QA) procedure, in which machines proactively raise questions and humans supply the answers.

Question Answering

Blockchain for Future Smart Grid: A Comprehensive Survey

1 code implementation8 Nov 2019 Muhammad Baqer Mollah, Jun Zhao, Dusit Niyato, Kwok-Yan Lam, Xin Zhang, Amer M. Y. M. Ghias, Leong Hai Koh, Lei Yang

In this paper, we aim to provide a comprehensive survey on application of blockchain in smart grid.

Cryptography and Security Distributed, Parallel, and Cluster Computing Networking and Internet Architecture Social and Information Networks Systems and Control Systems and Control

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

no code implementations31 Oct 2019 Weiwen Jiang, Qiuwen Lou, Zheyu Yan, Lei Yang, Jingtong Hu, Xiaobo Sharon Hu, Yiyu Shi

In this paper, we are the first to bring the computing-in-memory architecture, which can easily transcend the memory wall, to interplay with the neural architecture search, aiming to find the most efficient neural architectures with high network accuracy and maximized hardware efficiency.

Neural Architecture Search

Practical Low Latency Proof of Work Consensus

2 code implementations25 Sep 2019 Lei Yang, Vivek Bagaria, Gerui Wang, Mohammad Alizadeh, David Tse, Giulia Fanti, Pramod Viswanath

Bitcoin is the first fully-decentralized permissionless blockchain protocol to achieve a high level of security, but at the expense of poor throughput and latency.

Distributed, Parallel, and Cluster Computing Cryptography and Security Networking and Internet Architecture

Hardware/Software Co-Exploration of Neural Architectures

1 code implementation6 Jul 2019 Weiwen Jiang, Lei Yang, Edwin Sha, Qingfeng Zhuge, Shouzhen Gu, Sakyasingha Dasgupta, Yiyu Shi, Jingtong Hu

We propose a novel hardware and software co-exploration framework for efficient neural architecture search (NAS).

Neural Architecture Search

A Pvalue-guided Anomaly Detection Approach Combining Multiple Heterogeneous Log Parser Algorithms on IIoT Systems

no code implementations5 Jul 2019 Xueshuo Xie, Zhi Wang, Xuhang Xiao, Lei Yang, Shenwei Huang, Tao Li

In this paper, we use blockchain to prevent logs from being tampered with and propose a pvalue-guided anomaly detection approach.

Cryptography and Security

Sparse Solutions of a Class of Constrained Optimization Problems

no code implementations1 Jul 2019 Lei Yang, Xiaojun Chen, Shuhuang Xiang

Specifically, without any condition on the matrix $A$, we provide upper bounds in cardinality and infinity norm for the optimal solutions, and show that all optimal solutions must be on the boundary of the feasible set when $0<p<1$.

Denoising

SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing

1 code implementation19 Jun 2019 Haonan Qiu, Chaowei Xiao, Lei Yang, Xinchen Yan, Honglak Lee, Bo Li

In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples".

Attribute Face Recognition +1

Adaptive Intelligent Secondary Control of Microgrids Using a Biologically-Inspired Reinforcement Learning

no code implementations2 May 2019 Mohammad Jafari, Vahid Sarfi, Amir Ghasemkhani, Hanif Livani, Lei Yang, Hao Xu

In this paper, a biologically-inspired adaptive intelligent secondary controller is developed for microgrids to tackle system dynamics uncertainties, faults, and/or disturbances.

reinforcement-learning Reinforcement Learning (RL)

Learning to Cluster Faces on an Affinity Graph

3 code implementations CVPR 2019 Lei Yang, Xiaohang Zhan, Dapeng Chen, Junjie Yan, Chen Change Loy, Dahua Lin

Face recognition sees remarkable progress in recent years, and its performance has reached a very high level.

Clustering Face Recognition +1

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

no code implementations31 Jan 2019 Weiwen Jiang, Xinyi Zhang, Edwin H. -M. Sha, Lei Yang, Qingfeng Zhuge, Yiyu Shi, Jingtong Hu

In addition, with a performance abstraction model to analyze the latency of neural architectures without training, our framework can quickly prune architectures that do not satisfy the specification, leading to higher efficiency.

Neural Architecture Search

RPC: A Large-Scale Retail Product Checkout Dataset

no code implementations22 Jan 2019 Xiu-Shen Wei, Quan Cui, Lei Yang, Peng Wang, Lingqiao Liu

The main challenge of this problem comes from the large scale and the fine-grained nature of the product categories as well as the difficulty for collecting training images that reflect the realistic checkout scenarios due to continuous update of the products.

A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenters

no code implementations12 Sep 2018 Lei Yang, Jia Li, Defeng Sun, Kim-Chuan Toh

When the support points of the barycenter are pre-specified, this problem can be modeled as a linear programming (LP) problem whose size can be extremely large.

Feature Fusion through Multitask CNN for Large-scale Remote Sensing Image Segmentation

no code implementations24 Jul 2018 Shihao Sun, Lei Yang, Wenjie Liu, Ruirui Li

In recent years, Fully Convolutional Networks (FCN) has been widely used in various semantic segmentation tasks, including multi-modal remote sensing imagery.

Image Segmentation Segmentation +1

TreeSegNet: Adaptive Tree CNNs for Subdecimeter Aerial Image Segmentation

no code implementations29 Apr 2018 Kai Yue, Lei Yang, Ruirui Li, Wei Hu, Fan Zhang, Wei Li

For the task of subdecimeter aerial imagery segmentation, fine-grained semantic segmentation results are usually difficult to obtain because of complex remote sensing content and optical conditions.

Image Segmentation Segmentation +1

Accelerated Training for Massive Classification via Dynamic Class Selection

no code implementations5 Jan 2018 Xingcheng Zhang, Lei Yang, Junjie Yan, Dahua Lin

Massive classification, a classification task defined over a vast number of classes (hundreds of thousands or even millions), has become an essential part of many real-world systems, such as face recognition.

Classification Face Recognition +1

Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems

no code implementations18 Nov 2017 Lei Yang

To solve this class of problems, we propose a proximal gradient method with extrapolation and line search (PGels).

Variable Selection

On Scalable Inference with Stochastic Gradient Descent

no code implementations1 Jul 2017 Yixin Fang, Jinfeng Xu, Lei Yang

In many applications involving large dataset or online updating, stochastic gradient descent (SGD) provides a scalable way to compute parameter estimates and has gained increasing popularity due to its numerical convenience and memory efficiency.

A Non-monotone Alternating Updating Method for A Class of Matrix Factorization Problems

no code implementations18 May 2017 Lei Yang, Ting Kei Pong, Xiaojun Chen

Finally, we conduct some numerical experiments using real datasets to compare our method with some existing efficient methods for non-negative matrix factorization and matrix completion.

Matrix Completion

Minimum $n$-Rank Approximation via Iterative Hard Thresholding

no code implementations18 Nov 2013 Min Zhang, Lei Yang, Zheng-Hai Huang

Additionally, combining an effective heuristic for determining $n$-rank, we can also apply the proposed algorithm to solve MnRA when $n$-rank is unknown in advance.

Image Inpainting Video Inpainting

Cannot find the paper you are looking for? You can Submit a new open access paper.