Search Results for author: Wei Sun

Found 148 papers, 51 papers with code

VQA$^2$:Visual Question Answering for Video Quality Assessment

no code implementations6 Nov 2024 Ziheng Jia, ZiCheng Zhang, Jiaying Qian, HaoNing Wu, Wei Sun, Chunyi Li, Xiaohong Liu, Weisi Lin, Guangtao Zhai, Xiongkuo Min

Video Quality Assessment (VQA), a classic field in low-level visual quality evaluation, originally focused on quantitative video quality scoring.

Question Answering Video Quality Assessment +1

MOLA: Enhancing Industrial Process Monitoring Using Multi-Block Orthogonal Long Short-Term Memory Autoencoder

no code implementations10 Oct 2024 Fangyuan Ma, Cheng Ji, Jingde Wang, Wei Sun, Xun Tang, Zheyu Jiang

In this work, we introduce MOLA: a Multi-block Orthogonal Long short-term memory Autoencoder paradigm, to conduct accurate, reliable fault detection of industrial processes.

Fault Detection

R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?

1 code implementation7 Oct 2024 Chunyi Li, Jianbo Zhang, ZiCheng Zhang, HaoNing Wu, Yuan Tian, Wei Sun, Guo Lu, Xiaohong Liu, Xiongkuo Min, Weisi Lin, Guangtao Zhai

However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs.

Addition is All You Need for Energy-efficient Language Models

no code implementations1 Oct 2024 Hongyin Luo, Wei Sun

The new algorithm costs significantly less computation resource than 8-bit floating point multiplication but achieves higher precision.

Natural Language Understanding Question Answering

Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs

no code implementations30 Sep 2024 ZiCheng Zhang, Ziheng Jia, HaoNing Wu, Chunyi Li, Zijian Chen, Yingjie Zhou, Wei Sun, Xiaohong Liu, Xiongkuo Min, Weisi Lin, Guangtao Zhai

With the rising interest in research on Large Multi-modal Models (LMMs) for video understanding, many studies have emphasized general video comprehension capabilities, neglecting the systematic exploration into video quality understanding.

Benchmarking Multiple-choice +2

Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

no code implementations26 Sep 2024 Zehao Zhu, Wei Sun, Jun Jia, Wei Wu, Sibin Deng, Kai Li, Ying Chen, Xiongkuo Min, Jia Wang, Guangtao Zhai

For the subjective QoE study, we introduce the first live video streaming QoE dataset, TaoLive QoE, which consists of $42$ source videos collected from real live broadcasts and $1, 155$ corresponding distorted ones degraded due to a variety of streaming distortions, including conventional streaming distortions such as compression, stalling, as well as live streaming-specific distortions like frame skipping, variable frame rate, etc.

Optical Flow Estimation

Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency

1 code implementation1 Sep 2024 Wei Sun, Weixia Zhang, Yuqin Cao, Linhan Cao, Jun Jia, Zijian Chen, ZiCheng Zhang, Xiongkuo Min, Guangtao Zhai

To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception.

4k Image Quality Assessment

LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models

no code implementations26 Aug 2024 Qihang Ge, Wei Sun, Yu Zhang, Yunhao Li, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, Guangtao Zhai

Then, we design a spatiotemporal vision encoder to extract spatial and temporal features to represent the quality characteristics of videos, which are subsequently mapped into the language space by the spatiotemporal projector for modality alignment.

Large Language Model Video Quality Assessment +1

AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results

1 code implementation21 Aug 2024 Maksim Smirnov, Aleksandr Gushchin, Anastasia Antsiferova, Dmitry Vatolin, Radu Timofte, Ziheng Jia, ZiCheng Zhang, Wei Sun, Jiaying Qian, Yuqin Cao, Yinan Sun, Yuxin Zhu, Xiongkuo Min, Guangtao Zhai, Kanjar De, Qing Luo, Ao-Xiang Zhang, Peng Zhang, Haibo Lei, Linyan Jiang, Yaqing Li, Wenhui Meng, Zhenzhong Chen, Zhengxue Cheng, Jiahao Xiao, Jun Xu, Chenlong He, Qi Zheng, Ruoxi Zhu, Min Li, Yibo Fan, Zhengzhong Tu

The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H. 264, HEVC/H. 265, AV1, and VVC/H. 266) and containing a comprehensive collection of compression artifacts.

Image Manipulation valid +3

Depth-guided Texture Diffusion for Image Semantic Segmentation

no code implementations17 Aug 2024 Wei Sun, Yuan Li, Qixiang Ye, Jianbin Jiao, Yanzhao Zhou

By integrating this enriched depth map with the original RGB image into a joint feature embedding, our method effectively bridges the disparity between the depth map and the image, enabling more accurate semantic segmentation.

Object object-detection +4

Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVS

no code implementations16 Aug 2024 Wei Sun, Xiaosong Zhang, Fang Wan, Yanzhao Zhou, Yuan Li, Qixiang Ye, Jianbin Jiao

In SfM-free methods, inaccurate initial poses lead to misalignment issue, which, under the constraints of per-pixel image loss functions, results in excessive gradients, causing unstable optimization and poor convergence for NVS.

Camera Pose Estimation Novel View Synthesis +1

SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression

no code implementations8 Aug 2024 Linhan Cao, Wei Sun, Xiongkuo Min, Jun Jia, ZiCheng Zhang, Zijian Chen, Yucheng Zhu, Lizhou Liu, Qiubo Chen, Jing Chen, Guangtao Zhai

Just noticeable distortion (JND), representing the threshold of distortion in an image that is minimally perceptible to the human visual system (HVS), is crucial for image compression algorithms to achieve a trade-off between transmission bit rate and image quality.

Image Compression

Benchmarking AIGC Video Quality Assessment: A Dataset and Unified Model

no code implementations31 Jul 2024 Zhichao Zhang, Xinyue Li, Wei Sun, Jun Jia, Xiongkuo Min, ZiCheng Zhang, Chunyi Li, Zijian Chen, Puyi Wang, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Guangtao Zhai

For the objective perspective, we establish a benchmark for evaluating existing quality assessment metrics on the LGVQ dataset, which reveals that current metrics perform poorly on the LGVQ dataset.

Benchmarking Large Language Model +4

UNQA: Unified No-Reference Quality Assessment for Audio, Image, Video, and Audio-Visual Content

no code implementations29 Jul 2024 Yuqin Cao, Xiongkuo Min, Yixuan Gao, Wei Sun, Weisi Lin, Guangtao Zhai

In this paper, we propose the Unified No-reference Quality Assessment model (UNQA) for audio, image, video, and A/V content, which tries to train a single QA model across different media modalities.

Domain Adaptable Prescriptive AI Agent for Enterprise

no code implementations29 Jul 2024 Piero Orderique, Wei Sun, Kristjan Greenewald

Despite advancements in causal inference and prescriptive AI, its adoption in enterprise settings remains hindered primarily due to its technical complexity.

AI Agent Causal Inference +1

DiffStega: Towards Universal Training-Free Coverless Image Steganography with Diffusion Models

1 code implementation15 Jul 2024 Yiwei Yang, Zheyuan Liu, Jun Jia, Zhongpai Gao, Yunhao Li, Wei Sun, Xiaohong Liu, Guangtao Zhai

Traditional image steganography focuses on concealing one image within another, aiming to avoid steganalysis by unauthorized entities.

Diversity Image Steganography +1

Unlocking the Potential of Early Epochs: Uncertainty-aware CT Metal Artifact Reduction

no code implementations18 Jun 2024 Xinquan Yang, Guanqun Zhou, Wei Sun, Youjian Zhang, Zhongya Wang, Jiahui He, Zhicheng Zhang

In this paper, we have discovered that the uncertainty image computed from the restoration result of initial training weights can effectively highlight high-frequency regions, including metal artifacts.

Computed Tomography (CT) Metal Artifact Reduction

Perceiver-Prompt: Flexible Speaker Adaptation in Whisper for Chinese Disordered Speech Recognition

no code implementations14 Jun 2024 Yicong Jiang, Tianzi Wang, Xurong Xie, Juan Liu, Wei Sun, Nan Yan, Hui Chen, Lan Wang, Xunying Liu, Feng Tian

Disordered speech recognition profound implications for improving the quality of life for individuals afflicted with, for example, dysarthria.

speech-recognition Speech Recognition

GAIA: Rethinking Action Quality Assessment for AI-Generated Videos

1 code implementation10 Jun 2024 Zijian Chen, Wei Sun, Yuan Tian, Jun Jia, ZiCheng Zhang, Jiarui Wang, Ru Huang, Xiongkuo Min, Guangtao Zhai, Wenjun Zhang

Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nature of actions within AI-generated video (AIGV).

Action Quality Assessment

A-Bench: Are LMMs Masters at Evaluating AI-generated Images?

1 code implementation5 Jun 2024 ZiCheng Zhang, HaoNing Wu, Chunyi Li, Yingjie Zhou, Wei Sun, Xiongkuo Min, Zijian Chen, Xiaohong Liu, Weisi Lin, Guangtao Zhai

How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models.

Enhancing Blind Video Quality Assessment with Rich Quality-aware Features

1 code implementation14 May 2024 Wei Sun, HaoNing Wu, ZiCheng Zhang, Jun Jia, Zhichao Zhang, Linhan Cao, Qiubo Chen, Xiongkuo Min, Weisi Lin, Guangtao Zhai

Motivated by previous researches that leverage pre-trained features extracted from various computer vision models as the feature representation for BVQA, we further explore rich quality-aware features from pre-trained blind image quality assessment (BIQA) and BVQA models as auxiliary features to help the BVQA model to handle complex distortions and diverse content of social media videos.

Video Quality Assessment

Dual-Branch Network for Portrait Image Quality Assessment

1 code implementation14 May 2024 Wei Sun, Weixia Zhang, Yanwei Jiang, HaoNing Wu, ZiCheng Zhang, Jun Jia, Yingjie Zhou, Zhongpeng Ji, Xiongkuo Min, Weisi Lin, Guangtao Zhai

We employ the fidelity loss to train the model via a learning-to-rank manner to mitigate inconsistencies in quality scores in the portrait image quality assessment dataset PIQ.

Image Quality Assessment Learning-To-Rank +2

Deep Learning-Based Object Pose Estimation: A Comprehensive Survey

1 code implementation13 May 2024 Jian Liu, Wei Sun, Hui Yang, Zhiwen Zeng, Chongpei Liu, Jin Zheng, Xingyu Liu, Hossein Rahmani, Nicu Sebe, Ajmal Mian

Object pose estimation is a fundamental computer vision problem with broad applications in augmented reality and robotics.

Deep Learning Object +2

LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM

1 code implementation28 Apr 2024 ZiCheng Zhang, HaoNing Wu, Yingjie Zhou, Chunyi Li, Wei Sun, Chaofeng Chen, Xiongkuo Min, Xiaohong Liu, Weisi Lin, Guangtao Zhai

Although large multi-modality models (LMMs) have seen extensive exploration and application in various quality assessment studies, their integration into Point Cloud Quality Assessment (PCQA) remains unexplored.

Point Cloud Quality Assessment

Large Multi-modality Model Assisted AI-Generated Image Quality Assessment

1 code implementation27 Apr 2024 Puyi Wang, Wei Sun, ZiCheng Zhang, Jun Jia, Yanwei Jiang, Zhichao Zhang, Xiongkuo Min, Guangtao Zhai

Traditional deep neural network (DNN)-based image quality assessment (IQA) models leverage convolutional neural networks (CNN) or Transformer to learn the quality-aware feature representation, achieving commendable performance on natural scene images.

Image Quality Assessment

NTIRE 2024 Quality Assessment of AI-Generated Content Challenge

no code implementations25 Apr 2024 Xiaohong Liu, Xiongkuo Min, Guangtao Zhai, Chunyi Li, Tengchuan Kou, Wei Sun, HaoNing Wu, Yixuan Gao, Yuqin Cao, ZiCheng Zhang, Xiele Wu, Radu Timofte, Fei Peng, Huiyuan Fu, Anlong Ming, Chuanming Wang, Huadong Ma, Shuai He, Zifei Dou, Shu Chen, Huacong Zhang, Haiyi Xie, Chengwei Wang, Baoying Chen, Jishen Zeng, Jianquan Yang, Weigang Wang, Xi Fang, Xiaoxin Lv, Jun Yan, Tianwu Zhi, Yabin Zhang, Yaohui Li, Yang Li, Jingwen Xu, Jianzhao Liu, Yiting Liao, Junlin Li, Zihao Yu, Yiting Lu, Xin Li, Hossein Motamednia, S. Farhad Hosseini-Benvidi, Fengbin Guan, Ahmad Mahmoudi-Aznaveh, Azadeh Mansouri, Ganzorig Gankhuyag, Kihwan Yoon, Yifang Xu, Haotian Fan, Fangyuan Kong, Shiling Zhao, Weifeng Dong, Haibing Yin, Li Zhu, Zhiling Wang, Bingchen Huang, Avinab Saha, Sandeep Mishra, Shashank Gupta, Rajesh Sureddi, Oindrila Saha, Luigi Celona, Simone Bianco, Paolo Napoletano, Raimondo Schettini, Junfeng Yang, Jing Fu, Wei zhang, Wenzhi Cao, Limei Liu, Han Peng, Weijun Yuan, Zhan Li, Yihang Cheng, Yifan Deng, Haohui Li, Bowen Qu, Yao Li, Shuqing Luo, Shunzhou Wang, Wei Gao, Zihao Lu, Marcos V. Conde, Xinrui Wang, Zhibo Chen, Ruling Liao, Yan Ye, Qiulin Wang, Bing Li, Zhaokun Zhou, Miao Geng, Rui Chen, Xin Tao, Xiaoyu Liang, Shangkun Sun, Xingyuan Ma, Jiaze Li, Mengduo Yang, Haoran Xu, Jie zhou, Shiding Zhu, Bohan Yu, Pengfei Chen, Xinrui Xu, Jiabin Shen, Zhichao Duan, Erfan Asadi, Jiahe Liu, Qi Yan, Youran Qu, Xiaohui Zeng, Lele Wang, Renjie Liao

A total of 196 participants have registered in the video track.

Image Quality Assessment Image Restoration +2

THQA: A Perceptual Quality Assessment Database for Talking Heads

1 code implementation13 Apr 2024 Yingjie Zhou, ZiCheng Zhang, Wei Sun, Xiaohong Liu, Xiongkuo Min, Zhihua Wang, Xiao-Ping Zhang, Guangtao Zhai

In the realm of media technology, digital humans have gained prominence due to rapid advancements in computer technology.

Video Quality Assessment

AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment

no code implementations4 Apr 2024 Chunyi Li, Tengchuan Kou, Yixuan Gao, Yuqin Cao, Wei Sun, ZiCheng Zhang, Yingjie Zhou, Zhichao Zhang, Weixia Zhang, HaoNing Wu, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

With the rapid advancements in AI-Generated Content (AIGC), AI-Generated Images (AIGIs) have been widely applied in entertainment, education, and social media.

Image Quality Assessment

A resource-constrained stochastic scheduling algorithm for homeless street outreach and gleaning edible food

no code implementations15 Mar 2024 Conor M. Artman, Aditya Mate, Ezinne Nwankwo, Aliza Heching, Tsuyoshi Idé, Jiří\, Navrátil, Karthikeyan Shanmugam, Wei Sun, Kush R. Varshney, Lauri Goldkind, Gidi Kroch, Jaclyn Sawyer, Ian Watson

We developed a common algorithmic solution addressing the problem of resource-constrained outreach encountered by social change organizations with different missions and operations: Breaking Ground -- an organization that helps individuals experiencing homelessness in New York transition to permanent housing and Leket -- the national food bank of Israel that rescues food from farms and elsewhere to feed the hungry.

Scheduling Thompson Sampling

Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay

no code implementations27 Feb 2024 Hong Cheng, Huan Luo, Zhi Liu, Wei Sun, Weitao Li, Qiyue Li

Due to the fluctuation and intermittency of PV generation, the state gap, arising from time-inconsistent states and exacerbated by imprecisely known system delays, significantly impacts the accuracy of voltage control.

Multi-agent Reinforcement Learning

API Pack: A Massive Multi-Programming Language Dataset for API Call Generation

1 code implementation14 Feb 2024 Zhen Guo, Adriana Meza Soria, Wei Sun, Yikang Shen, Rameswar Panda

We introduce API Pack, a massive multi-programming language dataset containing more than 1 million instruction-API call pairs to improve the API call generation capabilities of large language models.

Perceptual Video Quality Assessment: A Survey

no code implementations5 Feb 2024 Xiongkuo Min, Huiyu Duan, Wei Sun, Yucheng Zhu, Guangtao Zhai

Perceptual video quality assessment plays a vital role in the field of video processing due to the existence of quality degradations introduced in various stages of video signal acquisition, compression, transmission and display.

Survey Video Quality Assessment

PresAIse, A Prescriptive AI Solution for Enterprises

no code implementations3 Feb 2024 Wei Sun, Scott McFaddin, Linh Ha Tran, Shivaram Subramanian, Kristjan Greenewald, Yeshi Tenzin, Zack Xue, Youssef Drissi, Markus Ettl

The first challenge is caused by the limitations of observational data for accurate causal inference which is typically a prerequisite for good decision-making.

Causal Inference Decision Making

Exploring the Naturalness of AI-Generated Images

1 code implementation9 Dec 2023 Zijian Chen, Wei Sun, HaoNing Wu, ZiCheng Zhang, Jun Jia, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, Guangtao Zhai, Wenjun Zhang

In this paper, we take the first step to benchmark and assess the visual naturalness of AI-generated images.

FS-BAND: A Frequency-Sensitive Banding Detector

no code implementations30 Nov 2023 Zijian Chen, Wei Sun, ZiCheng Zhang, Ru Huang, Fangfang Lu, Xiongkuo Min, Guangtao Zhai, Wenjun Zhang

Banding artifact, as known as staircase-like contour, is a common quality annoyance that happens in compression, transmission, etc.

Image Quality Assessment

OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition

1 code implementation CVPR 2024 Tongjia Chen, Hongshan Yu, Zhengeng Yang, Zechuan Li, Wei Sun, Chen Chen

Due to the resource-intensive nature of training vision-language models on expansive video data, a majority of studies have centered on adapting pre-trained image-language models to the video domain.

Descriptive Language Modelling +5

BAND-2k: Banding Artifact Noticeable Database for Banding Detection and Quality Assessment

1 code implementation29 Nov 2023 Zijian Chen, Wei Sun, Jun Jia, Fangfang Lu, ZiCheng Zhang, Jing Liu, Ru Huang, Xiongkuo Min, Guangtao Zhai

The quality score of a banding image is generated by pooling the banding detection maps masked by the spatial frequency filters.

2k Image Quality Assessment +1

A No-Reference Quality Assessment Method for Digital Human Head

no code implementations25 Oct 2023 Yingjie Zhou, ZiCheng Zhang, Wei Sun, Xiongkuo Min, Xianghe Ma, Guangtao Zhai

In this paper, we develop a novel no-reference (NR) method based on Transformer to deal with DHQA in a multi-task manner.

Geometry-Aware Video Quality Assessment for Dynamic Digital Human

no code implementations24 Oct 2023 ZiCheng Zhang, Yingjie Zhou, Wei Sun, Xiongkuo Min, Guangtao Zhai

Usually, DDHs are displayed as 2D rendered animation videos and it is natural to adapt video quality assessment (VQA) methods to DDH quality assessment (DDH-QA) tasks.

Attribute Video Quality Assessment +1

Generating Explanations in Medical Question-Answering by Expectation Maximization Inference over Evidence

no code implementations2 Oct 2023 Wei Sun, Mingxiao Li, Damien Sileo, Jesse Davis, Marie-Francine Moens

Medical Question Answering~(medical QA) systems play an essential role in assisting healthcare workers in finding answers to their questions.

Explanation Generation Question Answering

Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design

no code implementations22 Sep 2023 Chao Fang, Wei Sun, Aojun Zhou, Zhongfeng Wang

At the algorithm level, a bidirectional weight pruning method, dubbed BDWP, is proposed to leverage the N:M sparsity of weights during both forward and backward passes of DNN training, which can significantly reduce the computational cost while maintaining model accuracy.

Computational Efficiency Scheduling

Content Reduction, Surprisal and Information Density Estimation for Long Documents

no code implementations12 Sep 2023 Shaoxiong Ji, Wei Sun, Pekka Marttinen

We consider two interesting research questions: 1) how is information distributed over long documents, and 2) how does content reduction, such as token selection and text summarization, affect the information density in long documents.

Density Estimation Text Summarization

Joint Gaze-Location and Gaze-Object Detection

no code implementations26 Aug 2023 Danyang Tu, Wei Shen, Wei Sun, Xiongkuo Min, Guangtao Zhai

In contrast, we reframe the gaze following detection task as detecting human head locations and their gaze followings simultaneously, aiming at jointly detect human gaze location and gaze object in a unified and single-stage pipeline.

Object object-detection +1

Agglomerative Transformer for Human-Object Interaction Detection

no code implementations ICCV 2023 Danyang Tu, Wei Sun, Guangtao Zhai, Wei Shen

We propose an agglomerative Transformer (AGER) that enables Transformer-based human-object interaction (HOI) detectors to flexibly exploit extra instance-level cues in a single-stage and end-to-end manner for the first time.

Clustering Decoder +2

StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability

1 code implementation9 Aug 2023 Tengchuan Kou, Xiaohong Liu, Wei Sun, Jun Jia, Xiongkuo Min, Guangtao Zhai, Ning Liu

Indeed, most existing quality assessment models evaluate video quality as a whole without specifically taking the subjective experience of video stability into consideration.

Video Quality Assessment Video Stabilization +1

Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models

1 code implementation26 Jul 2023 Wei Sun, Wen Wen, Xiongkuo Min, Long Lan, Guangtao Zhai, Kede Ma

By minimalistic, we restrict our family of BVQA models to build only upon basic blocks: a video preprocessor (for aggressive spatiotemporal downsampling), a spatial quality analyzer, an optional temporal quality analyzer, and a quality regressor, all with the simplest possible instantiations.

Video Quality Assessment Visual Question Answering (VQA)

Subjective and Objective Audio-Visual Quality Assessment for User Generated Content

1 code implementation IEEE Transactions on Image Processing 2023 Yuqin Cao, Xiongkuo Min, Wei Sun, Guangtao Zhai

Then, to facilitate the development of AVQA fields, we construct a benchmark of AVQA models on the proposed SJTU-UAV database and other two AVQA databases, of which the benchmark models consist of AVQA models designed for synthetically distorted A/V sequences and AVQA models built through combining the popular VQA methods and audio features via support vector regressor (SVR).

Video Quality Assessment Visual Question Answering (VQA)

Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation

1 code implementation6 Jul 2023 ZiCheng Zhang, Wei Sun, Yingjie Zhou, HaoNing Wu, Chunyi Li, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, Weisi Lin

To address this gap, we propose SJTU-H3D, a subjective quality assessment database specifically designed for full-body digital humans.

First Place Solution to the CVPR'2023 AQTC Challenge: A Function-Interaction Centric Approach with Spatiotemporal Visual-Language Alignment

1 code implementation23 Jun 2023 Tom Tongjia Chen, Hongshan Yu, Zhengeng Yang, Ming Li, Zechuan Li, Jingwen Wang, Wei Miao, Wei Sun, Chen Chen

Affordance-Centric Question-driven Task Completion (AQTC) has been proposed to acquire knowledge from videos to furnish users with comprehensive and systematic instructions.

Human-Object Interaction Detection

GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment

1 code implementation9 Jun 2023 ZiCheng Zhang, Wei Sun, Houning Wu, Yingjie Zhou, Chunyi Li, Xiongkuo Min, Guangtao Zhai, Weisi Lin

Model-based 3DQA methods extract features directly from the 3D models, which are characterized by their high degree of complexity.

Point Cloud Quality Assessment

AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment

1 code implementation7 Jun 2023 Chunyi Li, ZiCheng Zhang, HaoNing Wu, Wei Sun, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, Weisi Lin

With the rapid advancements of the text-to-image generative model, AI-generated images (AGIs) have been widely applied to entertainment, education, social media, etc.

Image Quality Assessment

Learning Prescriptive ReLU Networks

no code implementations1 Jun 2023 Wei Sun, Asterios Tsiourvas

We study the problem of learning optimal policy from a set of discrete treatment options using observational data.

Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps

1 code implementation24 May 2023 Mingxiao Li, Tingyu Qu, Ruicong Yao, Wei Sun, Marie-Francine Moens

In this work, we conduct a systematic study of exposure bias in DPM and, intriguingly, we find that the exposure bias could be alleviated with a novel sampling method that we propose, without retraining the model.

Denoising

MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos

1 code implementation CVPR 2023 ZiCheng Zhang, Wei Wu, Wei Sun, Dangyang Tu, Wei Lu, Xiongkuo Min, Ying Chen, Guangtao Zhai

User-generated content (UGC) live videos are often bothered by various distortions during capture procedures and thus exhibit diverse visual qualities.

Video Quality Assessment Visual Question Answering (VQA)

A Perceptual Quality Assessment Exploration for AIGC Images

1 code implementation22 Mar 2023 ZiCheng Zhang, Chunyi Li, Wei Sun, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

\underline{AI} \underline{G}enerated \underline{C}ontent (\textbf{AIGC}) has gained widespread attention with the increasing efficiency of deep learning in content creation.

Image Quality Assessment

VDPVE: VQA Dataset for Perceptual Video Enhancement

1 code implementation16 Mar 2023 Yixuan Gao, Yuqin Cao, Tengchuan Kou, Wei Sun, Yunlong Dong, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

Few researchers have specifically proposed a video quality assessment method for video enhancement, and there is also no comprehensive video quality assessment dataset available in public.

Deblurring valid +3

Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Images

1 code implementation14 Mar 2023 ZiCheng Zhang, Wei Sun, Yingjie Zhou, Jun Jia, Zhichao Zhang, Jing Liu, Xiongkuo Min, Guangtao Zhai

Computer graphics images (CGIs) are artificially generated by means of computer programs and are widely perceived under various scenarios, such as games, streaming media, etc.

Full Point Encoding for Local Feature Aggregation in 3D Point Clouds

no code implementations8 Mar 2023 Yong He, Hongshan Yu, Zhengeng Yang, Xiaoyan Liu, Wei Sun, Ajmal Mian

In particular, we achieve state-of-the-art semantic segmentation results of 76% mIoU on S3DIS 6-fold and 72. 2% on S3DIS Area5.

object-detection Object Detection +2

Audio-Visual Quality Assessment for User Generated Content: Database and Method

no code implementations4 Mar 2023 Yuqin Cao, Xiongkuo Min, Wei Sun, XiaoPing Zhang, Guangtao Zhai

Specifically, we construct the first UGC AVQA database named the SJTU-UAV database, which includes 520 in-the-wild UGC audio and video (A/V) sequences, and conduct a user study to obtain the mean opinion scores of the A/V sequences.

Video Quality Assessment Visual Question Answering (VQA)

EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessment

no code implementations17 Feb 2023 ZiCheng Zhang, Wei Sun, Yingjie Zhou, Wei Lu, Yucheng Zhu, Xiongkuo Min, Guangtao Zhai

Currently, great numbers of efforts have been put into improving the effectiveness of 3D model quality assessment (3DQA) methods.

Scalable Optimal Multiway-Split Decision Trees with Constraints

no code implementations14 Feb 2023 Shivaram Subramanian, Wei Sun

However, existing MIP methods that build on an arc-based formulation do not scale well as the number of binary variables is in the order of $\mathcal{O}(2^dN)$, where $d$ and $N$ refer to the depth of the tree and the size of the dataset.

ARC

Learning Complementary Policies for Human-AI Teams

no code implementations6 Feb 2023 Ruijiang Gao, Maytal Saar-Tsechansky, Maria De-Arteaga, Ligong Han, Wei Sun, Min Kyung Lee, Matthew Lease

We then extend our approach to leverage opportunities and mitigate risks that arise in important contexts in practice: 1) when a team is composed of multiple humans with differential and potentially complementary abilities, 2) when the observational data includes consistent deterministic actions, and 3) when the covariate distribution of future decisions differ from that in the historical data.

DDH-QA: A Dynamic Digital Humans Quality Assessment Database

1 code implementation24 Dec 2022 ZiCheng Zhang, Yingjie Zhou, Wei Sun, Wei Lu, Xiongkuo Min, Yu Wang, Guangtao Zhai

In recent years, large amounts of effort have been put into pushing forward the real-world application of dynamic digital human (DDH).

Video Quality Assessment

DQnet: Cross-Model Detail Querying for Camouflaged Object Detection

no code implementations16 Dec 2022 Wei Sun, Chengao Liu, Linyan Zhang, Yu Li, Pengxu Wei, Chang Liu, Jialing Zou, Jianbin Jiao, Qixiang Ye

Optimizing a convolutional neural network (CNN) for camouflaged object detection (COD) tends to activate local discriminative regions while ignoring complete object extent, causing the partial activation issue which inevitably leads to missing or redundant regions of objects.

Object object-detection +2

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

no code implementations23 Nov 2022 Tingting Zhao, Ying Wang, Wei Sun, Yarui Chen, Gang Niub, Masashi Sugiyama

Meanwhile, we divide the whole learning task into learning with the large-scale representation models in an unsupervised manner and learning with the small-scale policy model in the RL manner. The small policy model facilitates policy learning, while not sacrificing generalization and expressiveness via the large representation model.

reinforcement-learning Reinforcement Learning +2

Perceptual Quality Assessment for Digital Human Heads

1 code implementation20 Sep 2022 ZiCheng Zhang, Yingjie Zhou, Wei Sun, Xiongkuo Min, Yuzhe Wu, Guangtao Zhai

Digital humans are attracting more and more research interest during the last decade, the generation, representation, rendering, and animation of which have been put into large amounts of effort.

MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment

1 code implementation1 Sep 2022 ZiCheng Zhang, Wei Sun, Xiongkuo Min, Quan Zhou, Jun He, Qiyuan Wang, Guangtao Zhai

In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling.

Point Cloud Quality Assessment

Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric

1 code implementation30 Aug 2022 ZiCheng Zhang, Wei Sun, Yucheng Zhu, Xiongkuo Min, Wei Wu, Ying Chen, Guangtao Zhai

To tackle the challenge of point cloud quality assessment (PCQA), many PCQA methods have been proposed to evaluate the visual quality levels of point clouds by assessing the rendered static 2D projections.

Image Quality Assessment Point Cloud Quality Assessment +2

Towards Learning in Grey Spatiotemporal Systems: A Prophet to Non-consecutive Spatiotemporal Dynamics

no code implementations17 Aug 2022 Zhengyang Zhou, Yang Kuo, Wei Sun, Binwu Wang, Min Zhou, Yunan Zong, Yang Wang

To infer region-wise proximity under flexible factor-wise combinations and enable dynamic neighborhood aggregations, we further disentangle compounded influences of exogenous factors on region-wise proximity and learn to aggregate them.

Uncertainty Quantification

Domain-invariant Prototypes for Semantic Segmentation

no code implementations12 Aug 2022 Zhengeng Yang, Hongshan Yu, Wei Sun, Li-Cheng, Ajmal Mian

In this paper, we present an easy-to-train framework that learns domain-invariant prototypes for domain adaptive semantic segmentation.

Domain Adaptation Few-Shot Learning +2

MLRIP: Pre-training a military language representation model with informative factual knowledge and professional knowledge base

no code implementations28 Jul 2022 Hui Li, Xuekang Yang, Xin Zhao, Lin Yu, Jiping Zheng, Wei Sun

Incorporating prior knowledge into pre-trained language models has proven to be effective for knowledge-driven NLP tasks, such as entity typing and relation extraction.

Entity Typing Relation Extraction

Constrained Prescriptive Trees via Column Generation

no code implementations20 Jul 2022 Shivaram Subramanian, Wei Sun, Youssef Drissi, Markus Ettl

We introduce a novel path-based mixed-integer program (MIP) formulation which identifies a (near) optimal policy efficiently via column generation.

A Distributionally Robust Resilience Enhancement Strategy for Distribution Networks Considering Decision-Dependent Contingencies

no code implementations2 Jul 2022 Yujia Li, Shunbo Lei, Wei Sun, Chenxi Hu, Yunhe Hou

When performing the resilience enhancement for distribution networks, there are two obstacles to reliably model the uncertain contingencies: 1) decision-dependent uncertainty (DDU) due to various line hardening decisions, and 2) distributional ambiguity due to limited outage information during extreme weather events (EWEs).

Subjective Quality Assessment for Images Generated by Computer Graphics

no code implementations10 Jun 2022 Tao Wang, ZiCheng Zhang, Wei Sun, Xiongkuo Min, Wei Lu, Guangtao Zhai

However, limited work has been put forward to tackle the problem of computer graphics generated images' quality assessment (CG-IQA).

NR-IQA

A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences

no code implementations9 Jun 2022 Yu Fan, ZiCheng Zhang, Wei Sun, Xiongkuo Min, Wei Lu, Tao Wang, Ning Liu, Guangtao Zhai

Point cloud is one of the most widely used digital formats of 3D models, the visual quality of which is quite sensitive to distortions such as downsampling, noise, and compression.

Point Cloud Quality Assessment

Deep Neural Network for Blind Visual Quality Assessment of 4K Content

no code implementations9 Jun 2022 Wei Lu, Wei Sun, Xiongkuo Min, Wenhan Zhu, Quan Zhou, Jun He, Qiyuan Wang, ZiCheng Zhang, Tao Wang, Guangtao Zhai

In this paper, we propose a deep learning-based BIQA model for 4K content, which on one hand can recognize true and pseudo 4K content and on the other hand can evaluate their perceptual visual quality.

4k Blind Image Quality Assessment +1

A No-Reference Deep Learning Quality Assessment Method for Super-resolution Images Based on Frequency Maps

no code implementations9 Jun 2022 ZiCheng Zhang, Wei Sun, Xiongkuo Min, Wenhan Zhu, Tao Wang, Wei Lu, Guangtao Zhai

Therefore, in this paper, we propose a no-reference deep-learning image quality assessment method based on frequency maps because the artifacts caused by SISR algorithms are quite sensitive to frequency information.

Image Quality Assessment Image Super-Resolution

Blind Surveillance Image Quality Assessment via Deep Neural Network Combined with the Visual Saliency

no code implementations9 Jun 2022 Wei Lu, Wei Sun, Wenhan Zhu, Xiongkuo Min, ZiCheng Zhang, Tao Wang, Guangtao Zhai

In this paper, we first conduct an example experiment (i. e. the face detection task) to demonstrate that the quality of the SIs has a crucial impact on the performance of the IVSS, and then propose a saliency-based deep neural network for the blind quality assessment of the SIs, which helps IVSS to filter the low-quality SIs and improve the detection and recognition performance.

Face Detection Image Quality Assessment

Perceptual Quality Assessment for Fine-Grained Compressed Images

no code implementations8 Jun 2022 ZiCheng Zhang, Wei Sun, Wei Wu, Ying Chen, Xiongkuo Min, Guangtao Zhai

Nowadays, the mainstream full-reference (FR) metrics are effective to predict the quality of compressed images at coarse-grained levels (the bit rates differences of compressed images are obvious), however, they may perform poorly for fine-grained compressed images whose bit rates differences are quite subtle.

Full-Reference Image Quality Assessment Image Compression

Video-based Human-Object Interaction Detection from Tubelet Tokens

no code implementations4 Jun 2022 Danyang Tu, Wei Sun, Xiongkuo Min, Guangtao Zhai, Wei Shen

We present a novel vision Transformer, named TUTOR, which is able to learn tubelet tokens, served as highly-abstracted spatiotemporal representations, for video-based human-object interaction (V-HOI) detection.

Human-Object Interaction Detection

A Deep Learning based No-reference Quality Assessment Model for UGC Videos

1 code implementation29 Apr 2022 Wei Sun, Xiongkuo Min, Wei Lu, Guangtao Zhai

The proposed model utilizes very sparse frames to extract spatial features and dense frames (i. e. the video chunk) with a very low spatial resolution to extract motion features, which thereby has low computational complexity.

Image Quality Assessment Video Quality Assessment

Cyber-Physical Vulnerability Assessment of P2P Energy Exchanges in Active Distribution Networks

no code implementations26 Apr 2022 Hamed Haggi, Wei Sun

Owing to the decreasing costs of distributed energy resources (DERs) as well as decarbonization policies, power systems are undergoing a modernization process.

energy trading

A Unified Review of Deep Learning for Automated Medical Coding

no code implementations8 Jan 2022 Shaoxiong Ji, Wei Sun, Xiaobo Li, Hang Dong, Ara Taalas, Yijia Zhang, Honghan Wu, Esa Pitkänen, Pekka Marttinen

Automated medical coding, an essential task for healthcare operation and delivery, makes unstructured data manageable by predicting medical codes from clinical documents.

Decoder Deep Learning

Enhancing Counterfactual Classification via Self-Training

1 code implementation8 Dec 2021 Ruijiang Gao, Max Biggs, Wei Sun, Ligong Han

We approach this task as a domain adaptation problem and propose a self-training algorithm which imputes outcomes with categorical values for finite unseen actions in the observational data to simulate a randomized trial through pseudolabeling, which we refer to as Counterfactual Self-Training (CST).

Classification counterfactual +2

DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks

1 code implementation NeurIPS 2021 Wei Sun, Aojun Zhou, Sander Stuijk, Rob Wijnhoven, Andrew Oakleigh Nelson, Hongsheng Li, Henk Corporaal

However, the existing N:M algorithms only address the challenge of how to train N:M sparse neural networks in a uniform fashion (i. e. every layer has the same N:M sparsity) and suffer from a significant accuracy drop for high sparsity (i. e. when sparsity > 80\%).

Network Pruning

Loss Functions for Discrete Contextual Pricing with Observational Data

no code implementations18 Nov 2021 Max Biggs, Ruijiang Gao, Wei Sun

The goal of this paper is to formulate loss functions that can be used for evaluating pricing policies directly from observational data, rather than going through an intermediate demand estimation stage, which may suffer from bias.

Management Off-policy evaluation

iShape: A First Step Towards Irregular Shape Instance Segmentation

no code implementations30 Sep 2021 Lei Yang, Yan Zi Wei, Yisheng He, Wei Sun, Zhenhang Huang, Haibin Huang, Haoqiang Fan

In this paper, we introduce a brand new dataset to promote the study of instance segmentation for objects with irregular shapes.

Instance Segmentation Segmentation +1

Multitask Balanced and Recalibrated Network for Medical Code Prediction

2 code implementations6 Sep 2021 Wei Sun, Shaoxiong Ji, Erik Cambria, Pekka Marttinen

Nevertheless, automated medical coding is still challenging because of the imbalanced class problem, complex code association, and noise in lengthy documents.

Medical Code Prediction Multi-Task Learning

Proactive Rolling-Horizon based Scheduling of Hydrogen Systems for Resilient Power Grids

no code implementations17 Jul 2021 Hamed Haggi, Wei Sun, James M. Fenton, Paul Brooker

Deploying distributed energy resources (DERs) and other smart grid technologies have increased the complexity of power grids and made them more vulnerable to natural disasters and cyber-physical-human (CPH) threats.

Scheduling

Learning-Based Nonlinear $H^\infty$ Control via Game-Theoretic Differential Dynamic Programming

no code implementations9 Jul 2021 Wei Sun, Theodore B. Trafalis

In this work, we present a learning-based nonlinear $H^\infty$ control algorithm that guarantee system performance under learned dynamics and disturbance estimate.

regression

No-Reference Quality Assessment for 3D Colored Point Cloud and Mesh Models

2 code implementations5 Jul 2021 ZiCheng Zhang, Wei Sun, Xiongkuo Min, Tao Wang, Wei Lu, Guangtao Zhai

Therefore, many related studies such as point cloud quality assessment (PCQA) and mesh quality assessment (MQA) have been carried out to measure the visual quality degradations of 3D models.

Point Cloud Quality Assessment

Learned Interpretable Residual Extragradient ISTA for Sparse Coding

no code implementations22 Jun 2021 Lin Kong, Wei Sun, Fanhua Shang, Yuanyuan Liu, Hongying Liu

Recently, the study on learned iterative shrinkage thresholding algorithm (LISTA) has attracted increasing attentions.

Learning Audio-Visual Dereverberation

1 code implementation14 Jun 2021 Changan Chen, Wei Sun, David Harwath, Kristen Grauman

We introduce Visually-Informed Dereverberation of Audio (VIDA), an end-to-end approach that learns to remove reverberation based on both the observed monaural sound and visual scene.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Fisher-Pitman permutation tests based on nonparametric Poisson mixtures with application to single cell genomics

no code implementations6 Jun 2021 Zhen Miao, Weihao Kong, Ramya Korlakai Vinayak, Wei Sun, Fang Han

This paper investigates the theoretical and empirical performance of Fisher-Pitman-type permutation tests for assessing the equality of unknown Poisson mixture distributions.

Deep Learning based Full-reference and No-reference Quality Assessment Models for Compressed UGC Videos

1 code implementation2 Jun 2021 Wei Sun, Tao Wang, Xiongkuo Min, Fuwang Yi, Guangtao Zhai

The proposed VQA framework consists of three modules, the feature extraction module, the quality regression module, and the quality pooling module.

regression Video Quality Assessment

Proactive Scheduling of Hydrogen Systems for Resilience Enhancement of Distribution Networks

no code implementations1 Jun 2021 Hamed Haggi, Wei Sun, James M. Fenton, Paul Brooker

Recent advances in smart grid technologies bring opportunities to better control the modern and complex power grids with renewable integration.

Scheduling

End-to-End Jet Classification of Boosted Top Quarks with the CMS Open Data

no code implementations19 Apr 2021 Michael Andrews, Bjorn Burkle, Yi-fan Chen, Davide DiCroce, Sergei Gleyzer, Ulrich Heintz, Meenakshi Narain, Manfred Paulini, Nikolas Pervan, Yusef Shafi, Wei Sun, Emanuele Usai, Kun Yang

We describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon.

Deep Learning

Multitask Recalibrated Aggregation Network for Medical Code Prediction

1 code implementation2 Apr 2021 Wei Sun, Shaoxiong Ji, Erik Cambria, Pekka Marttinen

Medical coding translates professionally written medical reports into standardized codes, which is an essential part of medical information systems and health insurance reimbursement.

Medical Code Prediction Representation Learning

Deep Consensus Learning

no code implementations15 Mar 2021 Wei Sun, Tianfu Wu

For the real image corresponding to the input layout, its mask also is computed by the inference network, and then used by the generator to reconstruct the real image.

Image Generation Segmentation +1

Deep Learning Based 3D Segmentation: A Survey

no code implementations9 Mar 2021 Yong He, Hongshan Yu, Xiaoyan Liu, Zhengeng Yang, Wei Sun, Saeed Anwar, Ajmal Mian

3D segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving and robotics.

Autonomous Driving Deep Learning +4

Sum-Rate Maximization in Distributed Intelligent Reflecting Surfaces-Aided mmWave Communications

no code implementations18 Jan 2021 Yue Xiu, Wei Sun, Jiao Wu, Guan Gui, Ning Wei, Zhongpei Zhang

The solution to transmit beamforming at the BS and the phase shifts at the IRS are derived by using the successive convex approximation (SCA)-based algorithm, and a greedy algorithm is proposed to design the IRS switch vector.

Non-uniform Motion Deblurring with Blurry Component Divided Guidance

no code implementations15 Jan 2021 Pei Wang, Wei Sun, Qingsen Yan, Axi Niu, Rui Li, Yu Zhu, Jinqiu Sun, Yanning Zhang

To tackle the above problems, we present a deep two-branch network to deal with blurry images via a component divided module, which divides an image into two components based on the representation of blurry degree.

Blind Image Deblurring Decoder +2

Counterfactual Self-Training

no code implementations1 Jan 2021 Ruijiang Gao, Max Biggs, Wei Sun, Ligong Han

We approach this task as a domain adaptation problem and propose a self-training algorithm which imputes outcomes for the unseen actions in the observational data to simulate a randomized trial.

counterfactual Domain Adaptation +1

Uplink Achievable Rate Maximization for Reconfigurable Intelligent Surface Aided Millimeter Wave Systems with Resolution-Adaptive ADCs

no code implementations27 Nov 2020 Yue Xiu, Jun Zhao, Ertugrul Basar, Marco Di Renzo, Wei Sun, Guan Gui, Ning Wei

In this letter, we investigate the uplink of a reconfigurable intelligent surface (RIS)-aided millimeter-wave (mmWave) multi-user system.

Quantization

Multi-Objective PMU Allocation for Resilient Power System Monitoring

no code implementations15 Oct 2020 Hamed Haggi, Wei Sun, Junjian Qi

Phasor measurement units (PMUs) enable better system monitoring and security enhancement in smart grids.

Fatigue-aware Bandits for Dependent Click Models

no code implementations22 Aug 2020 Junyu Cao, Wei Sun, Zuo-Jun, Shen, Markus Ettl

Based on user's feedback, the platform learns the relevance of the underlying content as well as the discounting effect due to content fatigue.

Recommendation Systems

Secrecy Rate Maximization for Intelligent Reflecting Surface Aided SWIPT Systems

no code implementations22 Jul 2020 Wei Sun, Qingyang Song, Lei Guo, Jun Zhao

Simultaneous wireless information and power transfer (SWIPT) and intelligent reflecting surface (IRS) are two promising techniques for providing enhanced wireless communication capability and sustainable energy supply to energy-constrained wireless devices.

Model Distillation for Revenue Optimization: Interpretable Personalized Pricing

no code implementations3 Jul 2020 Max Biggs, Wei Sun, Markus Ettl

Data-driven pricing strategies are becoming increasingly common, where customers are offered a personalized price based on features that are predictive of their valuation of a product.

BIG-bench Machine Learning Fairness

Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis

3 code implementations25 Mar 2020 Wei Sun, Tianfu Wu

This paper focuses on a recent emerged task, layout-to-image, to learn generative models that are capable of synthesizing photo-realistic images from spatial layout (i. e., object bounding boxes configured in an image lattice) and style (i. e., structural and appearance variations encoded by latent vectors).

Layout-to-Image Generation Object

Machine learning based co-creative design framework

no code implementations23 Jan 2020 Brian Quanz, Wei Sun, Ajay Deshpande, Dhruv Shah, Jae-Eun Park

We propose a flexible, co-creative framework bringing together multiple machine learning techniques to assist human users to efficiently produce effective creative designs.

BIG-bench Machine Learning

Learning to Zoom-in via Learning to Zoom-out: Real-world Super-resolution by Generating and Adapting Degradation

no code implementations8 Jan 2020 Dong Gong, Wei Sun, Qinfeng Shi, Anton Van Den Hengel, Yanning Zhang

Most learning-based super-resolution (SR) methods aim to recover high-resolution (HR) image from a given low-resolution (LR) image via learning on LR-HR image pairs.

Super-Resolution

Making Predictive Coding Networks Generative

no code implementations26 Oct 2019 Jeff Orchard, Wei Sun

This paper studies this phenomenon, and proposes a simple solution that promotes the generation of input samples that resemble the training inputs.

Image Synthesis From Reconfigurable Layout and Style

4 code implementations ICCV 2019 Wei Sun, Tianfu Wu

Despite remarkable recent progress on both unconditional and conditional image synthesis, it remains a long-standing problem to learn generative models that are capable of synthesizing realistic and sharp images from reconfigurable spatial layout (i. e., bounding boxes + class labels in an image lattice) and style (i. e., structural and appearance variations encoded by latent vectors), especially at high resolution.

Layout-to-Image Generation

Attentive Normalization

2 code implementations ECCV 2020 Xilai Li, Wei Sun, Tianfu Wu

In state-of-the-art deep neural networks, both feature normalization and feature attention have become ubiquitous.

Image Classification Instance Segmentation +3

3D Virtual Garment Modeling from RGB Images

no code implementations31 Jul 2019 Yi Xu, Shanglin Yang, Wei Sun, Li Tan, Kefeng Li, Hui Zhou

The predicted landmarks are used for estimating sizing information of the garment.

Mixed Reality Multi-Task Learning

A support vector regression-based multi-fidelity surrogate model

no code implementations22 Jun 2019 Maolin Shi, Shuo Wang, Wei Sun, Liye Lv, Xueguan Song

Computational simulations with different fidelity have been widely used in engineering design.

regression

High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes

no code implementations31 May 2019 Maolin Shi, Wei Sun, Xueguan Song, Hongyou Li

The proposed approach is consisted of low-level SVR models for the input parameters of larger sample sizes and high-level SVR model for the input parameters of smaller sample sizes.

Dynamic Learning with Frequent New Product Launches: A Sequential Multinomial Logit Bandit Problem

no code implementations29 Apr 2019 Junyu Cao, Wei Sun

Motivated by the phenomenon that companies introduce new products to keep abreast with customers' rapidly changing tastes, we consider a novel online learning setting where a profit-maximizing seller needs to learn customers' preferences through offering recommendations, which may contain existing products and new products that are launched in the middle of a selling period.

Product Recommendation

Ranking-Based Autoencoder for Extreme Multi-label Classification

no code implementations NAACL 2019 Bingyu Wang, Li Chen, Wei Sun, Kechen Qin, Kefeng Li, Hui Zhou

Extreme Multi-label classification (XML) is an important yet challenging machine learning task, that assigns to each instance its most relevant candidate labels from an extremely large label collection, where the numbers of labels, features and instances could be thousands or millions.

Classification Extreme Multi-Label Classification +2

Dynamic Learning of Sequential Choice Bandit Problem under Marketing Fatigue

1 code implementation19 Mar 2019 Junyu Cao, Wei Sun

Based on user feedback, the platform dynamically learns users' abandonment distribution and their valuations of messages to determine the length of the sequence and the order of the messages, while maximizing the cumulative payoff over a horizon of length T. We refer to this online learning task as the sequential choice bandit problem.

Combinatorial Optimization Marketing

Real time backbone for semantic segmentation

no code implementations16 Mar 2019 Zhengeng Yang, Hongshan Yu, Qiang Fu, Wei Sun, Wenyan Jia, Mingui Sun, Zhi-Hong Mao

The rapid development of autonomous driving in recent years presents lots of challenges for scene understanding.

Autonomous Driving Model Compression +3

Learning Spatial Pyramid Attentive Pooling in Image Synthesis and Image-to-Image Translation

no code implementations18 Jan 2019 Wei Sun, Tianfu Wu

In experiments, the proposed SPAP is tested in GANs on the Celeba-HQ-128 dataset~\cite{karras2017progressive}, and tested in CycleGANs on the Image-to-Image translation datasets including the Cityscape dataset~\cite{cordts2016cityscapes}, Facade and Aerial Maps dataset~\cite{zhu2017unpaired}, both obtaining better performance.

Image-to-Image Translation Translation

Sketching Method for Large Scale Combinatorial Inference

no code implementations NeurIPS 2018 Wei Sun, Junwei Lu, Han Liu

In order to test the hypotheses on their topological structures, we propose two adjacency matrix sketching frameworks: neighborhood sketching and subgraph sketching.

regression

Interpretable Spatio-temporal Attention for Video Action Recognition

no code implementations1 Oct 2018 Lili Meng, Bo Zhao, Bo Chang, Gao Huang, Wei Sun, Frederich Tung, Leonid Sigal

Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition.

Action Recognition Temporal Action Localization