Search Results for author: Yifan Jiang

Found 43 papers, 19 papers with code

FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting

no code implementations1 Dec 2023 Zehao Zhu, Zhiwen Fan, Yifan Jiang, Zhangyang Wang

Novel view synthesis from limited observations remains an important and persistent task.

Novel View Synthesis

BRAINTEASER: Lateral Thinking Puzzles for Large Language Models

no code implementations8 Oct 2023 Yifan Jiang, Filip Ilievski, Kaixin Ma, Zhivar Sourati

The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms.

Distractor Generation Language Modelling +3

Pose-Free Generalizable Rendering Transformer

1 code implementation5 Oct 2023 Zhiwen Fan, Panwang Pan, Peihao Wang, Yifan Jiang, Hanwen Jiang, Dejia Xu, Zehao Zhu, Dilin Wang, Zhangyang Wang

To address this challenge, we introduce PF-GRT, a new Pose-Free framework for Generalizable Rendering Transformer, eliminating the need for pre-computed camera poses and instead leveraging feature-matching learned directly from data.

Generalizable Novel View Synthesis Novel View Synthesis

Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day

no code implementations4 Oct 2023 Yifan Jiang, Hao Tang, Jen-Hao Rick Chang, Liangchen Song, Zhangyang Wang, Liangliang Cao

Although the fidelity and generalizability are greatly improved, training such a powerful diffusion model requires a vast volume of training data and model parameters, resulting in a notoriously long time and high computational costs.

Image Generation Novel View Synthesis

ARN: A Comprehensive Framework and Benchmark for Analogical Reasoning on Narratives

no code implementations2 Oct 2023 Zhivar Sourati, Filip Ilievski, Pia Sommerauer, Yifan Jiang

This ability has been studied extensively in natural language processing (NLP) and in cognitive psychology.

POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference

1 code implementation25 May 2023 Zhiwen Fan, Panwang Pan, Peihao Wang, Yifan Jiang, Dejia Xu, Hanwen Jiang, Zhangyang Wang

To mitigate this issue, we propose a general paradigm for object pose estimation, called Promptable Object Pose Estimation (POPE).

Pose Estimation

The Weighted Möbius Score: A Unified Framework for Feature Attribution

1 code implementation16 May 2023 Yifan Jiang, Shane Steinert-Threlkeld

Feature attribution aims to explain the reasoning behind a black-box model's prediction by identifying the impact of each feature on the prediction.

Sentiment Analysis

Transferring Procedural Knowledge across Commonsense Tasks

1 code implementation26 Apr 2023 Yifan Jiang, Filip Ilievski, Kaixin Ma

Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories.

Story Completion

Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models

1 code implementation NeurIPS 2023 Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, Mingyuan Zhou

Patch Diffusion meanwhile improves the performance of diffusion models trained on relatively small datasets, $e. g.$, as few as 5, 000 images to train from scratch.

PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor

1 code implementation30 Mar 2023 Vidit Goel, Elia Peruzzo, Yifan Jiang, Dejia Xu, Xingqian Xu, Nicu Sebe, Trevor Darrell, Zhangyang Wang, Humphrey Shi

We propose \textbf{PAIR} Diffusion, a generic framework that can enable a diffusion model to control the structure and appearance properties of each object in the image.

Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition

no code implementations26 Feb 2023 Yifan Jiang, Han Chen, Hanseok Ko

In this paper, we introduce a novel data augmentation method for skeleton-based action recognition tasks, which can effectively generate high-quality and diverse sequential actions.

Action Recognition Data Augmentation +2

NeuralLift-360: Lifting an In-the-Wild 2D Photo to a 3D Object With 360deg Views

no code implementations CVPR 2023 Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, Zhangyang Wang

In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360deg views that corresponds well with the given reference image.

Denoising Depth Estimation

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views

1 code implementation29 Nov 2022 Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, Zhangyang Wang

In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360{\deg} views that correspond well with the given reference image.

3D Reconstruction Image to 3D +3

Signal Processing for Implicit Neural Representations

no code implementations17 Oct 2022 Dejia Xu, Peihao Wang, Yifan Jiang, Zhiwen Fan, Zhangyang Wang

We answer this question by proposing an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.

Deblurring Denoising +1

Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition

no code implementations10 Oct 2022 Han Chen, Yifan Jiang, Hanseok Ko

Graph convolutional networks (GCNs), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition.

Action Recognition Skeleton Based Action Recognition +1

NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes

1 code implementation19 Sep 2022 Zhiwen Fan, Peihao Wang, Yifan Jiang, Xinyu Gong, Dejia Xu, Zhangyang Wang

Our framework, called NeRF with Self-supervised Object Segmentation NeRF-SOS, couples object segmentation and neural radiance field to segment objects in any view within a scene.

Segmentation Self-Supervised Learning +1

Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN

1 code implementation27 Apr 2022 Qiucheng Wu, Yifan Jiang, Junru Wu, Kai Wang, Gong Zhang, Humphrey Shi, Zhangyang Wang, Shiyu Chang

To study the motion features in the latent space of StyleGAN, in this paper, we hypothesize and demonstrate that a series of meaningful, natural, and versatile small, local movements (referred to as "micromotion", such as expression, head movement, and aging effect) can be represented in low-rank spaces extracted from the latent space of a conventionally pre-trained StyleGAN-v2 model for face generation, with the guidance of proper "anchors" in the form of either short text or video clips.

Disentanglement Face Generation

Unified Implicit Neural Stylization

1 code implementation5 Apr 2022 Zhiwen Fan, Yifan Jiang, Peihao Wang, Xinyu Gong, Dejia Xu, Zhangyang Wang

Representing visual signals by implicit representation (e. g., a coordinate based deep network) has prevailed among many vision tasks.

Neural Stylization Novel View Synthesis

SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image

1 code implementation2 Apr 2022 Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Humphrey Shi, Zhangyang Wang

Despite the rapid development of Neural Radiance Field (NeRF), the necessity of dense covers largely prohibits its wider applications.

Novel View Synthesis

VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer

no code implementations17 Jan 2022 Mengshu Sun, Haoyu Ma, Guoliang Kang, Yifan Jiang, Tianlong Chen, Xiaolong Ma, Zhangyang Wang, Yanzhi Wang

To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate.


Fast and High-Quality Image Denoising via Malleable Convolutions

no code implementations2 Jan 2022 Yifan Jiang, Bartlomiej Wronski, Ben Mildenhall, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue

These spatially-varying kernels are produced by an efficient predictor network running on a downsampled input, making them much more efficient to compute than per-pixel kernels produced by a full-resolution image, and also enlarging the network's receptive field compared with static kernels.

Image Denoising Image Restoration +1

Action Recognition with Domain Invariant Features of Skeleton Image

no code implementations19 Nov 2021 Han Chen, Yifan Jiang, Hanseok Ko

Due to the fast processing-speed and robustness it can achieve, skeleton-based action recognition has recently received the attention of the computer vision community.

Action Recognition Skeleton Based Action Recognition

A Teacher-Student Framework with Fourier Augmentation for COVID-19 Infection Segmentation in CT Images

no code implementations13 Oct 2021 Han Chen, Yifan Jiang, Hanseok Ko, Murray Loew

Automatic segmentation of infected regions in computed tomography (CT) images is necessary for the initial diagnosis of COVID-19.

Computed Tomography (CT) Segmentation

Lottery Image Prior

no code implementations29 Sep 2021 Qiming Wu, Xiaohan Chen, Yifan Jiang, Pan Zhou, Zhangyang Wang

Drawing inspirations from the recently prosperous research on lottery ticket hypothesis (LTH), we conjecture and study a novel “lottery image prior” (LIP), stated as: given an (untrained or trained) DNN-based image prior, it will have a sparse subnetwork that can be training in isolation, to match the original DNN’s performance when being applied as a prior to various image inverse problems.

Compressive Sensing Image Reconstruction +1

SSH: A Self-Supervised Framework for Image Harmonization

1 code implementation ICCV 2021 Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang

Image harmonization aims to improve the quality of image compositing by matching the "appearance" (\eg, color tone, brightness and contrast) between foreground and background images.

Benchmarking Data Augmentation +1

CERL: A Unified Optimization Framework for Light Enhancement with Realistic Noise

1 code implementation1 Aug 2021 Zeyuan Chen, Yifan Jiang, Dong Liu, Zhangyang Wang

We present \underline{C}oordinated \underline{E}nhancement for \underline{R}eal-world \underline{L}ow-light Noisy Images (CERL), that seamlessly integrates light enhancement and noise suppression parts into a unified and physics-grounded optimization framework.


IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers

no code implementations NeurIPS 2021 Bowen Pan, Rameswar Panda, Yifan Jiang, Zhangyang Wang, Rogerio Feris, Aude Oliva

The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.

InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks

1 code implementation22 Apr 2021 Yonggan Fu, Zhongzhi Yu, Yongan Zhang, Yifan Jiang, Chaojian Li, Yongyuan Liang, Mingchao Jiang, Zhangyang Wang, Yingyan Lin

The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT) devices has motivated a tremendous demand for automated solutions to enable fast development and deployment of efficient (1) DNNs equipped with instantaneous accuracy-efficiency trade-off capability to accommodate the time-varying resources at IoT devices and (2) dataflows to optimize DNNs' execution efficiency on different devices.

TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up

10 code implementations NeurIPS 2021 Yifan Jiang, Shiyu Chang, Zhangyang Wang

Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures.

Data Augmentation Image Generation

Few-shot Learning for CT Scan based COVID-19 Diagnosis

no code implementations1 Feb 2021 Yifan Jiang, Han Chen, David K. Han, Hanseok Ko

To compensate for the sparseness of labeled data, the proposed method utilizes a large amount of synthetic COVID-19 CT images and adjusts the networks from the source domain (synthetic data) to the target domain (real data) with a cross-domain training mechanism.

Computed Tomography (CT) COVID-19 Diagnosis +2

Unsupervised domain adaptation based COVID-19 CT infection segmentation network

no code implementations23 Nov 2020 Han Chen, Yifan Jiang, Murray Loew, Hanseok Ko

In this paper, we propose an unsupervised domain adaptation based segmentation network to improve the segmentation performance of the infection areas in COVID-19 CT images.

Computed Tomography (CT) Segmentation +1

COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network

no code implementations29 Jul 2020 Yifan Jiang, Han Chen, Murray Loew, Hanseok Ko

However, training a deep-learning model requires large volumes of data, and medical staff faces a high risk when collecting COVID-19 CT data due to the high infectivity of the disease.

Computed Tomography (CT) COVID-19 Diagnosis +2

EnlightenGAN: Deep Light Enhancement without Paired Supervision

8 code implementations17 Jun 2019 Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, Zhangyang Wang

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?

Image Restoration Low-Light Image Enhancement +1

StrokeNet: A Neural Painting Environment

no code implementations ICLR 2019 Ningyuan Zheng, Yifan Jiang, Dingjiang Huang

In this paper we try to address the discrete nature of software environment with an intermediate, differentiable simulation.

Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond

2 code implementations5 Apr 2018 Xi Ouyang, Yu Cheng, Yifan Jiang, Chun-Liang Li, Pan Zhou

The results show that our framework can smoothly synthesize pedestrians on background images of variations and different levels of details.

Pedestrian Detection Scene Text Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.