Search Results for author: Tao Yu

Found 78 papers, 34 papers with code

RobustFusion: Human Volumetric Capture with Data-driven Visual Cues using a RGBD Camera

no code implementations ECCV 2020 Zhuo Su, Lan Xu, Zerong Zheng, Tao Yu, Yebin Liu, Lu Fang

To enable robust tracking, we embrace both the initial model and the various visual cues into a novel performance capture scheme with hybrid motion optimization and semantic volumetric fusion, which can successfully capture challenging human motions under the monocular setting without pre-scanned detailed template and owns the reinitialization ability to recover from tracking failures and the disappear-reoccur scenarios.

Effective Fine-Tuning Methods for Cross-lingual Adaptation

no code implementations EMNLP 2021 Tao Yu, Shafiq Joty

In this work, we propose a novel fine-tuning method based on co-training that aims to learn more generalized semantic equivalences as a complementary to multilingual language modeling using the unlabeled data in the target language.

Contrastive Learning Language Modelling +1

SwinIQA: Learned Swin Distance for Compressed Image Quality Assessment

no code implementations9 May 2022 Jianzhao Liu, Xin Li, Yanding Peng, Tao Yu, Zhibo Chen

In this paper, we design a full-reference image quality assessment metric SwinIQA to measure the perceptual quality of compressed images in a learned Swin distance space.

Image Compression Image Quality Assessment

GIMO: Gaze-Informed Human Motion Prediction in Context

no code implementations20 Apr 2022 Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, Karen Liu, Leonidas J. Guibas

Our network achieves the top performance in human motion prediction on the proposed dataset, thanks to the intent information from the gaze and the denoised gaze feature modulated by the motion.

Human motion prediction motion prediction

ProbNVS: Fast Novel View Synthesis with Learned Probability-Guided Sampling

no code implementations7 Apr 2022 Yuemei Zhou, Tao Yu, Zerong Zheng, Ying Fu, Yebin Liu

Existing state-of-the-art novel view synthesis methods rely on either fairly accurate 3D geometry estimation or sampling of the entire space for neural volumetric rendering, which limit the overall efficiency.

Novel View Synthesis

Structured Local Radiance Fields for Human Avatar Modeling

no code implementations28 Mar 2022 Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yandong Guo, Yebin Liu

These local radiance fields not only leverage the flexibility of implicit representation in shape and appearance modeling, but also factorize cloth deformations into skeleton motions, node residual translations and the dynamic detail variations inside each individual radiance field.

FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset

1 code implementation26 Mar 2022 Lizhen Wang, ZhiYuan Chen, Tao Yu, Chenguang Ma, Liang Li, Yebin Liu

In the coarse module, we generate a base parametric model from large-scale RGB-D images, which is able to predict accurate rough 3D face models in different genders, ages, etc.

3D Face Reconstruction Face Model

Interacting Attention Graph for Single Image Two-Hand Reconstruction

1 code implementation17 Mar 2022 Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu, Feng Chen, Tao Yu, Yebin Liu

To solve occlusion and interaction challenges of two-hand reconstruction, we introduce two novel attention based modules in each upsampling step of the original GCN.

In-Context Learning for Few-Shot Dialogue State Tracking

1 code implementation16 Mar 2022 Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

In this work, we propose an in-context (IC) learning framework for zero-shot and few-shot learning dialogue state tracking (DST), where a large pretrained language model (LM) takes a test instance and a few exemplars as input, and directly decodes the dialogue state without any parameter updates.

Dialogue State Tracking Few-Shot Learning +2

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

1 code implementation16 Feb 2022 Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs).

Knowledge Distillation Natural Language Inference +4

HyLa: Hyperbolic Laplacian Features For Graph Learning

no code implementations14 Feb 2022 Tao Yu, Christopher De Sa

Due to its geometric properties, hyperbolic space can support high-fidelity embeddings of tree- and graph-structured data.

Graph Learning Node Classification +1

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

no code implementations10 Feb 2022 Tao Yu, Yichi Zhang, Zhiru Zhang, Christopher De Sa

Using representation theory, we characterize which similarity matrices can be "expressed" by finite group VSA hypervectors, and we show how these VSAs can be constructed.

Mask-based Latent Reconstruction for Reinforcement Learning

no code implementations28 Jan 2022 Tao Yu, Zhizheng Zhang, Cuiling Lan, Zhibo Chen, Yan Lu

For deep reinforcement learning (RL) from pixels, learning effective state representations is crucial for achieving high performance.

reinforcement-learning Representation Learning

HDhuman: High-quality Human Performance Capture with Sparse Views

no code implementations20 Jan 2022 Tiansong Zhou, Tao Yu, Ruizhi Shao, Kun Li

To this end, the proposed HDhuman uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network that uses geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering.

Neural Rendering Surface Reconstruction

Representing Hyperbolic Space Accurately using Multi-Component Floats

no code implementations NeurIPS 2021 Tao Yu, Christopher M. De Sa

Hyperbolic space is particularly useful for embedding data with hierarchical structure; however, representing hyperbolic space with ordinary floating-point numbers greatly affects the performance due to its \emph{ineluctable} numerical errors.

S-DCCRN: Super Wide Band DCCRN with learnable complex feature for speech enhancement

no code implementations16 Nov 2021 Shubo Lv, Yihui Fu, Mengtao Xing, Jiayao Sun, Lei Xie, Jun Huang, Yannan Wang, Tao Yu

In speech enhancement, complex neural network has shown promising performance due to their effectiveness in processing complex-valued spectrum.

Denoising Speech Denoising +1

Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions

no code implementations15 Sep 2021 Naihao Deng, Shuaichen Chang, Peng Shi, Tao Yu, Rui Zhang

Existing text-to-SQL research only considers complete questions as the input, but lay-users might strive to formulate a complete question.


An Exploratory Study on Long Dialogue Summarization: What Works and What's Next

1 code implementation10 Sep 2021 Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev

Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series.

SummerTime: Text Summarization Toolkit for Non-experts

1 code implementation EMNLP (ACL) 2021 Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng, Yusen Zhang, Tao Yu, Ahmed Hassan Awadallah, Dragomir Radev

We also provide explanations for models and evaluation metrics to help users understand the model behaviors and select models that best suit their needs.

Document Summarization Multi-Document Summarization

Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras

no code implementations ICCV 2021 Yuxiang Zhang, Zhe Li, Liang An, Mengcheng Li, Tao Yu, Yebin Liu

Overall, we propose the first light-weight total capture system and achieves fast, robust and accurate multi-person total motion capture performance.


Logic-Consistency Text Generation from Semantic Parses

1 code implementation Findings (ACL) 2021 Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi, Tao Yu, Rui Zhang

Text generation from semantic parses is to generate textual descriptions for formal representation inputs such as logic forms and SQL queries.

Text Generation

End-to-End Cross-Domain Text-to-SQL Semantic Parsing with Auxiliary Task

no code implementations17 Jun 2021 Peng Shi, Tao Yu, Patrick Ng, Zhiguo Wang

Furthermore, we propose two value filling methods to build the bridge from the existing zero-shot semantic parsers to real-world applications, considering most of the existing parsers ignore the values filling in the synthesized SQL.

Semantic Parsing Text-To-Sql

DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering

no code implementations7 Jun 2021 Ruizhi Shao, Hongwen Zhang, He Zhang, Mingjia Chen, YanPei Cao, Tao Yu, Yebin Liu

We introduce DoubleField, a novel framework combining the merits of both surface field and radiance field for high-fidelity human reconstruction and rendering.

Transfer Learning

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

1 code implementation NAACL 2021 Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev

As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed.

Meeting Summarization

POSEFusion: Pose-guided Selective Fusion for Single-view Human Volumetric Capture

no code implementations CVPR 2021 Zhe Li, Tao Yu, Zerong Zheng, Kaiwen Guo, Yebin Liu

By contributing a novel reconstruction framework which contains pose-guided keyframe selection and robust implicit surface fusion, our method fully utilizes the advantages of both tracking-based methods and tracking-free inference methods, and finally enables the high-fidelity reconstruction of dynamic surface details even in the invisible regions.

3D Reconstruction

Local Patch AutoAugment with Multi-Agent Collaboration

2 code implementations20 Mar 2021 Shiqi Lin, Tao Yu, Ruoyu Feng, Xin Li, Xin Jin, Zhibo Chen

We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image.

Data Augmentation Fine-Grained Image Recognition +2

Nematicity Arising from a Chiral Superconducting Ground State in Magic-Angle Twisted Bilayer Graphene under In-Plane Magnetic Fields

no code implementations5 Jan 2021 Tao Yu, Dante M. Kennes, Angel Rubio, Michael A. Sentef

Recent measurements of the resistivity in magic-angle twisted bilayer graphene near the superconducting transition temperature show two-fold anisotropy, or nematicity, when changing the direction of an in-plane magnetic field [Cao \textit{et al.}, Science \textbf{372}, 264 (2021)].


SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing

no code implementations NeurIPS Workshop CAP 2020 Tao Yu, Rui Zhang, Alex Polozov, Christopher Meek, Ahmed Hassan Awadallah

Conversational Semantic Parsing (CSP) is the task of converting a sequence of natural language queries to formal language (e. g., SQL, SPARQL) that can be executed against a structured ontology (e. g. databases, knowledge bases).

Ranked #3 on Multi-domain Dialogue State Tracking on MULTIWOZ 2.1 (using extra training data)

Language Modelling Multi-domain Dialogue State Tracking +1

Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution

no code implementations11 Dec 2020 Xin Li, Xin Jin, Tao Yu, Yingxue Pang, Simeng Sun, Zhizheng Zhang, Zhibo Chen

Traditional single image super-resolution (SISR) methods that focus on solving single and uniform degradation (i. e., bicubic down-sampling), typically suffer from poor performance when applied into real-world low-resolution (LR) images due to the complicated realistic degradations.

Image Super-Resolution

Spin-Wave Doppler Shift by Magnon Drag in Magnetic Insulators

no code implementations30 Nov 2020 Tao Yu, Chen Wang, Michael A. Sentef, Gerrit E. W. Bauer

The Doppler shift of the quasiparticle dispersion by charge currents is responsible for the critical supercurrents in superconductors and instabilities of the magnetic ground state of metallic ferromagnets.

Mesoscale and Nanoscale Physics

Deep Implicit Templates for 3D Shape Representation

1 code implementation CVPR 2021 Zerong Zheng, Tao Yu, Qionghai Dai, Yebin Liu

Deep implicit functions (DIFs), as a kind of 3D shape representation, are becoming more and more popular in the 3D vision community due to their compactness and strong representation power.

3D Shape Representation

DeepCloth: Neural Garment Representation for Shape and Style Editing

no code implementations30 Nov 2020 Zhaoqi Su, Tao Yu, Yangang Wang, Yebin Liu

In this work, we introduce, DeepCloth, a unified framework for garment representation, reconstruction, animation and editing.

Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping

no code implementations30 Nov 2020 Xiaochen Zhao, Zerong Zheng, Chaonan Ji, Zhenyi Liu, Siyou Lin, Tao Yu, Jinli Suo, Yebin Liu

We introduce VERTEX, an effective solution to recover 3D shape and intrinsic texture of vehicles from uncalibrated monocular input in real-world street environments.

Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL

1 code implementation23 Oct 2020 Yusen Zhang, Xiangyu Dong, Shuaichen Chang, Tao Yu, Peng Shi, Rui Zhang

Neural models have achieved significant results on the text-to-SQL task, in which most current work assumes all the input questions are legal and generates a SQL query for any input.


Semantic Evaluation for Text-to-SQL with Distilled Test Suites

3 code implementations EMNLP 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models.


GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

1 code implementation ICLR 2021 Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, Richard Socher, Caiming Xiong

We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data.

Language Modelling Masked Language Modeling +2

NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image

1 code implementation ECCV 2020 Lizhen Wang, Xiaochen Zhao, Tao Yu, Songtao Wang, Yebin Liu

We propose NormalGAN, a fast adversarial learning-based method to reconstruct the complete and detailed 3D human from a single RGB-D image.

3D Human Reconstruction Denoising

PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction

no code implementations8 Jul 2020 Zerong Zheng, Tao Yu, Yebin Liu, Qionghai Dai

To overcome the limitations of regular 3D representations, we propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function.

3D Human Reconstruction Camera Calibration

Semantic Evaluation for Text-to-SQL with Distilled Test Suite

no code implementations2 Jul 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models, where a predicted query is semantically correct if its denotation is the same as the gold for every possible database.

Semantic Parsing Text-To-Sql

MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera

no code implementations13 Apr 2020 Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu

We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning.


Robust 3D Self-portraits in Seconds

no code implementations CVPR 2020 Zhe Li, Tao Yu, Chuanyu Pan, Zerong Zheng, Yebin Liu

In this paper, we propose an efficient method for robust 3D self-portraits using a single RGBD camera.

Salvaging Federated Learning by Local Adaptation

2 code implementations12 Feb 2020 Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov

First, we show that on standard tasks such as next-word prediction, many participants gain no benefit from FL because the federated model is less accurate on their data than the models they can train locally on their own.

Federated Learning Knowledge Distillation +1

Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models

2 code implementations NeurIPS 2019 Tao Yu, Christopher M. De Sa

Hyperbolic embeddings achieve excellent performance when embedding hierarchical data structures like synonym or type hierarchies, but they can be limited by numerical error when ordinary floating-point numbers are used to represent points in hyperbolic space.

Region Normalization for Image Inpainting

1 code implementation23 Nov 2019 Tao Yu, Zongyu Guo, Xin Jin, Shilin Wu, Zhibo Chen, Weiping Li, Zhizheng Zhang, Sen Liu

Most previous image inpainting methods apply FN in their networks without considering the impact of the corrupted regions of the input image on normalization, e. g. mean and variance shifts.

Image Inpainting

A New Defense Against Adversarial Images: Turning a Weakness into a Strength

1 code implementation NeurIPS 2019 Tao Yu, Shengyuan Hu, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger

Natural images are virtually surrounded by low-density misclassified regions that can be efficiently discovered by gradient-guided search --- enabling the generation of adversarial images.

Adversarial Defense

Progressive Image Inpainting with Full-Resolution Residual Network

2 code implementations24 Jul 2019 Zongyu Guo, Zhibo Chen, Tao Yu, Jiale Chen, Sen Liu

Recently, learning-based algorithms for image inpainting achieve remarkable progress dealing with squared or irregular holes.

Image Inpainting

SParC: Cross-Domain Semantic Parsing in Context

5 code implementations ACL 2019 Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev

The best model obtains an exact match accuracy of 20. 2% over all questions and less than10% over all interaction sequences, indicating that the cross-domain setting and the con-textual phenomena of the dataset present significant challenges for future research.

Semantic Parsing Text-To-Sql

DeepHuman: 3D Human Reconstruction from a Single Image

1 code implementation ICCV 2019 Zerong Zheng, Tao Yu, Yixuan Wei, Qionghai Dai, Yebin Liu

We propose DeepHuman, an image-guided volume-to-volume translation CNN for 3D human reconstruction from a single RGB image.

3D Human Reconstruction Pose Estimation +1

SimulCap : Single-View Human Performance Capture with Cloth Simulation

no code implementations CVPR 2019 Tao Yu, Zerong Zheng, Yuan Zhong, Jianhui Zhao, Qionghai Dai, Gerard Pons-Moll, Yebin Liu

This paper proposes a new method for live free-viewpoint human performance capture with dynamic details (e. g., cloth wrinkles) using a single RGBD camera.


Simplifying Graph Convolutional Networks

6 code implementations19 Feb 2019 Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr., Christopher Fifty, Tao Yu, Kilian Q. Weinberger

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations.

Ranked #2 on Text Classification on 20NEWS (using extra training data)

Graph Regression Image Classification +5

SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task

2 code implementations11 Oct 2018 Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, Dragomir Radev

In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and cross-domain text-to-SQL generation task.

Semantic Parsing Text-To-Sql

HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs

no code implementations ECCV 2018 Zerong Zheng, Tao Yu, Hao Li, Kaiwen Guo, Qionghai Dai, Lu Fang, Yebin Liu

We propose a light-weight and highly robust real-time human performance capture method based on a single depth camera and sparse inertial measurement units (IMUs).

Frame Surface Reconstruction

TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation

no code implementations NAACL 2018 Tao Yu, Zifan Li, Zilin Zhang, Rui Zhang, Dragomir Radev

Interacting with relational databases through natural language helps users of any background easily query and analyze a vast amount of data.

Slot Filling Text-To-Sql

DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor

no code implementations CVPR 2018 Tao Yu, Zerong Zheng, Kaiwen Guo, Jianhui Zhao, Qionghai Dai, Hao Li, Gerard Pons-Moll, Yebin Liu

We further propose a joint motion tracking method based on the double layer representation to enable robust and fast motion tracking performance.

Curvature-based Comparison of Two Neural Networks

no code implementations21 Jan 2018 Tao Yu, Huan Long, John E. Hopcroft

In this paper we show the similarities and differences of two deep neural networks by comparing the manifolds composed of activation vectors in each fully connected layer of them.

Preliminary theoretical troubleshooting in Variational Autoencoder

no code implementations ICLR 2018 Shiqi Liu, Qian Zhao, Xiangyong Cao, Deyu Meng, Zilu Ma, Tao Yu

This paper tries to preliminarily address VAE's intrinsic dimension, real factor, disentanglement and indicator issues theoretically in the idealistic situation and implementation issue practically through noise modeling perspective in the realistic case.


The Local Dimension of Deep Manifold

no code implementations5 Nov 2017 Mengxiao Zhang, Wangquan Wu, Yanren Zhang, Kun He, Tao Yu, Huan Long, John E. Hopcroft

Our results show that the dimensions of different categories are close to each other and decline quickly along the convolutional layers and fully connected layers.

BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera

no code implementations ICCV 2017 Tao Yu, Kaiwen Guo, Feng Xu, Yuan Dong, Zhaoqi Su, Jianhui Zhao, Jianguo Li, Qionghai Dai, Yebin Liu

To reduce the ambiguities of the non-rigid deformation parameterization on the surface graph nodes, we take advantage of the internal articulated motion prior for human performance and contribute a skeleton-embedded surface fusion (SSF) method.

Frame Surface Reconstruction

Leveraging Sparse and Dense Feature Combinations for Sentiment Classification

no code implementations13 Aug 2017 Tao Yu, Christopher Hidey, Owen Rambow, Kathleen McKeown

This model outperforms many deep learning models and achieves comparable results to other deep learning models with complex architectures on sentiment analysis datasets.

Classification General Classification +1

Cannot find the paper you are looking for? You can Submit a new open access paper.