Search Results for author: Tao Yu

Found 118 papers, 62 papers with code

RobustFusion: Human Volumetric Capture with Data-driven Visual Cues using a RGBD Camera

no code implementations ECCV 2020 Zhuo Su, Lan Xu, Zerong Zheng, Tao Yu, Yebin Liu, Lu Fang

To enable robust tracking, we embrace both the initial model and the various visual cues into a novel performance capture scheme with hybrid motion optimization and semantic volumetric fusion, which can successfully capture challenging human motions under the monocular setting without pre-scanned detailed template and owns the reinitialization ability to recover from tracking failures and the disappear-reoccur scenarios.

4D reconstruction

Effective Fine-Tuning Methods for Cross-lingual Adaptation

no code implementations EMNLP 2021 Tao Yu, Shafiq Joty

In this work, we propose a novel fine-tuning method based on co-training that aims to learn more generalized semantic equivalences as a complementary to multilingual language modeling using the unlabeled data in the target language.

Contrastive Learning Language Modelling +1

Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning

no code implementations20 Sep 2023 Tianbao Xie, Siheng Zhao, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu

Unlike inverse RL and recent work that uses LLMs to write sparse reward codes, Text2Reward produces interpretable, free-form dense reward codes that cover a wide range of tasks, utilize existing packages, and allow iterative refinement with human feedback.

reinforcement-learning Reinforcement Learning (RL)

Dynamic Spectrum Mixer for Visual Recognition

no code implementations13 Sep 2023 Zhiqiang Hu, Tao Yu

Recently, MLP-based vision backbones have achieved promising performance in several visual recognition tasks.

Image Classification object-detection +2

ImmersiveNeRF: Hybrid Radiance Fields for Unbounded Immersive Light Field Reconstruction

no code implementations4 Sep 2023 Xiaohang Yu, Haoxiang Wang, Yuqi Han, Lei Yang, Tao Yu, Qionghai Dai

This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation.

Empowering LLM to use Smartphone for Intelligent Task Automation

no code implementations29 Aug 2023 Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, Yunxin Liu

Mobile task automation is an attractive technique that aims to enable voice-based hands-free user interaction with smartphones.

Language Modelling

DIALGEN: Collaborative Human-LM Generated Dialogues for Improved Understanding of Human-Human Conversations

no code implementations13 Jul 2023 Bo-Ru Lu, Nikita Haduong, Chia-Hsuan Lee, Zeqiu Wu, Hao Cheng, Paul Koester, Jean Utke, Tao Yu, Noah A. Smith, Mari Ostendorf

Applications that could benefit from automatic understanding of human-human conversations often come with challenges associated with private information in real-world data such as call center or clinical conversations.

Dialogue Generation Dialogue State Tracking +1

Fast Segment Anything

1 code implementation21 Jun 2023 Xu Zhao, Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, Jinqiao Wang

In this paper, we propose a speed-up alternative method for this fundamental task with comparable performance.

Edge Detection Image Segmentation +4

Coneheads: Hierarchy Aware Attention

1 code implementation1 Jun 2023 Albert Tseng, Tao Yu, Toni J. B. Liu, Christopher De Sa

These networks rely heavily on the dot product attention operator, which computes the similarity between two points by taking their inner product.

Shadow Cones: Unveiling Partial Orders in Hyperbolic Space

1 code implementation24 May 2023 Tao Yu, Toni J. B. Liu, Albert Tseng, Christopher De Sa

Our findings indicate that shadow cones offer an innovative, general approach to geometrically encode partial orders, enabling better representation and analysis of datasets with hierarchical structures.

Generating Data for Symbolic Language with Large Language Models

1 code implementation23 May 2023 Jiacheng Ye, Chengzu Li, Lingpeng Kong, Tao Yu

However, such an approach has primarily been applied to natural language tasks and has not yet been explored for symbolic language tasks with complex structured outputs (e. g., semantic parsing and code generation).

Code Generation Semantic Parsing

Hybrid AHS: A Hybrid of Kalman Filter and Deep Learning for Acoustic Howling Suppression

no code implementations4 May 2023 Hao Zhang, Meng Yu, Yuzhong Wu, Tao Yu, Dong Yu

During offline training, a pre-processed signal obtained from the Kalman filter and an ideal microphone signal generated via teacher-forced training strategy are used to train the deep neural network (DNN).

StyleAvatar: Real-time Photo-realistic Portrait Avatar from a Single Video

1 code implementation1 May 2023 Lizhen Wang, Xiaochen Zhao, Jingxiang Sun, Yuxiang Zhang, Hongwen Zhang, Tao Yu, Yebin Liu

Results and experiments demonstrate the superiority of our method in terms of image quality, full portrait video generation, and real-time re-animation compared to existing facial reenactment methods.

Face Reenactment Translation +1

Super-NeRF: View-consistent Detail Generation for NeRF super-resolution

no code implementations26 Apr 2023 Yuqi Han, Tao Yu, Xiaohang Yu, Yuwang Wang, Qionghai Dai

Given multi-view low-resolution images, Super-NeRF constructs a consistency-controlling super-resolution module to generate view-consistent high-resolution details for NeRF.

Image Super-Resolution

The Seven Worlds and Experiences of the Wireless Metaverse: Challenges and Opportunities

no code implementations20 Apr 2023 Omar Hashash, Christina Chaccour, Walid Saad, Tao Yu, Kei Sakaguchi, Merouane Debbah

We then articulate how these experiences bring forth interactions between diverse metaverse constituents, namely, a) humans and avatars and b) connected intelligence systems and their digital twins (DTs).

Inpaint Anything: Segment Anything Meets Image Inpainting

1 code implementation13 Apr 2023 Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, Zhibo Chen

We are also very willing to help everyone share and promote new projects based on our Inpaint Anything (IA).

Image Inpainting

Hi Sheldon! Creating Deep Personalized Characters from TV Shows

no code implementations9 Apr 2023 Meidai Xuanyuan, Yuwang Wang, Honglei Guo, Xiao Ma, Yuchen Guo, Tao Yu, Qionghai Dai

To support this novel task, we further collect a character centric multimodal dialogue dataset, named Deep Personalized Character Dataset (DPCD), from TV shows.

ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection

1 code implementation CVPR 2023 Yongqi An, Xu Zhao, Tao Yu, Haiyun Guo, Chaoyang Zhao, Ming Tang, Jinqiao Wang

However, previous unsupervised deep learning BGS algorithms perform poorly in sophisticated scenarios such as shadows or night lights, and they cannot detect objects outside the pre-defined categories.

Foreground Segmentation object-detection +1

Automated Self-Supervised Learning for Recommendation

2 code implementations14 Mar 2023 Lianghao Xia, Chao Huang, Chunzhen Huang, Kangyi Lin, Tao Yu, Ben Kao

This does not generalize across different datasets and downstream recommendation tasks, which is difficult to be adaptive for data augmentation and robust to noise perturbation.

Collaborative Filtering Contrastive Learning +2

Compositional Exemplars for In-context Learning

1 code implementation11 Feb 2023 Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, Lingpeng Kong

The performance of ICL is highly dominated by the quality of the selected in-context examples.

Code Generation Contrastive Learning +5

Batch Prompting: Efficient Inference with Large Language Model APIs

1 code implementation19 Jan 2023 Zhoujun Cheng, Jungo Kasai, Tao Yu

We extensively validate the effectiveness of batch prompting on ten datasets across commonsense QA, arithmetic reasoning, and NLI/NLU: batch prompting significantly~(up to $5\times$ with six samples in batch) reduces the LLM (Codex) inference token and time costs while achieving better or comparable performance.

Arithmetic Reasoning Language Modelling +1

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

2 code implementations19 Dec 2022 Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu

Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets.

Information Retrieval Learning Word Embeddings +3

Realization Scheme for Visual Cryptography with Computer-generated Holograms

no code implementations10 Dec 2022 Tao Yu, Jinge Ma, Guilin Li, Dongyu Yang, Rui Ma, Yishi Shi

This method can expand the application range of visual cryptography and further increase the security of visual cryptography.

Coder Reviewer Reranking for Code Generation

1 code implementation29 Nov 2022 Tianyi Zhang, Tao Yu, Tatsunori B. Hashimoto, Mike Lewis, Wen-tau Yih, Daniel Fried, Sida I. Wang

Sampling diverse programs from a code language model and reranking with model likelihood is a popular method for code generation but it is prone to preferring degenerate solutions.

Code Generation Language Modelling

DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation

1 code implementation18 Nov 2022 Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, Tao Yu

We introduce DS-1000, a code generation benchmark with a thousand data science problems spanning seven Python libraries, such as NumPy and Pandas.

Code Generation Memorization

Task Residual for Tuning Vision-Language Models

1 code implementation CVPR 2023 Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, Xinchao Wang

Large-scale vision-language models (VLMs) pre-trained on billion-level data have learned general visual representations and broad visual concepts.

Transfer Learning

ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback

1 code implementation22 Oct 2022 Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

To improve the quality of dataset synthesis, we propose a progressive zero-shot dataset generation framework, ProGen, which leverages the feedback from the task-specific model to guide the generation of new training data via in-context examples.

Informativeness text-classification +2

Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play

1 code implementation21 Oct 2022 Qi Liu, Zihuiwen Ye, Tao Yu, Phil Blunsom, Linfeng Song

We first design a SQL-to-text model conditioned on a sampled goal query, which represents a user's intent, that then converses with a text-to-SQL semantic parser to generate new interactions.

Domain Generalization SQL-to-Text +1

Binding Language Models in Symbolic Languages

1 code implementation6 Oct 2022 Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

We propose Binder, a training-free neural-symbolic framework that maps the task input to a program, which (1) allows binding a unified API of language model (LM) functionalities to a programming language (e. g., SQL, Python) to extend its grammar coverage and thus tackle more diverse questions, (2) adopts an LM as both the program parser and the underlying model called by the API during execution, and (3) requires only a few in-context exemplar annotations.

Language Modelling Semantic Parsing +1

NL2INTERFACE: Interactive Visualization Interface Generation from Natural Language Queries

no code implementations19 Sep 2022 Yiru Chen, Ryan Li, Austin Mac, Tianbao Xie, Tao Yu, Eugene Wu

We develop NL2INTERFACE to explore the potential of generating usable interactive multi-visualization interfaces from natural language queries.

Natural Language Queries

Selective Annotation Makes Language Models Better Few-Shot Learners

1 code implementation5 Sep 2022 Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

Departing from recent in-context learning methods, we formulate an annotation-efficient, two-step framework: selective annotation that chooses a pool of examples to annotate from unlabeled data in advance, followed by prompt retrieval that retrieves task examples from the annotated pool at test time.

Code Generation Retrieval

MCTensor: A High-Precision Deep Learning Library with Multi-Component Floating-Point

1 code implementation18 Jul 2022 Tao Yu, Wentao Guo, Jianan Canal Li, Tiancheng Yuan, Christopher De Sa

In this paper, we introduce MCTensor, a library based on PyTorch for providing general-purpose and high-precision arithmetic for DL training.

Geometry-aware Single-image Full-body Human Relighting

no code implementations11 Jul 2022 Chaonan Ji, Tao Yu, Kaiwen Guo, Jingxin Liu, Yebin Liu

For the relighting, we introduce a ray tracing-based per-pixel lighting representation that explicitly models high-frequency shadows and propose a learning-based shading refinement module to restore realistic shadows (including hard cast shadows) from the ray-traced shading maps.

Disentanglement Neural Rendering

Design and Analysis of Robust Resilient Diffusion over Multi-Task Networks Against Byzantine Attacks

no code implementations25 Jun 2022 Tao Yu, Rodrigo C. de Lamare, Yi Yu

This paper studies distributed diffusion adaptation over clustered multi-task networks in the presence of impulsive interferences and Byzantine attacks.

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

1 code implementation9 Jun 2022 Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, ZiRui Wang, Ziyi Wu

BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.

Common Sense Reasoning Memorization

SwinIQA: Learned Swin Distance for Compressed Image Quality Assessment

1 code implementation9 May 2022 Jianzhao Liu, Xin Li, Yanding Peng, Tao Yu, Zhibo Chen

In this paper, we design a full-reference image quality assessment metric SwinIQA to measure the perceptual quality of compressed images in a learned Swin distance space.

Compressed Image Quality Assessment Image Compression +1

GIMO: Gaze-Informed Human Motion Prediction in Context

1 code implementation20 Apr 2022 Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C. Karen Liu, Leonidas J. Guibas

We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.

Human motion prediction motion prediction

ProbNVS: Fast Novel View Synthesis with Learned Probability-Guided Sampling

no code implementations7 Apr 2022 Yuemei Zhou, Tao Yu, Zerong Zheng, Ying Fu, Yebin Liu

Existing state-of-the-art novel view synthesis methods rely on either fairly accurate 3D geometry estimation or sampling of the entire space for neural volumetric rendering, which limit the overall efficiency.

Novel View Synthesis

Structured Local Radiance Fields for Human Avatar Modeling

no code implementations CVPR 2022 Zerong Zheng, Han Huang, Tao Yu, Hongwen Zhang, Yandong Guo, Yebin Liu

These local radiance fields not only leverage the flexibility of implicit representation in shape and appearance modeling, but also factorize cloth deformations into skeleton motions, node residual translations and the dynamic detail variations inside each individual radiance field.

FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset

1 code implementation CVPR 2022 Lizhen Wang, ZhiYuan Chen, Tao Yu, Chenguang Ma, Liang Li, Yebin Liu

In the coarse module, we generate a base parametric model from large-scale RGB-D images, which is able to predict accurate rough 3D face models in different genders, ages, etc.

3D Face Reconstruction Face Model

Interacting Attention Graph for Single Image Two-Hand Reconstruction

1 code implementation CVPR 2022 Mengcheng Li, Liang An, Hongwen Zhang, Lianpeng Wu, Feng Chen, Tao Yu, Yebin Liu

To solve occlusion and interaction challenges of two-hand reconstruction, we introduce two novel attention based modules in each upsampling step of the original GCN.

Vocal Bursts Valence Prediction

In-Context Learning for Few-Shot Dialogue State Tracking

1 code implementation16 Mar 2022 Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

In this work, we propose an in-context learning (ICL) framework for zero-shot and few-shot learning DST, where a large pre-trained language model (LM) takes a test instance and a few exemplars as input, and directly decodes the dialogue state without any parameter updates.

Dialogue State Tracking Few-Shot Learning +2

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

2 code implementations16 Feb 2022 Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs).

Knowledge Distillation Natural Language Inference +5

Random Laplacian Features for Learning with Hyperbolic Space

1 code implementation14 Feb 2022 Tao Yu, Christopher De Sa

Due to its geometric properties, hyperbolic space can support high-fidelity embeddings of tree- and graph-structured data, upon which various hyperbolic networks have been developed.

Graph Learning Node Classification +2

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

1 code implementation10 Feb 2022 Tao Yu, Yichi Zhang, Zhiru Zhang, Christopher De Sa

Using representation theory, we characterize which similarity matrices can be "expressed" by finite group VSA hypervectors, and we show how these VSAs can be constructed.

Mask-based Latent Reconstruction for Reinforcement Learning

1 code implementation28 Jan 2022 Tao Yu, Zhizheng Zhang, Cuiling Lan, Yan Lu, Zhibo Chen

For deep reinforcement learning (RL) from pixels, learning effective state representations is crucial for achieving high performance.

reinforcement-learning Reinforcement Learning (RL) +1

HDhuman: High-quality Human Performance Capture with Sparse Views

no code implementations20 Jan 2022 Tiansong Zhou, Tao Yu, Ruizhi Shao, Kun Li

To this end, the proposed HDhuman uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network that uses geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering.

Neural Rendering Surface Reconstruction +1

HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars

no code implementations19 Dec 2021 Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, Matthias Zwicker

To handle complicated motions (e. g., self-occlusions), we then leverage the encoded information on the UV manifold to construct a 3D volumetric representation based on a dynamic pose-conditioned neural radiance field.

Neural Rendering

Representing Hyperbolic Space Accurately using Multi-Component Floats

no code implementations NeurIPS 2021 Tao Yu, Christopher M. De Sa

Hyperbolic space is particularly useful for embedding data with hierarchical structure; however, representing hyperbolic space with ordinary floating-point numbers greatly affects the performance due to its \emph{ineluctable} numerical errors.

S-DCCRN: Super Wide Band DCCRN with learnable complex feature for speech enhancement

no code implementations16 Nov 2021 Shubo Lv, Yihui Fu, Mengtao Xing, Jiayao Sun, Lei Xie, Jun Huang, Yannan Wang, Tao Yu

In speech enhancement, complex neural network has shown promising performance due to their effectiveness in processing complex-valued spectrum.

Denoising Speech Denoising +1

Prefix-to-SQL: Text-to-SQL Generation from Incomplete User Questions

no code implementations15 Sep 2021 Naihao Deng, Shuaichen Chang, Peng Shi, Tao Yu, Rui Zhang

Existing text-to-SQL research only considers complete questions as the input, but lay-users might strive to formulate a complete question.


An Exploratory Study on Long Dialogue Summarization: What Works and What's Next

1 code implementation10 Sep 2021 Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev

Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series.


SummerTime: Text Summarization Toolkit for Non-experts

1 code implementation EMNLP (ACL) 2021 Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng, Yusen Zhang, Tao Yu, Ahmed Hassan Awadallah, Dragomir Radev

We also provide explanations for models and evaluation metrics to help users understand the model behaviors and select models that best suit their needs.

Document Summarization Multi-Document Summarization

Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras

no code implementations ICCV 2021 Yuxiang Zhang, Zhe Li, Liang An, Mengcheng Li, Tao Yu, Yebin Liu

Overall, we propose the first light-weight total capture system and achieves fast, robust and accurate multi-person total motion capture performance.

3D Multi-Person Pose Estimation

Logic-Consistency Text Generation from Semantic Parses

1 code implementation Findings (ACL) 2021 Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi, Tao Yu, Rui Zhang

Text generation from semantic parses is to generate textual descriptions for formal representation inputs such as logic forms and SQL queries.

Text Generation

End-to-End Cross-Domain Text-to-SQL Semantic Parsing with Auxiliary Task

no code implementations17 Jun 2021 Peng Shi, Tao Yu, Patrick Ng, Zhiguo Wang

Furthermore, we propose two value filling methods to build the bridge from the existing zero-shot semantic parsers to real-world applications, considering most of the existing parsers ignore the values filling in the synthesized SQL.

Semantic Parsing Text-To-SQL

DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering

no code implementations CVPR 2022 Ruizhi Shao, Hongwen Zhang, He Zhang, Mingjia Chen, YanPei Cao, Tao Yu, Yebin Liu

We introduce DoubleField, a novel framework combining the merits of both surface field and radiance field for high-fidelity human reconstruction and rendering.

Transfer Learning

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

1 code implementation NAACL 2021 Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev

As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed.

Meeting Summarization

POSEFusion: Pose-guided Selective Fusion for Single-view Human Volumetric Capture

no code implementations CVPR 2021 Zhe Li, Tao Yu, Zerong Zheng, Kaiwen Guo, Yebin Liu

By contributing a novel reconstruction framework which contains pose-guided keyframe selection and robust implicit surface fusion, our method fully utilizes the advantages of both tracking-based methods and tracking-free inference methods, and finally enables the high-fidelity reconstruction of dynamic surface details even in the invisible regions.

3D Reconstruction

Local Patch AutoAugment with Multi-Agent Collaboration

2 code implementations20 Mar 2021 Shiqi Lin, Tao Yu, Ruoyu Feng, Xin Li, Xin Jin, Zhibo Chen

We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image.

Data Augmentation Fine-Grained Image Recognition +2

Nematicity Arising from a Chiral Superconducting Ground State in Magic-Angle Twisted Bilayer Graphene under In-Plane Magnetic Fields

no code implementations5 Jan 2021 Tao Yu, Dante M. Kennes, Angel Rubio, Michael A. Sentef

Recent measurements of the resistivity in magic-angle twisted bilayer graphene near the superconducting transition temperature show two-fold anisotropy, or nematicity, when changing the direction of an in-plane magnetic field [Cao \textit{et al.}, Science \textbf{372}, 264 (2021)].


SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing

no code implementations NeurIPS Workshop CAP 2020 Tao Yu, Rui Zhang, Alex Polozov, Christopher Meek, Ahmed Hassan Awadallah

Conversational Semantic Parsing (CSP) is the task of converting a sequence of natural language queries to formal language (e. g., SQL, SPARQL) that can be executed against a structured ontology (e. g. databases, knowledge bases).

Ranked #3 on Multi-domain Dialogue State Tracking on MULTIWOZ 2.1 (using extra training data)

Dialogue State Tracking Language Modelling +4

Learning Omni-frequency Region-adaptive Representations for Real Image Super-Resolution

no code implementations11 Dec 2020 Xin Li, Xin Jin, Tao Yu, Yingxue Pang, Simeng Sun, Zhizheng Zhang, Zhibo Chen

Traditional single image super-resolution (SISR) methods that focus on solving single and uniform degradation (i. e., bicubic down-sampling), typically suffer from poor performance when applied into real-world low-resolution (LR) images due to the complicated realistic degradations.

Image Super-Resolution

Deep Implicit Templates for 3D Shape Representation

1 code implementation CVPR 2021 Zerong Zheng, Tao Yu, Qionghai Dai, Yebin Liu

Deep implicit functions (DIFs), as a kind of 3D shape representation, are becoming more and more popular in the 3D vision community due to their compactness and strong representation power.

3D Shape Representation

DeepCloth: Neural Garment Representation for Shape and Style Editing

no code implementations30 Nov 2020 Zhaoqi Su, Tao Yu, Yangang Wang, Yebin Liu

In this work, we introduce, DeepCloth, a unified framework for garment representation, reconstruction, animation and editing.

Spin-Wave Doppler Shift by Magnon Drag in Magnetic Insulators

no code implementations30 Nov 2020 Tao Yu, Chen Wang, Michael A. Sentef, Gerrit E. W. Bauer

The Doppler shift of the quasiparticle dispersion by charge currents is responsible for the critical supercurrents in superconductors and instabilities of the magnetic ground state of metallic ferromagnets.

Mesoscale and Nanoscale Physics

Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping

no code implementations30 Nov 2020 Xiaochen Zhao, Zerong Zheng, Chaonan Ji, Zhenyi Liu, Siyou Lin, Tao Yu, Jinli Suo, Yebin Liu

We introduce VERTEX, an effective solution to recover 3D shape and intrinsic texture of vehicles from uncalibrated monocular input in real-world street environments.

Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL

1 code implementation23 Oct 2020 Yusen Zhang, Xiangyu Dong, Shuaichen Chang, Tao Yu, Peng Shi, Rui Zhang

Neural models have achieved significant results on the text-to-SQL task, in which most current work assumes all the input questions are legal and generates a SQL query for any input.


Semantic Evaluation for Text-to-SQL with Distilled Test Suites

3 code implementations EMNLP 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models.


GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

1 code implementation ICLR 2021 Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, Richard Socher, Caiming Xiong

We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data.

Inductive Bias Language Modelling +3

NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image

1 code implementation ECCV 2020 Lizhen Wang, Xiaochen Zhao, Tao Yu, Songtao Wang, Yebin Liu

We propose NormalGAN, a fast adversarial learning-based method to reconstruct the complete and detailed 3D human from a single RGB-D image.

3D Human Reconstruction Denoising

PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction

1 code implementation8 Jul 2020 Zerong Zheng, Tao Yu, Yebin Liu, Qionghai Dai

To overcome the limitations of regular 3D representations, we propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function.

3D Human Reconstruction Camera Calibration

Semantic Evaluation for Text-to-SQL with Distilled Test Suite

no code implementations2 Jul 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models, where a predicted query is semantically correct if its denotation is the same as the gold for every possible database.

Semantic Parsing Text-To-SQL

MulayCap: Multi-layer Human Performance Capture Using A Monocular Video Camera

no code implementations13 Apr 2020 Zhaoqi Su, Weilin Wan, Tao Yu, Lingjie Liu, Lu Fang, Wenping Wang, Yebin Liu

We introduce MulayCap, a novel human performance capture method using a monocular video camera without the need for pre-scanning.

Robust 3D Self-portraits in Seconds

no code implementations CVPR 2020 Zhe Li, Tao Yu, Chuanyu Pan, Zerong Zheng, Yebin Liu

In this paper, we propose an efficient method for robust 3D self-portraits using a single RGBD camera.

Salvaging Federated Learning by Local Adaptation

2 code implementations12 Feb 2020 Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov

First, we show that on standard tasks such as next-word prediction, many participants gain no benefit from FL because the federated model is less accurate on their data than the models they can train locally on their own.

Federated Learning Knowledge Distillation +1

Numerically Accurate Hyperbolic Embeddings Using Tiling-Based Models

2 code implementations NeurIPS 2019 Tao Yu, Christopher M. De Sa

Hyperbolic embeddings achieve excellent performance when embedding hierarchical data structures like synonym or type hierarchies, but they can be limited by numerical error when ordinary floating-point numbers are used to represent points in hyperbolic space.

Region Normalization for Image Inpainting

1 code implementation23 Nov 2019 Tao Yu, Zongyu Guo, Xin Jin, Shilin Wu, Zhibo Chen, Weiping Li, Zhizheng Zhang, Sen Liu

In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation.

Image Inpainting

A New Defense Against Adversarial Images: Turning a Weakness into a Strength

1 code implementation NeurIPS 2019 Tao Yu, Shengyuan Hu, Chuan Guo, Wei-Lun Chao, Kilian Q. Weinberger

Natural images are virtually surrounded by low-density misclassified regions that can be efficiently discovered by gradient-guided search --- enabling the generation of adversarial images.

Adversarial Defense

Progressive Image Inpainting with Full-Resolution Residual Network

2 code implementations24 Jul 2019 Zongyu Guo, Zhibo Chen, Tao Yu, Jiale Chen, Sen Liu

Recently, learning-based algorithms for image inpainting achieve remarkable progress dealing with squared or irregular holes.

Image Inpainting

SParC: Cross-Domain Semantic Parsing in Context

5 code implementations ACL 2019 Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev

The best model obtains an exact match accuracy of 20. 2% over all questions and less than10% over all interaction sequences, indicating that the cross-domain setting and the con-textual phenomena of the dataset present significant challenges for future research.

Semantic Parsing Text-To-SQL

DC-SPP-YOLO: Dense Connection and Spatial Pyramid Pooling Based YOLO for Object Detection

no code implementations20 Mar 2019 Zhanchao Huang, Jianlin Wang, Xuesong Fu, Tao Yu, Yongqi Guo, Rutong Wang

Therefore, a dense connection (DC) and spatial pyramid pooling (SPP) based YOLO (DC-SPP-YOLO) method for ameliorating the object detection accuracy of YOLOv2 is proposed in this paper.

object-detection Object Detection

SimulCap : Single-View Human Performance Capture with Cloth Simulation

no code implementations CVPR 2019 Tao Yu, Zerong Zheng, Yuan Zhong, Jianhui Zhao, Qionghai Dai, Gerard Pons-Moll, Yebin Liu

This paper proposes a new method for live free-viewpoint human performance capture with dynamic details (e. g., cloth wrinkles) using a single RGBD camera.

DeepHuman: 3D Human Reconstruction from a Single Image

1 code implementation ICCV 2019 Zerong Zheng, Tao Yu, Yixuan Wei, Qionghai Dai, Yebin Liu

We propose DeepHuman, an image-guided volume-to-volume translation CNN for 3D human reconstruction from a single RGB image.

3D Human Reconstruction Pose Estimation +1

Simplifying Graph Convolutional Networks

7 code implementations19 Feb 2019 Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr., Christopher Fifty, Tao Yu, Kilian Q. Weinberger

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations.

Graph Regression Image Classification +5

SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task

2 code implementations11 Oct 2018 Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, Dragomir Radev

In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and cross-domain text-to-SQL generation task.

Semantic Parsing Text-To-SQL

HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs

no code implementations ECCV 2018 Zerong Zheng, Tao Yu, Hao Li, Kaiwen Guo, Qionghai Dai, Lu Fang, Yebin Liu

We propose a light-weight and highly robust real-time human performance capture method based on a single depth camera and sparse inertial measurement units (IMUs).

Surface Reconstruction

TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation

1 code implementation NAACL 2018 Tao Yu, Zifan Li, Zilin Zhang, Rui Zhang, Dragomir Radev

Interacting with relational databases through natural language helps users of any background easily query and analyze a vast amount of data.

slot-filling Slot Filling +2

Curvature-based Comparison of Two Neural Networks

no code implementations21 Jan 2018 Tao Yu, Huan Long, John E. Hopcroft

In this paper we show the similarities and differences of two deep neural networks by comparing the manifolds composed of activation vectors in each fully connected layer of them.

Vocal Bursts Valence Prediction

Preliminary theoretical troubleshooting in Variational Autoencoder

no code implementations ICLR 2018 Shiqi Liu, Qian Zhao, Xiangyong Cao, Deyu Meng, Zilu Ma, Tao Yu

This paper tries to preliminarily address VAE's intrinsic dimension, real factor, disentanglement and indicator issues theoretically in the idealistic situation and implementation issue practically through noise modeling perspective in the realistic case.


The Local Dimension of Deep Manifold

no code implementations5 Nov 2017 Mengxiao Zhang, Wangquan Wu, Yanren Zhang, Kun He, Tao Yu, Huan Long, John E. Hopcroft

Our results show that the dimensions of different categories are close to each other and decline quickly along the convolutional layers and fully connected layers.

BodyFusion: Real-Time Capture of Human Motion and Surface Geometry Using a Single Depth Camera

no code implementations ICCV 2017 Tao Yu, Kaiwen Guo, Feng Xu, Yuan Dong, Zhaoqi Su, Jianhui Zhao, Jianguo Li, Qionghai Dai, Yebin Liu

To reduce the ambiguities of the non-rigid deformation parameterization on the surface graph nodes, we take advantage of the internal articulated motion prior for human performance and contribute a skeleton-embedded surface fusion (SSF) method.

Surface Reconstruction

Leveraging Sparse and Dense Feature Combinations for Sentiment Classification

no code implementations13 Aug 2017 Tao Yu, Christopher Hidey, Owen Rambow, Kathleen McKeown

This model outperforms many deep learning models and achieves comparable results to other deep learning models with complex architectures on sentiment analysis datasets.

BIG-bench Machine Learning Classification +3

Cannot find the paper you are looking for? You can Submit a new open access paper.