Search Results for author: Tianyi Zhang

Found 40 papers, 15 papers with code

Splitting vs. Merging: Mining Object Regions with Discrepancy and Intersection Loss for Weakly Supervised Semantic Segmentation

no code implementations ECCV 2020 Tianyi Zhang, Guosheng Lin, Weide Liu, Jianfei Cai, Alex Kot

Finally, by training the segmentation model with the masks generated by our Splitting vs Merging strategy, we achieve the state-of-the-art weakly-supervised segmentation results on the Pascal VOC 2012 benchmark.

Weakly supervised segmentation Weakly-Supervised Semantic Segmentation

Pancreatic Cancer ROSE Image Classification Based on Multiple Instance Learning with Shuffle Instances

no code implementations7 Jun 2022 Tianyi Zhang, Youdan Feng, Yunlu Feng, Guanglei Zhang

Computer-aided diagnosis (CAD) using the deep learning method has the potential to solve the problem of insufficient pathology staffing.

Image Classification Multiple Instance Learning

Decentralized Training of Foundation Models in Heterogeneous Environments

1 code implementation2 Jun 2022 Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Re, Ce Zhang

Our key technical contribution is a scheduling algorithm that allocates different computational "tasklets" in the training of foundation models to a group of decentralized GPU devices connected by a slow heterogeneous network.

TempLM: Distilling Language Models into Template-Based Generators

no code implementations23 May 2022 Tianyi Zhang, Mina Lee, Lisa Li, Ende Shen, Tatsunori B. Hashimoto

While pretrained language models (PLMs) have greatly improved text generation, they have also been known to produce unfaithful or inappropriate content.

Pretrained Language Models Text Generation

Model-Based Neural Network and Its Application to Line Spectral Estimation

no code implementations14 Feb 2022 Yi Jiang, Tianyi Zhang, Wei zhang

Owing to the same layered form as an ANN, a MNN can also be optimized using the back-propagation (BP) algorithm.

Rethinking Importance Weighting for Transfer Learning

no code implementations19 Dec 2021 Nan Lu, Tianyi Zhang, Tongtong Fang, Takeshi Teshima, Masashi Sugiyama

A key assumption in supervised learning is that training and test data follow the same probability distribution.

Selection bias Transfer Learning

PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records

no code implementations10 Dec 2021 Tianyi Zhang, Shirui Zhang, Ziwei Chen, Dianbo Liu

Federated machine learning is a versatile and flexible tool to utilize distributed data from different sources, especially when communication technology develops rapidly and an unprecedented amount of data could be collected on mobile devices nowadays.

Federated Learning Meta-Learning +1

On the Opportunities and Risks of Foundation Models

no code implementations16 Aug 2021 Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Kohd, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, aditi raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang

AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.

Transfer Learning

From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers

1 code implementation16 Jul 2021 Krzysztof Choromanski, Han Lin, Haoxian Chen, Tianyi Zhang, Arijit Sehanobish, Valerii Likhosherstov, Jack Parker-Holder, Tamas Sarlos, Adrian Weller, Thomas Weingarten

In this paper we provide, to the best of our knowledge, the first comprehensive approach for incorporating various masking mechanisms into Transformers architectures in a scalable way.

Graph Attention

PSRR-MaxpoolNMS: Pyramid Shifted MaxpoolNMS with Relationship Recovery

no code implementations CVPR 2021 Tianyi Zhang, Jie Lin, Peng Hu, Bin Zhao, Mohamed M. Sabry Aly

Unlike convolutions which are inherently parallel, the de-facto standard for NMS, namely GreedyNMS, cannot be easily parallelized and thus could be the performance bottleneck in convolutional object detection pipelines.

object-detection Object Detection

On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies

1 code implementation NAACL 2021 Tianyi Zhang, Tatsunori Hashimoto

We study how masking and predicting tokens in an unsupervised fashion can give rise to linguistic structures and downstream performance gains.

Inductive Bias Language Modelling +2

Sinusoidal Parameter Estimation from Signed Measurements via Majorization-Minimization Based RELAX

no code implementations21 Mar 2021 Jiaying Ren, Tianyi Zhang, Jian Li, Petre Stoica

In a previous paper, a relaxation-based algorithm, referred to as 1bRELAX, has been proposed to iteratively maximize the likelihood function.

Joint RFI Mitigation and Radar Echo Recovery for One-Bit UWB Radar

no code implementations19 Mar 2021 Tianyi Zhang, Jiaying Ren, Jian Li, Lam H. Nguyen, Petre Stoica

Radio frequency interference (RFI) mitigation and radar echo recovery are critically important for the proper functioning of ultra-wideband (UWB) radar systems using one-bit sampling techniques.

Learning to Stop with Surprisingly Few Samples

no code implementations19 Feb 2021 Daniel Russo, Assaf Zeevi, Tianyi Zhang

We consider a discounted infinite horizon optimal stopping problem.

RFI Mitigation for One-bit UWB Radar Systems

no code implementations17 Feb 2021 Tianyi Zhang, Jiaying Ren, Jian Li, Lam H. Nguyen, Petre Stoica

A one-bit UWB system obtains its signed measurements via a low-cost and high rate sampling scheme, referred to as the Continuous Time Binary Value (CTBV) technology.

Quantization

Generative Adversarial Network based Heuristics for Sampling-based Path Planning

1 code implementation7 Dec 2020 Tianyi Zhang, Jiankun Wang, Max Q. -H. Meng

Sampling-based path planning is a popular methodology for robot path planning.

Can Steering Wheel Detect Your Driving Fatigue?

no code implementations18 Oct 2020 Jianchao Lu, Xi Zheng, Tianyi Zhang, Michael Sheng, Chen Wang, Jiong Jin, Shui Yu, Wanlei Zhou

In this paper, we propose a novel driver fatigue detection method by embedding surface electromyography (sEMG) sensors on a steering wheel.

ICS-Assist: Intelligent Customer Inquiry Resolution Recommendation in Online Customer Service for Large E-Commerce Businesses

no code implementations22 Aug 2020 Min Fu, Jiwei Guan, Xi Zheng, Jie zhou, Jianchao Lu, Tianyi Zhang, Shoujie Zhuo, Lijun Zhan, Jian Yang

Existing solution recommendation methods for online customer service are unable to determine the best solutions at runtime, leading to poor satisfaction of end customers.

Modeling the Field Value Variations and Field Interactions Simultaneously for Fraud Detection

no code implementations8 Aug 2020 Dongbo Xi, Bowen Song, Fuzhen Zhuang, Yongchun Zhu, Shuai Chen, Tianyi Zhang, Yuan Qi, Qing He

In this paper, we propose the Dual Importance-aware Factorization Machines (DIFM), which exploits the internal field information among users' behavior sequence from dual perspectives, i. e., field value variations and field interactions simultaneously for fraud detection.

Fraud Detection

A One-step Approach to Covariate Shift Adaptation

no code implementations8 Jul 2020 Tianyi Zhang, Ikko Yamane, Nan Lu, Masashi Sugiyama

A default assumption in many machine learning scenarios is that the training and test samples are drawn from the same probability distribution.

Revisiting Few-sample BERT Fine-tuning

1 code implementation ICLR 2021 Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, Yoav Artzi

We empirically test the impact of these factors, and identify alternative practices that resolve the commonly observed instability of the process.

Demystifying Orthogonal Monte Carlo and Beyond

no code implementations NeurIPS 2020 Han Lin, Haoxian Chen, Tianyi Zhang, Clement Laroche, Krzysztof Choromanski

Orthogonal Monte Carlo (OMC) is a very effective sampling algorithm imposing structural geometric conditions (orthogonality) on samples for variance reduction.

reinforcement-learning

Stereo Endoscopic Image Super-Resolution Using Disparity-Constrained Parallel Attention

no code implementations19 Mar 2020 Tianyi Zhang, Yun Gu, Xiaolin Huang, Enmei Tu, Jie Yang

In particular, we incorporate a disparity-based constraint mechanism into the generation of SR images in a deep neural network framework with an additional atrous parallax-attention modules.

Image Super-Resolution

Supporting OpenMP 5.0 Tasks in hpxMP -- A study of an OpenMP implementation within Task Based Runtime Systems

1 code implementation19 Feb 2020 Tianyi Zhang, Shahrzad Shirzad, Bibek Wagle, Adrian S. Lemoine, Patrick Diehl, Hartmut Kaiser

This paper is a follow-up paper on the fundamental implementation of hpxMP, an implementation of the OpenMP standard which utilizes the C++ standard library for Parallelism and Concurrency (HPX) to schedule and manage tasks.

Distributed, Parallel, and Cluster Computing Programming Languages

An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models

1 code implementation6 Feb 2020 Yao Deng, Xi Zheng, Tianyi Zhang, Chen Chen, Guannan Lou, Miryung Kim

We derive several implications for system and middleware builders: (1) when adding a defense component against adversarial attacks, it is important to deploy multiple defense methods in tandem to achieve a good coverage of various attacks, (2) a blackbox attack is much less effective compared with a white-box attack, implying that it is important to keep model details (e. g., model architecture, hyperparameters) confidential via model obfuscation, and (3) driving models with a complex architecture are preferred if computing resources permit as they are more resilient to adversarial attacks than simple models.

Autonomous Driving

Identifying Mislabeled Data using the Area Under the Margin Ranking

2 code implementations NeurIPS 2020 Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger

Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled.

Mitigating Overfitting in Supervised Classification from Two Unlabeled Datasets: A Consistent Risk Correction Approach

no code implementations20 Oct 2019 Nan Lu, Tianyi Zhang, Gang Niu, Masashi Sugiyama

The recently proposed unlabeled-unlabeled (UU) classification method allows us to train a binary classifier only from two unlabeled datasets with different class priors.

Classification General Classification

QPyTorch: A Low-Precision Arithmetic Simulation Framework

2 code implementations9 Oct 2019 Tianyi Zhang, Zhiqiu Lin, Guandao Yang, Christopher De Sa

Low-precision training reduces computational cost and produces efficient models.

Quantization

Detecting Noisy Training Data with Loss Curves

no code implementations25 Sep 2019 Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger

This paper introduces a new method to discover mislabeled training samples and to mitigate their impact on the training process of deep networks.

Fixed-price Diffusion Mechanism Design

no code implementations14 May 2019 Tianyi Zhang, Dengji Zhao, Wen Zhang, Xuming He

We consider a fixed-price mechanism design setting where a seller sells one item via a social network, but the seller can only directly communicate with her neighbours initially.

SWALP : Stochastic Weight Averaging in Low-Precision Training

2 code implementations26 Apr 2019 Guandao Yang, Tianyi Zhang, Polina Kirichenko, Junwen Bai, Andrew Gordon Wilson, Christopher De Sa

Low precision operations can provide scalability, memory savings, portability, and energy efficiency.

An Introduction to hpxMP: A Modern OpenMP Implementation Leveraging HPX, An Asynchronous Many-Task System

1 code implementation7 Mar 2019 Tianyi Zhang, Shahrzad Shirzad, Patrick Diehl, R. Tohid, Weile Wei, Hartmut Kaiser

Not only must users port their own codes, but often users rely on highly optimized libraries such as BLAS and LAPACK which use OpenMP for parallization.

Distributed, Parallel, and Cluster Computing

Simplifying Graph Convolutional Networks

6 code implementations19 Feb 2019 Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr., Christopher Fifty, Tao Yu, Kilian Q. Weinberger

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations.

Ranked #3 on Text Classification on 20NEWS (using extra training data)

Graph Regression Image Classification +5

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

no code implementations29 Mar 2018 Zhize Li, Tianyi Zhang, Shuyu Cheng, Jun Zhu, Jian Li

In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics.

Bayesian Inference

Cannot find the paper you are looking for? You can Submit a new open access paper.