Search Results for author: Tianyu Zhang

Found 48 papers, 23 papers with code

Sketch-Guided Scene Image Generation

no code implementations9 Jul 2024 Tianyu Zhang, Xiaoxuan Xie, Xusheng Du, Haoran Xie

In this study, we propose a novel sketch-guided scene image generation framework, decomposing the task of scene image scene generation from sketch inputs into object-level cross-domain generation and scene-level image construction.

Image Generation Object +1

Non-Adversarial Learning: Vector-Quantized Common Latent Space for Multi-Sequence MRI

1 code implementation3 Jul 2024 Luyi Han, Tao Tan, Tianyu Zhang, Xin Wang, Yuan Gao, Chunyao Lu, Xinglong Liang, Haoran Dou, Yunzhi Huang, Ritse Mann

We propose a generative model that compresses discrete representations of each sequence to estimate the Gaussian distribution of vector-quantized common (VQC) latent space between multiple sequences.

Contrastive Learning One-Shot Segmentation

Advantage Alignment Algorithms

no code implementations20 Jun 2024 Juan Agustin Duque, Milad Aghajohari, Tim Cooijmans, Tianyu Zhang, Aaron Courville

The growing presence of artificially intelligent agents in everyday decision-making, from LLM assistants to autonomous vehicles, hints at a future in which conflicts may arise from each agent optimizing individual interests.

Autonomous Vehicles Decision Making

MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation

1 code implementation11 Jun 2024 Lu Li, Tianyu Zhang, Zhiqi Bu, Suyuchen Wang, Huan He, Jie Fu, Yonghui Wu, Jiang Bian, Yong Chen, Yoshua Bengio

MAP identifies a Pareto set of scaling coefficients for merging multiple models to reflect the trade-offs.

VCR: Visual Caption Restoration

1 code implementation10 Jun 2024 Tianyu Zhang, Suyuchen Wang, Lu Li, Ge Zhang, Perouz Taslakian, Sai Rajeswar, Jie Fu, Bang Liu, Yoshua Bengio

This task stems from the observation that text embedded in images is intrinsically different from common visual elements and natural language due to the need to align the modalities of vision, text, and text embedded in images.

Language Modelling Masked Language Modeling +3

TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion Model

no code implementations7 May 2024 Yongming Zhang, Tianyu Zhang, Haoran Xie

In this work, we propose TexControl, a sketch-based fashion generation framework that uses a two-stage pipeline to generate the fashion image corresponding to the sketch input.

Image Generation

Subtoxic Questions: Dive Into Attitude Change of LLM's Response in Jailbreak Attempts

no code implementations12 Apr 2024 Tianyu Zhang, Zixuan Zhao, Jiaqi Huang, Jingyu Hua, Sheng Zhong

As Large Language Models (LLMs) of Prompt Jailbreaking are getting more and more attention, it is of great significance to raise a generalized research paradigm to evaluate attack strengths and a basic model to conduct subtler experiments.

Dynamic Prompt Optimizing for Text-to-Image Generation

1 code implementation CVPR 2024 Wenyi Mo, Tianyu Zhang, Yalong Bai, Bing Su, Ji-Rong Wen, Qing Yang

Users assign weights or alter the injection time steps of certain words in the text prompts to improve the quality of generated images.

Text-to-Image Generation

Generative AI for Architectural Design: A Literature Review

no code implementations30 Mar 2024 Chengyuan Li, Tianyu Zhang, Xusheng Du, Ye Zhang, Haoran Xie

This paper explores the extensive applications of generative AI technologies in architectural design, a trend that has benefited from the rapid development of deep generative models.

Label Informed Contrastive Pretraining for Node Importance Estimation on Knowledge Graphs

1 code implementation26 Feb 2024 Tianyu Zhang, Chengbin Hou, Rui Jiang, Xuegong Zhang, Chenghu Zhou, Ke Tang, Hairong Lv

Considering the NIE problem, LICAP adopts a novel sampling strategy called top nodes preferred hierarchical sampling to first group all interested nodes into a top bin and a non-top bin based on node importance scores, and then divide the nodes within top bin into several finer bins also based on the scores.

Contrastive Learning Graph Attention +1

CoSSegGaussians: Compact and Swift Scene Segmenting 3D Gaussians with Dual Feature Fusion

no code implementations11 Jan 2024 Bin Dou, Tianyu Zhang, Yongjia Ma, Zhaohui Wang, Zejian yuan

We propose Compact and Swift Segmenting 3D Gaussians(CoSSegGaussians), a method for compact 3D-consistent scene segmentation at fast rendering speed with only RGB images input.

Panoptic Segmentation Scene Segmentation +2

CAMEL: CAusal Motion Enhancement Tailored for Lifting Text-driven Video Editing

no code implementations CVPR 2024 Guiwei Zhang, Tianyu Zhang, Guanglin Niu, Zichang Tan, Yalong Bai, Qing Yang

Second to enhance motion coherence and extend the generalization of appearance content to creative textual prompts we propose the causal motion-enhanced attention mechanism.

Video Editing

Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data

no code implementations18 Oct 2023 Tianyu Zhang, Jing Lei

We propose a weighted rolling-validation procedure, an online variant of leave-one-out cross-validation, that costs minimal extra computation for many typical stochastic gradient descent estimators.

Model Selection

AI For Global Climate Cooperation 2023 Competition Proceedings

no code implementations10 Jul 2023 Yoshua Bengio, Prateek Gupta, Lu Li, Soham Phade, Sunil Srinivasa, Andrew Williams, Tianyu Zhang, Yang Zhang, Stephan Zheng

On the other hand, an interdisciplinary panel of human experts in law, policy, sociology, economics and environmental science, evaluated the solutions qualitatively.

Decision Making Ethics +1

An Explainable Deep Framework: Towards Task-Specific Fusion for Multi-to-One MRI Synthesis

1 code implementation3 Jul 2023 Luyi Han, Tianyu Zhang, Yunzhi Huang, Haoran Dou, Xin Wang, Yuan Gao, Chunyao Lu, Tan Tao, Ritse Mann

Multi-sequence MRI is valuable in clinical settings for reliable diagnosis and treatment prognosis, but some sequences may be unusable or missing for various reasons.

Deep-Learning-based Vasculature Extraction for Single-Scan Optical Coherence Tomography Angiography

no code implementations17 Apr 2023 Jinpeng Liao, Tianyu Zhang, Yilong Zhang, Chunhui Li, Zhihong Huang

In comparison to OCTA images obtained via the SV-OCTA (PSNR: 17. 809) and ED-OCTA (PSNR: 18. 049) using four-repeated OCT scans, OCTA images extracted by VET exhibit moderate quality (PSNR: 17. 515) and higher image contrast while reducing the required data acquisition time from ~8 s to ~2 s. Based on visual observations, the proposed VET outperforms SV and ED algorithms when using neck and face OCTA data in areas that are challenging to scan.

Unsupervised Hashing with Similarity Distribution Calibration

1 code implementation15 Feb 2023 Kam Woh Ng, Xiatian Zhu, Jiun Tian Hoe, Chee Seng Chan, Tianyu Zhang, Yi-Zhe Song, Tao Xiang

However, these methods often overlook the fact that the similarity between data points in the continuous feature space may not be preserved in the discrete hash code space, due to the limited similarity range of hash codes.

Deep Hashing Image Retrieval

IMPORTANT-Net: Integrated MRI Multi-Parameter Reinforcement Fusion Generator with Attention Network for Synthesizing Absent Data

1 code implementation3 Feb 2023 Tianyu Zhang, Tao Tan, Luyi Han, Xin Wang, Yuan Gao, Jonas Teuwen, Regina Beets-Tan, Ritse Mann

Then the multi-parameter fusion with attention module enables the interaction of the encoded information from different parameters through a set of algorithmic strategies, and applies different weights to the information through the attention mechanism after information fusion to obtain refined representation information.

Lesion Classification Lesion Detection

Synthesis-based Imaging-Differentiation Representation Learning for Multi-Sequence 3D/4D MRI

1 code implementation1 Feb 2023 Luyi Han, Tao Tan, Tianyu Zhang, Yunzhi Huang, Xin Wang, Yuan Gao, Jonas Teuwen, Ritse Mann

Multi-sequence MRIs can be necessary for reliable diagnosis in clinical practice due to the complimentary information within sequences.

Representation Learning

Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads

no code implementations6 Jan 2023 Vincent Mai, Philippe Maisonneuve, Tianyu Zhang, Hadi Nekoei, Liam Paull, Antoine Lesage-Landry

To integrate high amounts of renewable energy resources, electrical power grids must be able to cope with high amplitude, fast timescale variations in power generation.

Multi-agent Reinforcement Learning reinforcement-learning +1

PHA: Patch-Wise High-Frequency Augmentation for Transformer-Based Person Re-Identification

no code implementations CVPR 2023 Guiwei Zhang, Yongfei Zhang, Tianyu Zhang, Bo Li, ShiLiang Pu

Although recent studies empirically show that injecting Convolutional Neural Networks (CNNs) into Vision Transformers (ViTs) can improve the performance of person re-identification, the rationale behind it remains elusive.

Person Re-Identification

SGDraw: Scene Graph Drawing Interface Using Object-Oriented Representation

1 code implementation30 Nov 2022 Tianyu Zhang, Xusheng Du, Chia-Ming Chang, Xi Yang, Haoran Xie

However, it is difficult to draw a proper scene graph for image retrieval, image generation, and multi-modal applications.

Graph Generation Image Generation +5

Data-Driven Deep Supervision for Skin Lesion Classification

no code implementations4 Sep 2022 Suraj Mishra, Yizhe Zhang, Li Zhang, Tianyu Zhang, X. Sharon Hu, Danny Z. Chen

Specifically, we analyze the convolutional network's behavior (field-of-view) to find the location of deep supervision for improved feature extraction.

Classification Lesion Classification +2

AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N

2 code implementations15 Aug 2022 Tianyu Zhang, Andrew Williams, Soham Phade, Sunil Srinivasa, Yang Zhang, Prateek Gupta, Yoshua Bengio, Stephan Zheng

To facilitate this research, here we introduce RICE-N, a multi-region integrated assessment model that simulates the global climate and economy, and which can be used to design and evaluate the strategic outcomes for different negotiation and agreement frameworks.

Ethics Multi-agent Reinforcement Learning

Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branch

no code implementations14 Jun 2022 Tianyu Zhang, Amin Banitalebi-Dehkordi, Yong Zhang

We propose a new approach for solving the data labeling and inference latency issues in combinatorial optimization based on the use of the reinforcement learning (RL) paradigm.

BIG-bench Machine Learning Combinatorial Optimization +4

Biological Sequence Design with GFlowNets

1 code implementation2 Mar 2022 Moksh Jain, Emmanuel Bengio, Alex-Hernandez Garcia, Jarrid Rector-Brooks, Bonaventure F. P. Dossou, Chanakya Ekbote, Jie Fu, Tianyu Zhang, Micheal Kilgour, Dinghuai Zhang, Lena Simine, Payel Das, Yoshua Bengio

In this work, we propose an active learning algorithm leveraging epistemic uncertainty estimation and the recently proposed GFlowNets as a generator of diverse candidate solutions, with the objective to obtain a diverse batch of useful (as defined by some utility function, for example, the predicted anti-microbial activity of a peptide) and informative candidates after each round.

Active Learning Diversity

FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer

1 code implementation27 Nov 2021 Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, Shuchang Zhou

Network quantization significantly reduces model inference complexity and has been widely used in real-world deployments.


CGCL: Collaborative Graph Contrastive Learning without Handcrafted Graph Data Augmentations

1 code implementation5 Nov 2021 Tianyu Zhang, Yuxiang Ren, Wenzheng Feng, Weitao Du, Xuecang Zhang

In this paper, we show the potential hazards of inappropriate augmentations and then propose a novel Collaborative Graph Contrastive Learning framework (CGCL).

Contrastive Learning Data Augmentation +2

On the Optimality of Nuclear-norm-based Matrix Completion for Problems with Smooth Non-linear Structure

no code implementations5 May 2021 Yunhua Xiang, Tianyu Zhang, Xu Wang, Ali Shojaie, Noah Simon

Originally developed for imputing missing entries in low rank, or approximately low rank matrices, matrix completion has proven widely effective in many problems where there is no reason to assume low-dimensional linear structure in the underlying matrix, as would be imposed by rank constraints.

Matrix Completion

Ternary Hashing

no code implementations16 Mar 2021 Chang Liu, Lixin Fan, Kam Woh Ng, Yilun Jin, Ce Ju, Tianyu Zhang, Chee Seng Chan, Qiang Yang

This paper proposes a novel ternary hash encoding for learning to hash methods, which provides a principled more efficient coding scheme with performances better than those of the state-of-the-art binary hashing counterparts.


Learning Contextualized Knowledge Graph Structures for Commonsense Reasoning

no code implementations1 Jan 2021 Jun Yan, Mrigank Raman, Tianyu Zhang, Ryan Rossi, Handong Zhao, Sungchul Kim, Nedim Lipka, Xiang Ren

Recently, neural-symbolic architectures have achieved success on commonsense reasoning through effectively encoding relational structures retrieved from external knowledge graphs (KGs) and obtained state-of-the-art results in tasks such as (commonsense) question answering and natural language inference.

Knowledge Graphs Natural Language Inference +1

UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification

1 code implementation CVPR 2021 Tianyu Zhang, Lingxi Xie, Longhui Wei, Zijie Zhuang, Yongfei Zhang, Bo Li, Qi Tian

The main difficulty of person re-identification (ReID) lies in collecting annotated data and transferring the model across different domains.

Domain Adaptation Image Generation +1

A robust solution of a statistical inverse problem in multiscale computational mechanics using an artificial neural network

no code implementations16 Nov 2020 Florent Pled, Christophe Desceliers, Tianyu Zhang

An initial database made up with input and target data is first generated from the computational model, from which a processed database is deduced by conditioning the input data with respect to the target data using the nonparametric statistics.

Neural Networks Weights Quantization: Target None-retraining Ternary (TNT)

no code implementations18 Dec 2019 Tianyu Zhang, Lei Zhu, Qian Zhao, Kilho Shin

Quantization of weights of deep neural networks (DNN) has proven to be an effective solution for the purpose of implementing DNNs on edge devices such as mobiles, ASICs and FPGAs, because they have no sufficient resources to support computation involving millions of high precision weights and multiply-accumulate operations.


Background Segmentation for Vehicle Re-Identification

no code implementations15 Oct 2019 Mingjie Wu, Yongfei Zhang, Tianyu Zhang, Wen-qi Zhang

Vehicle re-identification (Re-ID) is very important in intelligent transportation and video surveillance. Prior works focus on extracting discriminative features from visual appearance of vehicles or using visual-spatio-temporal information. However, background interference in vehicle re-identification have not been explored. In the actual large-scale spatio-temporal scenes, the same vehicle usually appears in different backgrounds while different vehicles might appear in the same background, which will seriously affect the re-identification performance.

Segmentation Vehicle Re-Identification

Single Camera Training for Person Re-identification

1 code implementation24 Sep 2019 Tianyu Zhang, Lingxi Xie, Longhui Wei, Yongfei Zhang, Bo Li, Qi Tian

Differently, this paper investigates ReID in an unexplored single-camera-training (SCT) setting, where each person in the training set appears in only one camera.

Metric Learning Person Re-Identification

Cannot find the paper you are looking for? You can Submit a new open access paper.