no code implementations • IWSLT 2016 • Yang Zhang, Jan Niehues, Alexander Waibel
Neural models have recently shown big improvements in the performance of phrase-based machine translation.
no code implementations • CL (ACL) 2022 • Yang Zhang, Qiang Ma
A neural network model is designed to maximize the similarity between the embedding of the three inputs (local context words, section headers, and structural contexts) and the target citation appearing in the context.
no code implementations • 29 May 2024 • Xinyue Shen, Yixin Wu, Michael Backes, Yang Zhang
This resistance is primarily due to GPT-4o's internal safeguards and the difficulty of adapting text jailbreak prompts to voice mode.
no code implementations • 28 May 2024 • Yang Zhang, Yawei Li, Xinpeng Wang, Qianli Shen, Barbara Plank, Bernd Bischl, Mina Rezaei, Kenji Kawaguchi
Overparametrized transformer networks are the state-of-the-art architecture for Large Language Models (LLMs).
no code implementations • 27 May 2024 • Yang Zhang, Mingying Li, Huilin Pan, Moyun Liu, Yang Zhou
In this work, we propose an efficient NAS-based framework for visual fault detection of freight trains to search for the task-specific detection head with capacities of multi-scale representation.
no code implementations • 23 May 2024 • Yang Zhang, Shixin Yang, Chenjia Bai, Fei Wu, Xiu Li, Zhen Wang, Xuelong Li
In this paper, we propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans.
1 code implementation • 10 May 2024 • Xiaoyu Wen, Chenjia Bai, Kang Xu, Xudong Yu, Yang Zhang, Xuelong Li, Zhen Wang
In this paper, we propose a novel representation-based approach to measure the domain gap, where the representation is learned through a contrastive objective by sampling transitions from different domains.
1 code implementation • 9 May 2024 • Yixin Wu, Xinlei He, Pascal Berrang, Mathias Humbert, Michael Backes, Neil Zhenqiang Gong, Yang Zhang
This paper fills the gap by conducting a systematic privacy analysis of inductive GNNs through the lens of link stealing attacks, one of the most popular attacks that are specifically designed for GNNs.
no code implementations • 6 May 2024 • Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, Yang Zhang
First, we curate a large dataset of 10K real-world and AI-generated images that are annotated as safe or unsafe based on a set of 11 unsafe categories of images (sexual, violent, hateful, etc.).
1 code implementation • 2 May 2024 • Tianhao Shi, Yang Zhang, Jizhi Zhang, Fuli Feng, Xiangnan He
To this end, we propose Distributionally Robust Fair Optimization (DRFO), which minimizes the worst-case unfairness over all potential probability distributions of missing sensitive attributes instead of the reconstructed one to account for the impact of the reconstruction errors.
no code implementations • 28 Apr 2024 • Jinbo Wen, Ruichen Zhang, Dusit Niyato, Jiawen Kang, Hongyang Du, Yang Zhang, Zhu Han
In this article, we explore the potential of GAI for carbon emissions reduction and propose a novel GAI-enabled solution for low-carbon AIoT.
1 code implementation • 25 Apr 2024 • William Huang, Sam Ghahremani, Siyou Pei, Yang Zhang
We present a data synthesis pipeline to address this disparity in data collection and subsequently improve pose estimation performance for wheelchair users.
1 code implementation • 19 Apr 2024 • Yi Guo, Fanliu Kong, Xiaoyang Li, Hui Li, Wei Chen, Xiaogang Tian, Jinping Cai, Yang Zhang, Shouda Liu
However, existing quantization schemes suffer from significant accuracy degradation at very low bits, or require some additional computational overhead when deployed, making it difficult to be applied to large-scale applications in industry.
no code implementations • 18 Apr 2024 • Yilun Hao, Yongchao Chen, Yang Zhang, Chuchu Fan
We evaluate our framework with TravelPlanner and achieve a success rate of 97%.
1 code implementation • 18 Apr 2024 • Jiabao Ji, Bairu Hou, Zhen Zhang, Guanhua Zhang, Wenqi Fan, Qing Li, Yang Zhang, Gaowen Liu, Sijia Liu, Shiyu Chang
Although large language models (LLMs) have achieved significant success, their vulnerability to adversarial perturbations, including recent jailbreak attacks, has raised considerable concerns.
1 code implementation • 17 Apr 2024 • Yue Zhou, Yada Zhu, Diego Antognini, Yoon Kim, Yang Zhang
This paper studies the relationship between the surface form of a mathematical problem and its solvability by large language models.
no code implementations • 16 Apr 2024 • Zhiyu Hu, Yang Zhang, Minghao Xiao, Wenjie Wang, Fuli Feng, Xiangnan He
The evolving paradigm of Large Language Model-based Recom- mendation (LLMRec) customizes Large Language Models (LLMs) through parameter-efficient fine-tuning (PEFT) using recommenda- tion data.
1 code implementation • 9 Apr 2024 • Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, Boris Ginsburg
Despite achieving nearly perfect accuracy in the vanilla NIAH test, all models exhibit large performance drops as the context length increases.
no code implementations • 22 Mar 2024 • Zhenrui Yue, Huimin Zeng, Yimeng Lu, Lanyu Shang, Yang Zhang, Dong Wang
The proliferation of online misinformation has posed significant threats to public interest.
no code implementations • 17 Mar 2024 • Boshi Wang, Cunhua Pan, Hong Ren, Zhiyuan Yu, Yang Zhang, Mengyu Liu, Gui Zhou
Due to the signal amplification capability of active RISs, the mutual influence between active RISs, which is termed as the "inter-excitation" effect, cannot be ignored.
1 code implementation • 11 Mar 2024 • Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, Tiviatis Sim, Kenji Kawaguchi
Recent advancements in diffusion models have notably improved the perceptual quality of generated images in text-to-image synthesis tasks.
no code implementations • 7 Mar 2024 • Wenjie Wang, Yang Zhang, Xinyu Lin, Fuli Feng, Weiwen Liu, Yong liu, Xiangyu Zhao, Wayne Xin Zhao, Yang song, Xiangnan He
The rise of generative models has driven significant advancements in recommender systems, leaving unique opportunities for enhancing users' personalized recommendations.
no code implementations • 6 Mar 2024 • Peng Dai, Yang Zhang, Tao Liu, Zhen Fan, Tianyuan Du, Zhuo Su, Xiaozheng Zheng, Zeming Li
It is especially challenging to achieve real-time human motion tracking on a standalone VR Head-Mounted Display (HMD) such as Meta Quest and PICO.
no code implementations • 5 Mar 2024 • Feng Hou, Jin Yuan, Ying Yang, Yang Liu, Yang Zhang, Cheng Zhong, Zhongchao shi, Jianping Fan, Yong Rui, Zhiqiang He
With the recent advance of vision-language models (VLMs), viewed as natural source models, the cross-domain task changes to directly adapt the pre-trained source model to arbitrary target domains equipped with prior domain knowledge, and we name this task Adaptive Domain Generalization (ADG).
no code implementations • 5 Mar 2024 • Hanlei Jin, Yang Zhang, Dan Meng, Jun Wang, Jinghua Tan
Automatic Text Summarization (ATS), utilizing Natural Language Processing (NLP) algorithms, aims to create concise and accurate summaries, thereby significantly reducing the human effort required in processing large volumes of text.
1 code implementation • 29 Feb 2024 • Wentao Shi, Xiangnan He, Yang Zhang, Chongming Gao, Xinyue Li, Jizhi Zhang, Qifan Wang, Fuli Feng
To this end, we propose a Bi-level Learnable LLM Planner framework, which consists of a set of LLM instances and breaks down the learning process into macro-learning and micro-learning to learn macro-level guidance and micro-level personalized recommendation policies, respectively.
no code implementations • 29 Feb 2024 • Wentao Shi, Chenxu Wang, Fuli Feng, Yang Zhang, Wenjie Wang, Junkang Wu, Xiangnan He
Compared to AUC, LLPAUC considers only the partial area under the ROC curve in the Lower-Left corner to push the optimization focus on Top-K. We provide theoretical validation of the correlation between LLPAUC and Top-K ranking metrics and demonstrate its robustness to noisy user feedback.
1 code implementation • 28 Feb 2024 • Jizhi Zhang, Keqin Bao, Wenjie Wang, Yang Zhang, Wentao Shi, Wanhong Xu, Fuli Feng, Tat-Seng Chua
Additionally, we prospect the evolution of Rec4Agentverse and conceptualize it into three stages based on the enhancement of the interaction and information exchange among Agent Items, Agent Recommender, and the user.
1 code implementation • 27 Feb 2024 • Guodong Sun, Yuting Peng, Le Cheng, Mengya Xu, An Wang, Bo Wu, Hongliang Ren, Yang Zhang
The precise segmentation of ore images is critical to the successful execution of the beneficiation process.
1 code implementation • 25 Feb 2024 • Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani, Yang Zhang, Eric Wong, Shiyu Chang
Aligned large language models (LLMs) are vulnerable to jailbreaking attacks, which bypass the safeguards of targeted LLMs and fool them into generating objectionable content.
no code implementations • 21 Feb 2024 • Yang Zhang, Haiyang Wu, Yuekui Yang
Comprehensive study on FlexHB shows that (1) our fine-grained fidelity method considerably enhances the efficiency of searching optimal configurations, (2) our FlexBand framework (self-adaptive allocation of SH brackets, and global ranking of configurations in both current and past SH procedures) grants the algorithm with more flexibility and improves the anytime performance.
no code implementations • 21 Feb 2024 • Jiahua Wan, Hong Ren, Zhiyuan Yu, Zhenkun Zhang, Yang Zhang, Cunhua Pan, Jiangzhou Wang
To address the formulated non-convex problem, we propose an algorithm based on the block coordinate descent (BCD) method to decouple the original problem into two subproblems, where the computational and beamforming settings are optimized alternately.
no code implementations • 20 Feb 2024 • Zeyang Sha, Yang Zhang
Our proposed prompt stealing attack aims to steal these well-designed prompts based on the generated answers.
1 code implementation • 20 Feb 2024 • Yan Pang, Yang Zhang, Tianhao Wang
Together with fake video detection and tracing, our multi-faceted set of solutions can effectively mitigate misuse of video generative models.
no code implementations • 19 Feb 2024 • Xinyu Hu, Mingqi Gao, Sen Hu, Yang Zhang, Yicheng Chen, Teng Xu, Xiaojun Wan
Some prior work has shown that LLMs perform well in NLG evaluation for different tasks.
1 code implementation • 14 Feb 2024 • Rui Zhang, Hongwei Li, Rui Wen, Wenbo Jiang, Yuan Zhang, Michael Backes, Yun Shen, Yang Zhang
The increasing demand for customized Large Language Models (LLMs) has led to the development of solutions like GPTs.
1 code implementation • 14 Feb 2024 • Pengfei Zhou, Weiqing Min, Jiajun Song, Yang Zhang, Shuqiang Jiang
The complexity of food semantic attributes further makes it more difficult for current ZSD methods to distinguish various food categories.
1 code implementation • 13 Feb 2024 • Yongchao Chen, Jacob Arkin, Yilun Hao, Yang Zhang, Nicholas Roy, Chuchu Fan
Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task.
1 code implementation • 8 Feb 2024 • Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang Li, Yang Zhang, Shouda Liu, Jie Luo, Xianglong Liu, Michele Magno
This paper proposes a novel IR-QLoRA for pushing quantized LLMs with LoRA to be highly accurate through information retention.
no code implementations • 8 Feb 2024 • Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, Yang Zhang
Some jailbreak prompt datasets, available from the Internet, can also achieve high attack success rates on many LLMs, such as ChatGLM3, GPT-3. 5, and PaLM2.
no code implementations • 5 Feb 2024 • Haibo Jin, Ruoxi Chen, Andy Zhou, Jinyin Chen, Yang Zhang, Haohan Wang
Our system of different roles will leverage this knowledge graph to generate new jailbreaks, which have proved effective in inducing LLMs to generate unethical or guideline-violating responses.
no code implementations • 5 Feb 2024 • Junjie Chu, Zeyang Sha, Michael Backes, Yang Zhang
We then introduce two advanced attacks aimed at better reconstructing previous conversations, specifically the UNR attack and the PBU attack.
no code implementations • 3 Feb 2024 • Yang Zhang, Hong Ren, Cunhua Pan, Boshi Wang, Zhiyuan Yu, Ruisong Weng, Tuo Wu, Yongchao He
This work considers a dual-functional radar and communication (DFRC) system with an active reconfigurable intelligent surface (RIS) and a potential eavesdropper.
2 code implementations • 29 Jan 2024 • Moyun Liu, Bing Chen, Youping Chen, Jingming Xie, Lei Yao, Yang Zhang, Joey Tianyi Zhou
Depth completion is a crucial task in autonomous driving, aiming to convert a sparse depth map into a dense depth prediction.
1 code implementation • 28 Jan 2024 • Yujian Liu, Jiabao Ji, Tong Yu, Ryan Rossi, Sungchul Kim, Handong Zhao, Ritwik Sinha, Yang Zhang, Shiyu Chang
Table question answering is a popular task that assesses a model's ability to understand and interact with structured data.
no code implementations • 7 Jan 2024 • Haonan Wang, Qianli Shen, Yao Tong, Yang Zhang, Kenji Kawaguchi
Our method strategically embeds connections between pieces of copyrighted information and text references in poisoning data while carefully dispersing that information, making the poisoning data inconspicuous when integrated into a clean dataset.
no code implementations • 25 Dec 2023 • Tianhao Shi, Yang Zhang, Zhijian Xu, Chong Chen, Fuli Feng, Xiangnan He, Qi Tian
Rather than directly dismissing the role of incremental learning, we ascribe this lack of anticipated performance improvement to the mismatch between the LLM4Recarchitecture and incremental learning: LLM4Rec employs a single adaptation module for learning recommendation, hampering its ability to simultaneously capture long-term and short-term user preferences in the incremental learning context.
no code implementations • 18 Dec 2023 • Yiting Qu, Zhikun Zhang, Yun Shen, Michael Backes, Yang Zhang
Take the open-world attribution as an example, FAKEPCD attributes point clouds to known sources with an accuracy of 0. 82-0. 98 and to unknown sources with an accuracy of 0. 73-1. 00.
no code implementations • 18 Dec 2023 • Tianyao He, Huabin Liu, Yuxi Li, Xiao Ma, Cheng Zhong, Yang Zhang, Weiyao Lin
Our framework comprises two core modules: collaborative step mining and frame-to-step alignment.
1 code implementation • 18 Dec 2023 • Yimeng Bai, Yang Zhang, Jing Lu, Jianxin Chang, Xiaoxue Zang, Yanan Niu, Yang song, Fuli Feng
Through meta-learning techniques, LabelCraft effectively addresses the bi-level optimization hurdle posed by the recommender and labeling models, enabling the automatic acquisition of intricate label generation mechanisms. Extensive experiments on real-world datasets corroborate LabelCraft's excellence across varied operational metrics, encompassing usage time, user engagement, and retention.
no code implementations • 14 Dec 2023 • Yi Guo, Yiqian He, Xiaoyang Li, Haotong Qin, Van Tung Pham, Yang Zhang, Shouda Liu
Knowledge Distillation (KD) emerges as one of the most promising compression technologies to run advanced deep neural networks on resource-limited devices.
1 code implementation • 14 Dec 2023 • Hao Shao, Yang Zhang, Qibin Hou
We present a new boundary sensitive framework for polyp segmentation, called Polyper.
1 code implementation • 10 Dec 2023 • Yujian Liu, Yang Zhang, Tommi Jaakkola, Shiyu Chang
Despite diffusion models' superior capabilities in modeling complex distributions, there are still non-trivial distributional discrepancies between generated and ground-truth images, which has resulted in several notable problems in image generation, including missing object errors in text-to-image generation and low image quality.
1 code implementation • 10 Dec 2023 • Yang Zhang, Huilin Pan, Mingying Li, An Wang, Yang Zhou, Hongliang Ren
Existing modeling shortcomings of spatial invariance and pooling layers in conventional CNNs often ignore the neglect of crucial global information, resulting in error localization for fault objection tasks of freight trains.
no code implementations • 2 Dec 2023 • Qiang Li, Xiaoyan Yang, Haowen Wang, Qin Wang, Lei Liu, Junjie Wang, Yang Zhang, Mingyuan Chu, Sen Hu, Yicheng Chen, Yue Shen, Cong Fan, Wangshu Zhang, Teng Xu, Jinjie Gu, Jing Zheng, Guannan Zhang Ant Group
(3) Specifically for multi-choice questions in the medical domain, we propose a novel Verification-of-Choice approach for prompting engineering, which significantly enhances the reasoning ability of LLMs.
no code implementations • 1 Dec 2023 • Yaoyao Zhong, Mengshi Qi, Rui Wang, Yuhan Qiu, Yang Zhang, Huadong Ma
Video Internet of Things (VIoT) has shown full potential in collecting an unprecedented volume of video data.
1 code implementation • 15 Nov 2023 • Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, Yang Zhang
Uncertainty decomposition refers to the task of decomposing the total uncertainty of a model into data (aleatoric) uncertainty, resulting from the inherent complexity or ambiguity of the data, and model (epistemic) uncertainty, resulting from the lack of knowledge in the model.
1 code implementation • 10 Nov 2023 • Guodong Sun, Delong Huang, Yuting Peng, Le Cheng, Bo Wu, Yang Zhang
At the same time, the ore distribution is stacked, and it is difficult to identify the complete features.
no code implementations • 3 Nov 2023 • Yang Zhang
For this purpose, we introduce the Cross-Modal Prominent Fragments Enhancement Aligning Network(CPFEAN), which achieves improved retrieval accuracy by diminishing the participation of irrelevant regions during alignment and relatively increasing the alignment similarity of prominent words.
no code implementations • 3 Nov 2023 • Boyang Zhang, Xinyue Shen, Wai Man Si, Zeyang Sha, Zeyuan Chen, Ahmed Salem, Yun Shen, Michael Backes, Yang Zhang
Moderating offensive, hateful, and toxic language has always been an important but challenging topic in the domain of safe use in NLP.
no code implementations • 30 Oct 2023 • Zhaowei Gao, Mingyang Song, Christopher Schroers, Yang Zhang
Our proposed method supports bidirectional spatio-temporal information propagation across multiple scales to leverage information in both space and time.
1 code implementation • 30 Oct 2023 • Minxing Zhang, Ning Yu, Rui Wen, Michael Backes, Yang Zhang
Several membership inference attacks (MIAs) have been proposed to exhibit the privacy vulnerability of generative models by classifying a query image as a training dataset member or nonmember.
1 code implementation • 30 Oct 2023 • Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, Xiangnan He
In pursuit of superior recommendations for both cold and warm start scenarios, we introduce CoLLM, an innovative LLMRec methodology that seamlessly incorporates collaborative information into LLMs for recommendation.
no code implementations • 27 Oct 2023 • Jinbo Wen, Jiangtian Nie, Jiawen Kang, Dusit Niyato, Hongyang Du, Yang Zhang, Mohsen Guizani
Generative Artificial Intelligence (GAI) possesses the capabilities of generating realistic data and facilitating advanced decision-making.
no code implementations • 18 Oct 2023 • Yuanyuan Wang, Yang Zhang, Zhiyong Wu, Zhihan Yang, Tao Wei, Kun Zou, Helen Meng
Existing augmentation methods for speaker verification manipulate the raw signal, which are time-consuming and the augmented samples lack diversity.
no code implementations • 17 Oct 2023 • Rui Wen, Tianhao Wang, Michael Backes, Yang Zhang, Ahmed Salem
Large Language Models (LLMs) are powerful tools for natural language processing, enabling novel applications and user experiences.
no code implementations • 16 Oct 2023 • Joann Qiongna Chen, Xinlei He, Zheng Li, Yang Zhang, Zhou Li
Training a machine learning model with data following a meaningful order, i. e., from easy to hard, has been proven to be effective in accelerating the training process and achieving better model performance.
no code implementations • 12 Oct 2023 • Jianchao Lu, Yuzhe Tian, Yang Zhang, Jiaqi Ge, Quan Z. Sheng, Xi Zheng
The efficiency, assessed on two public EEG datasets and two real-world EEG devices, significantly outperforms the state-of-the-art solution in accuracy ($82. 54\%$ versus $62. 22\%$) with fewer parameters (64. 9M compared to 183. 7M).
no code implementations • 11 Oct 2023 • Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David Cox, David Harwath, Yang Zhang, Karen Livescu, James Glass
We study phrase structure induction from visually-grounded speech.
1 code implementation • 11 Oct 2023 • Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, Yang Zhang
Such a Composite Backdoor Attack (CBA) is shown to be stealthier than implanting the same multiple trigger keys in only a single component.
no code implementations • 11 Oct 2023 • Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, Yang Zhang
Specifically, the VPPTaaS provider optimizes a visual prompt given downstream data, and downstream users can use this prompt together with the large pre-trained model for prediction.
no code implementations • 10 Oct 2023 • Yang Zhang, Yawei Li, Hannah Brown, Mina Rezaei, Bernd Bischl, Philip Torr, Ashkan Khakzar, Kenji Kawaguchi
Feature attribution explains neural network outputs by identifying relevant input features.
1 code implementation • 7 Oct 2023 • Pengfei Zhou, Weiqing Min, Yang Zhang, Jiajun Song, Ying Jin, Shuqiang Jiang
To tackle this, we propose the Semantic Separable Diffusion Synthesizer (SeeDS) framework for Zero-Shot Food Detection (ZSFD).
Ranked #1 on Generalized Zero-Shot Object Detection on MS-COCO
no code implementations • 23 Sep 2023 • Yang Zhang, Travis M. Bartley, Mariana Graterol-Fuenmayor, Vitaly Lavrukhin, Evelina Bakhturina, Boris Ginsburg
Through this new framework, we can identify strengths and weaknesses of GPT-based TN, opening opportunities for future work.
no code implementations • 18 Sep 2023 • Yang Zhang, YuFei Wang, Kai Wang, Quan Z. Sheng, Lina Yao, Adnan Mahmood, Wei Emma Zhang, Rongying Zhao
Such information could be incorporated into LLMs pre-training and improve the text representation in LLMs.
no code implementations • 15 Sep 2023 • Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, Haonan Wang, Kenji Kawaguchi
Specifically, we introduce a data generation pipeline to systematically produce data for studying copyright in diffusion models.
1 code implementation • 20 Aug 2023 • Jinghao Xin, Zhi Li, Yang Zhang, Ning li
Particle Swarm Optimization (PSO) has demonstrated efficacy in addressing static path planning problems.
no code implementations • 17 Aug 2023 • Yawei Li, Yang Zhang, Kenji Kawaguchi, Ashkan Khakzar, Bernd Bischl, Mina Rezaei
We apply these metrics to mainstream attribution methods, offering a novel lens through which to analyze and compare feature attribution methods.
1 code implementation • 16 Aug 2023 • Keqin Bao, Jizhi Zhang, Wenjie Wang, Yang Zhang, Zhengyi Yang, Yancheng Luo, Chong Chen, Fuli Feng, Qi Tian
As the focus on Large Language Models (LLMs) in the field of recommendation intensifies, the optimization of LLMs for recommendation purposes (referred to as LLM4Rec) assumes a crucial role in augmenting their effectiveness in providing recommendations.
no code implementations • 14 Aug 2023 • An Wang, Mobarakol Islam, Mengya Xu, Yang Zhang, Hongliang Ren
Our extensive evaluation results reveal that although SAM shows remarkable zero-shot generalization ability with bounding box prompts, it struggles to segment the whole instrument with point-based prompts and unprompted settings.
1 code implementation • 11 Aug 2023 • Yang Zhang, Chenyun Xiong, Junjie Liu, Xuhui Ye, Guodong Sun
Efficient RGB-D semantic segmentation has received considerable attention in mobile robots, which plays a vital role in analyzing and recognizing environmental information.
Ranked #57 on Semantic Segmentation on NYU Depth v2
1 code implementation • 10 Aug 2023 • Xinlei He, Savvas Zannettou, Yun Shen, Yang Zhang
We find that prompt learning achieves around 10\% improvement in the toxicity classification task compared to the baselines, while for the toxic span detection task we find better performance to the best baseline (0. 643 vs. 0. 640 in terms of $F_1$-score).
2 code implementations • 9 Aug 2023 • Yang Zhang, Krishna C. Puvvada, Vitaly Lavrukhin, Boris Ginsburg
We propose CONF-TSASR, a non-autoregressive end-to-end time-frequency domain architecture for single-channel target-speaker automatic speech recognition (TS-ASR).
no code implementations • 7 Aug 2023 • Wai Man Si, Michael Backes, Yang Zhang
In this paper, we discover a new attack strategy against LLM APIs, namely the prompt abstraction attack.
1 code implementation • 7 Aug 2023 • Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, Yang Zhang
We hope that our study can facilitate the research community and LLM vendors in promoting safer and regulated LLMs.
no code implementations • 4 Aug 2023 • Munazza Zaib, Wei Emma Zhang, Quan Z. Sheng, Subhash Sagar, Adnan Mahmood, Yang Zhang
In this paper, we propose a framework, DHS-ConvQA (Dynamic History Selection in Conversational Question Answering), that first generates the context and question entities for all the history turns, which are then pruned on the basis of similarity they share in common with the question at hand.
no code implementations • 29 Jul 2023 • Jiawen Kang, Jinbo Wen, Dongdong Ye, Bingkun Lai, Tianhao Wu, Zehui Xiong, Jiangtian Nie, Dusit Niyato, Yang Zhang, Shengli Xie
Given the revolutionary role of metaverses, healthcare metaverses are emerging as a transformative force, creating intelligent healthcare systems that offer immersive and personalized services.
1 code implementation • 23 Jul 2023 • Yuzhao Mao, Di Lu, Xiaojie Wang, Yang Zhang
This paper concentrates on the understanding of interlocutors' emotions evoked in conversational utterances.
Ranked #17 on Emotion Recognition in Conversation on IEMOCAP
1 code implementation • 14 Jul 2023 • Zhen Zhang, Guanhua Zhang, Bairu Hou, Wenqi Fan, Qing Li, Sijia Liu, Yang Zhang, Shiyu Chang
This largely falls into the study of certified robust LLMs, i. e., all predictions of LLM are certified to be correct in a local region around the input.
no code implementations • 11 Jul 2023 • Chen Chen, YuFei Wang, Yang Zhang, Quan Z. Sheng, Kwok-Yan Lam
Previous KGC methods typically represent knowledge graph entities and relations as trainable continuous embeddings and fuse the embeddings of the entity $h$ (or $t$) and relation $r$ into hidden representations of query $(h, r, ?
no code implementations • 10 Jul 2023 • Yoshua Bengio, Prateek Gupta, Lu Li, Soham Phade, Sunil Srinivasa, Andrew Williams, Tianyu Zhang, Yang Zhang, Stephan Zheng
On the other hand, an interdisciplinary panel of human experts in law, policy, sociology, economics and environmental science, evaluated the solutions qualitatively.
no code implementations • 5 Jul 2023 • Yang Zhang, Zhiyu Hu, Yimeng Bai, Fuli Feng, Jiancan Wu, Qifan Wang, Xiangnan He
In this work, we propose an Influence Function-based Recommendation Unlearning (IFRU) framework, which efficiently updates the model without retraining by estimating the influence of the unusable data on the model via the influence function.
no code implementations • 3 Jul 2023 • Yang Zhang, Huilin Pan, Yang Zhou, Mingying Li, Guodong Sun
Efficient visual fault detection of freight trains is a critical part of ensuring the safe operation of railways under the restricted hardware environment.
no code implementations • 30 Jun 2023 • Yang Zhang, Yimeng Bai, Jianxin Chang, Xiaoxue Zang, Song Lu, Jing Lu, Fuli Feng, Yanan Niu, Yang song
With the proliferation of short video applications, the significance of short video recommendations has vastly increased.
no code implementations • 23 Jun 2023 • Zhongzhi Yu, Yang Zhang, Kaizhi Qian, Yonggan Fu, Yingyan Lin
Despite the impressive performance recently achieved by automatic speech recognition (ASR), we observe two primary challenges that hinder its broader applications: (1) The difficulty of introducing scalability into the model to support more languages with limited training, inference, and storage overhead; (2) The low-resource adaptation ability that enables effective low-resource adaptation while avoiding over-fitting and catastrophic forgetting issues.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
1 code implementation • 13 Jun 2023 • Yihan Ma, Zhikun Zhang, Ning Yu, Xinlei He, Michael Backes, Yun Shen, Yang Zhang
Graph generative models become increasingly effective for data distribution approximation and data augmentation.
no code implementations • 13 Jun 2023 • Yihan Ma, Zhengyu Zhao, Xinlei He, Zheng Li, Michael Backes, Yang Zhang
In particular, to help the watermark survive the subject-driven synthesis, we incorporate the synthesis process in learning GenWatermark by fine-tuning the detector with synthesized images for a specific subject.
3 code implementations • 10 Jun 2023 • Yongchao Chen, Jacob Arkin, Charles Dawson, Yang Zhang, Nicholas Roy, Chuchu Fan
Rather than using LLMs to directly plan task sub-goals, we instead perform few-shot translation from natural language task descriptions to an intermediate task representation that can then be consumed by a TAMP algorithm to jointly solve the task and motion plan.
1 code implementation • 1 Jun 2023 • An Wang, Mengya Xu, Yang Zhang, Mobarakol Islam, Hongliang Ren
Furthermore, to produce reliable mixed pseudo labels, which enhance the effectiveness of ensemble learning, we introduce a novel adaptive pixel-wise fusion technique based on the entropy guidance from the spatial and spectral branches.
no code implementations • 30 May 2023 • Yang Zhang, Lingbo Liu, Xinyu Xiong, Guanbin Li, Guoli Wang, Liang Lin
In this work, we propose a novel end-to-end wind power forecasting model named Hierarchical Spatial-Temporal Transformer Network (HSTTN) to address the long-term WPF problems.
no code implementations • 30 May 2023 • Jin Yuan, Yang Zhang, Yangzhou Du, Zhongchao shi, Xin Geng, Jianping Fan, Yong Rui
In this paper, a novel Epistemic Graph Layer (EGLayer) is introduced to enable hybrid learning, enhancing the exchange of information between deep features and a structured knowledge graph.
1 code implementation • 28 May 2023 • Kai Mei, Zheng Li, Zhenting Wang, Yang Zhang, Shiqing Ma
Such attacks can be easily affected by retraining on downstream tasks and with different prompting strategies, limiting the transferability of backdoor attacks.
no code implementations • 25 May 2023 • Linfeng Liang, Yao Deng, Yang Zhang, Jianchao Lu, Chen Wang, Quanzheng Sheng, Xi Zheng
Discrepancies in decision-making between Autonomous Driving Systems (ADS) and human drivers underscore the need for intuitive human gaze predictors to bridge this gap, thereby improving user trust and experience.
1 code implementation • 23 May 2023 • Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, Yang Zhang
Our evaluation result shows that 24% of the generated images using DreamBooth are hateful meme variants that present the features of the original hateful meme and the target individual/community; these generated images are comparable to hateful meme variants collected from the real world.
1 code implementation • 22 May 2023 • Zhenrui Yue, Huimin Zeng, Yang Zhang, Lanyu Shang, Dong Wang
As such, MetaAdapt can learn how to adapt the misinformation detection model and exploit the source data for improved performance in the target domain.
no code implementations • 18 May 2023 • Peihua Ma, Yixin Wu, Ning Yu, Yang Zhang, Michael Backes, Qin Wang, Cheng-I Wei
Nutrition information is crucial in precision nutrition and the food industry.
1 code implementation • 12 May 2023 • Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, Xiangnan He
The remarkable achievements of Large Language Models (LLMs) have led to the emergence of a novel recommendation paradigm -- Recommendation via LLM (RecLLM).
no code implementations • 12 May 2023 • Wai Man Si, Michael Backes, Yang Zhang, Ahmed Salem
In this work, we broaden the scope of this attack to include text generation and classification models, hence showing its broader applicability.
3 code implementations • 12 May 2023 • Yongchao Chen, Rujul Gandhi, Yang Zhang, Chuchu Fan
Then, we finetune T5 models on the lifted versions (i. e., the specific Atomic Propositions (AP) are hidden) of the NL and TL.
1 code implementation • 2 May 2023 • Yang Zhang, Le Cheng, Yuting Peng, Chengming Xu, Yanwei Fu, Bo Wu, Guodong Sun
For the ore particle size detection, obtaining a sizable amount of high-quality ore labeled data is time-consuming and expensive.
1 code implementation • 30 Apr 2023 • Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, Xiangnan He
We have demonstrated that the proposed TALLRec framework can significantly enhance the recommendation capabilities of LLMs in the movie and book domains, even with a limited dataset of fewer than 100 samples.
no code implementations • 28 Apr 2023 • An Wang, Mobarakol Islam, Mengya Xu, Yang Zhang, Hongliang Ren
In this empirical study, we investigate the robustness and zero-shot generalizability of the SAM in the domain of robotic surgery in various settings of (i) prompted vs. unprompted; (ii) bounding box vs. points-based prompt; (iii) generalization under corruptions and perturbations with five severity levels; and (iv) state-of-the-art supervised model vs. SAM.
1 code implementation • 27 Apr 2023 • Yulong Huang, Yang Zhang, Qifan Wang, Chenxu Wang, Fuli Feng
To improve the accuracy of these models, some researchers have attempted to simulate human analogical reasoning to correct predictions for testing data by drawing analogies with the prediction errors of similar training data.
1 code implementation • 26 Apr 2023 • Yang Zhang, Tianhao Shi, Fuli Feng, Wenjie Wang, Dingxian Wang, Xiangnan He, Yongdong Zhang
However, such a manner inevitably learns unstable feature interactions, i. e., the ones that exhibit strong correlations in historical data but generalize poorly for future serving.
no code implementations • 18 Apr 2023 • Xinyue Shen, Zeyuan Chen, Michael Backes, Yang Zhang
In this paper, we perform the first large-scale measurement of ChatGPT's reliability in the generic QA scenario with a carefully curated set of 5, 695 questions across ten datasets and eight domains.
1 code implementation • 11 Apr 2023 • Shu Chen, Junyao Li, Yang Zhang, Beiji Zou
Through these explicit constraints and the implicit constraint from NeRF, our method improves the view synthesis as well as the 3D-scene geometry performance of NeRF at the same time.
1 code implementation • ICCV 2023 • Qiucheng Wu, Yujian Liu, Handong Zhao, Trung Bui, Zhe Lin, Yang Zhang, Shiyu Chang
We then impose spatial attention control by combining the attention over the entire text description and that over the local description of the particular object in the corresponding pixel region of that object.
1 code implementation • 6 Apr 2023 • Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, Shiyu Chang
COPAINT also uses the Bayesian framework to jointly modify both revealed and unrevealed regions, but approximates the posterior distribution in a way that allows the errors to gradually drop to zero throughout the denoising steps, thus strongly penalizing any mismatches with the reference image.
2 code implementations • 5 Apr 2023 • Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Yang Zhang
Few-shot-based facial recognition systems have gained increasing attention due to their scalability and ability to work with a few face images during the model deployment phase.
2 code implementations • 26 Mar 2023 • Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, Yang Zhang
Extensive evaluations on public datasets with curated texts generated by various powerful LLMs such as ChatGPT-turbo and Claude demonstrate the effectiveness of different detection methods.
1 code implementation • 17 Mar 2023 • Dongsheng Wang, Xu Jia, Yang Zhang, Xinyu Zhang, Yaoyuan Wang, Ziyang Zhang, Dong Wang, Huchuan Lu
To fully exploit information with event streams to detect objects, a dual-memory aggregation network (DMANet) is proposed to leverage both long and short memory along event streams to aggregate effective information for object detection.
no code implementations • 16 Mar 2023 • Mingyang Song, Yang Zhang, Tunç O. Aydın, Elham Amin Mansour, Christopher Schroers
To this end, we propose an effective generative model which utilizes clean features as guidance followed by noise injections into the network.
no code implementations • 9 Mar 2023 • Ziqing Yang, Zeyang Sha, Michael Backes, Yang Zhang
In this sense, we propose SeMap, a more effective mapping using the semantic alignment between the pre-trained model's knowledge and the downstream task.
1 code implementation • 23 Feb 2023 • Boyang Zhang, Xinlei He, Yun Shen, Tianhao Wang, Yang Zhang
Given the simplicity and effectiveness of the attack method, our study indicates scientific plots indeed constitute a valid side channel for model information stealing attacks.
1 code implementation • 23 Feb 2023 • Yang Zhang, Wenbing Huang, Zhewei Wei, Ye Yuan, Zhaohan Ding
Predicting the binding sites of the target proteins plays a fundamental role in drug discovery.
1 code implementation • 20 Feb 2023 • Xinyue Shen, Yiting Qu, Michael Backes, Yang Zhang
In this paper, we perform the first study on understanding the threat of a novel attack, namely prompt stealing attack, which aims to steal prompts from generated images by text-to-image generation models.
1 code implementation • 20 Feb 2023 • Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, Yang Zhang
At first, a top-n selection strategy is proposed to provide pseudo-labels for public data, and use pseudo-labels to guide the training of the cGAN.
2 code implementations • 3 Jan 2023 • Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, Yang Zhang
A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset.
1 code implementation • 19 Dec 2022 • Bairu Hou, Jinghan Jia, Yihua Zhang, Guanhua Zhang, Yang Zhang, Sijia Liu, Shiyu Chang
Robustness evaluation against adversarial examples has become increasingly important to unveil the trustworthiness of the prevailing deep models in natural language processing (NLP).
2 code implementations • 19 Dec 2022 • Bairu Hou, Joe O'Connor, Jacob Andreas, Shiyu Chang, Yang Zhang
Instead of directly optimizing in prompt space, PromptBoosting obtains a small pool of prompts via a gradient-free approach and then constructs a large pool of weak learners by pairing these prompts with different elements of the LM's output distribution.
no code implementations • 18 Dec 2022 • Zeyang Sha, Xinlei He, Pascal Berrang, Mathias Humbert, Yang Zhang
Backdoor attacks represent one of the major threats to machine learning models.
1 code implementation • CVPR 2023 • Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, Shiyu Chang
Based on this finding, we further propose a simple, light-weight image editing algorithm where the mixing weights of the two text embeddings are optimized for style matching and content preservation.
2 code implementations • 13 Dec 2022 • Yiting Qu, Xinlei He, Shannon Pierson, Michael Backes, Yang Zhang, Savvas Zannettou
The dissemination of hateful memes online has adverse effects on social media platforms and the real world.
1 code implementation • 8 Dec 2022 • Xinyu Lin, Yiyan Xu, Wenjie Wang, Yang Zhang, Fuli Feng
This objective requires to 1) automatically mask spurious features without supervision, and 2) block the negative effect transmission from spurious features to other features during SSL.
no code implementations • 3 Dec 2022 • Jiangcong Liu, Hao Ma, Yun Guan, Fan Wu, Le Xu, Yang Zhang, Lixia Tian
We evaluated the effectiveness of AINS with both statistical and predictive analyses on individual differences in sex and intelligence quotient (IQ), based on the four movie fMRI runs included in the Human Connectome Project dataset.
no code implementations • 26 Nov 2022 • Yang Zhang, Yang Zhou, Huilin Pan, Bo Wu, Guodong Sun
Fault detection for key components in the braking system of freight trains is critical for ensuring railway transportation safety.
1 code implementation • 13 Nov 2022 • Haotong Qin, Xudong Ma, Yifu Ding, Xiaoyang Li, Yang Zhang, Zejun Ma, Jiakai Wang, Jie Luo, Xianglong Liu
We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25. 1x speedup and 20. 2x storage-saving on edge hardware.
no code implementations • 4 Nov 2022 • Feng Hou, Yao Zhang, Yang Liu, Jin Yuan, Cheng Zhong, Yang Zhang, Zhongchao shi, Jianping Fan, Zhiqiang He
Due to domain shift, deep neural networks (DNNs) usually fail to generalize well on unknown test data in practice.
1 code implementation • CVPR 2023 • Yang Liu, Yao Zhang, Yixin Wang, Yang Zhang, Jiang Tian, Zhongchao shi, Jianping Fan, Zhiqiang He
To bridge the gap between the reference points of salient queries and Transformer detectors, we propose SAlient Point-based DETR (SAP-DETR) by treating object detection as a transformation from salient points to instance objects.
1 code implementation • 2 Nov 2022 • Yonggan Fu, Yang Zhang, Kaizhi Qian, Zhifan Ye, Zhongzhi Yu, Cheng-I Lai, Yingyan Lin
We believe S$^3$-Router has provided a new perspective for practical deployment of speech SSL models.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
1 code implementation • 1 Nov 2022 • Yufei Chen, Chao Shen, Yun Shen, Cong Wang, Yang Zhang
In this paper, we investigate the third type of exploitation of data poisoning - increasing the risks of privacy leakage of benign training samples.
no code implementations • 13 Oct 2022 • Zeyang Sha, Zheng Li, Ning Yu, Yang Zhang
To tackle this problem, we pioneer a systematic study on the detection and attribution of fake images generated by text-to-image generation models.
no code implementations • 6 Oct 2022 • Huimin Zeng, Zhenrui Yue, Ziyi Kou, Lanyu Shang, Yang Zhang, Dong Wang
Moreover, we leverage the power of domain adversarial examples to establish an intermediate domain mixup, where the latent representations of the input text from both domains could be mixed during the training process.
no code implementations • 4 Oct 2022 • Xinyue Shen, Xinlei He, Zheng Li, Yun Shen, Michael Backes, Yang Zhang
Different from previous work, we are the first to systematically threat modeling on SSL in every phase of the model supply chain, i. e., pre-training, release, and downstream phases.
no code implementations • 3 Oct 2022 • Huimin Zeng, Zhenrui Yue, Yang Zhang, Ziyi Kou, Lanyu Shang, Dong Wang
In many applications with real-world consequences, it is crucial to develop reliable uncertainty estimation for the predictions made by the AI decision systems.
1 code implementation • 3 Oct 2022 • Zheng Li, Ning Yu, Ahmed Salem, Michael Backes, Mario Fritz, Yang Zhang
Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods.
no code implementations • 3 Oct 2022 • Yixin Wu, Ning Yu, Zheng Li, Michael Backes, Yang Zhang
The empirical results show that all of the proposed attacks can achieve significant performance, in some cases even close to an accuracy of 1, and thus the corresponding risk is much more severe than that shown by existing membership inference attacks.
1 code implementation • 1 Oct 2022 • Shu Chen, Yang Zhang, Yaxin Xu, Beiji Zou
This two-stage strategy is not convenient to use and degrades the performance because the error in the pose extraction can propagate to the view synthesis.
1 code implementation • 30 Sep 2022 • Ziqing Yang, Xinlei He, Zheng Li, Michael Backes, Mathias Humbert, Pascal Berrang, Yang Zhang
Extensive evaluations on different datasets and model architectures show that all three attacks can achieve significant attack performance while maintaining model utility in both visual and linguistic modalities.
1 code implementation • 22 Sep 2022 • Chenxu Wang, Fuli Feng, Yang Zhang, Qifan Wang, Xunhan Hu, Xiangnan He
A standard choice is treating the missing data as negative training samples and estimating interaction likelihood between user-item pairs along with the observed interactions.
1 code implementation • 21 Sep 2022 • Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu, Shiyu Chang
Specifically, FairReprogram considers the case where models can not be changed and appends to the input a set of perturbations, called the fairness trigger, which is tuned towards the fairness criteria under a min-max formulation.
no code implementations • 18 Sep 2022 • Yang Zhang, Gengmo Zhou, Zhewei Wei, Hongteng Xu
The prediction of protein-ligand binding affinity is of great significance for discovering lead compounds in drug research.
no code implementations • 7 Sep 2022 • Wai Man Si, Michael Backes, Jeremy Blackburn, Emiliano De Cristofaro, Gianluca Stringhini, Savvas Zannettou, Yang Zhang
We show that publicly available chatbots are prone to providing toxic responses when fed toxic queries.
1 code implementation • 4 Sep 2022 • Hai Huang, Zhikun Zhang, Yun Shen, Michael Backes, Qi Li, Yang Zhang
Existing studies on neural architecture search (NAS) mainly focus on efficiently and effectively searching for network architectures with better performance.
1 code implementation • 31 Aug 2022 • Yiyong Liu, Zhengyu Zhao, Michael Backes, Yang Zhang
Machine learning models are vulnerable to membership inference attacks in which an adversary aims to predict whether or not a particular sample was contained in the target model's training dataset.
no code implementations • 23 Aug 2022 • Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, Yang Zhang
Furthermore, we propose a hybrid attack that exploits the exit information to improve the performance of existing attacks.
no code implementations • 22 Aug 2022 • Xinlei He, Zheng Li, Weilin Xu, Cory Cornelius, Yang Zhang
Finally, we find that data augmentation degrades the performance of existing attacks to a larger extent, and we propose an adaptive attack using augmentation to train shadow and attack models that improve attack performance.
no code implementations • 22 Aug 2022 • Yang Zhang
In addition, extending the obstacle avoidance algorithms to more competitive situations.
2 code implementations • 15 Aug 2022 • Tianyu Zhang, Andrew Williams, Soham Phade, Sunil Srinivasa, Yang Zhang, Prateek Gupta, Yoshua Bengio, Stephan Zheng
To facilitate this research, here we introduce RICE-N, a multi-region integrated assessment model that simulates the global climate and economy, and which can be used to design and evaluate the strategic outcomes for different negotiation and agreement frameworks.
1 code implementation • 25 Jul 2022 • Xinlei He, Hongbin Liu, Neil Zhenqiang Gong, Yang Zhang
The results show that early stopping can mitigate the membership inference attack, but with the cost of model's utility degradation.
no code implementations • 18 Jun 2022 • Haibin Wu, Jiawen Kang, Lingwei Meng, Yang Zhang, Xixin Wu, Zhiyong Wu, Hung-Yi Lee, Helen Meng
However, previous works show that state-of-the-art ASV models are seriously vulnerable to voice spoofing attacks, and the recently proposed high-performance spoofing countermeasure (CM) models only focus solely on the standalone anti-spoofing tasks, and ignore the subsequent speaker verification process.
1 code implementation • 9 Jun 2022 • Tianlong Chen, Zhenyu Zhang, Sijia Liu, Yang Zhang, Shiyu Chang, Zhangyang Wang
For example, on downstream CIFAR-10/100 datasets, we identify double-win matching subnetworks with the standard, fast adversarial, and adversarial pre-training from ImageNet, at 89. 26%/73. 79%, 89. 26%/79. 03%, and 91. 41%/83. 22% sparsity, respectively.
1 code implementation • 6 Jun 2022 • Yao Zhang, Nanjun He, Jiawei Yang, Yuexiang Li, Dong Wei, Yawen Huang, Yang Zhang, Zhiqiang He, Yefeng Zheng
Concretely, we propose a novel multimodal Medical Transformer (mmFormer) for incomplete multimodal learning with three main components: the hybrid modality-specific encoders that bridge a convolutional encoder and an intra-modal Transformer for both local and global context modeling within each modality; an inter-modal Transformer to build and align the long-range correlations across modalities for modality-invariant features with global semantics corresponding to tumor region; a decoder that performs a progressive up-sampling and fusion with the modality-invariant features to generate robust segmentation.
Ranked #65 on Semantic Segmentation on NYU Depth v2
no code implementations • 26 May 2022 • Yao Zhang, Jiawei Yang, Yang Liu, Jiang Tian, Siyun Wang, Cheng Zhong, Zhongchao shi, Yang Zhang, Zhiqiang He
In this paper, we propose a Decoupled Pyramid Correlation Network (DPC-Net) that exploits attention mechanisms to fully leverage both low- and high-level features embedded in FCN to segment liver tumor.
no code implementations • 25 May 2022 • Guodong Sun, Yang Zhou, Huilin Pan, Bo Wu, Ye Hu, Yang Zhang
In this paper, we propose a lightweight NMS-free framework to achieve real-time detection and high accuracy simultaneously.
no code implementations • 16 May 2022 • Xinyuan Zhu, Yang Zhang, Fuli Feng, Xun Yang, Dingxian Wang, Xiangnan He
Towards this goal, we propose a Hidden Confounder Removal (HCR) framework that leverages front-door adjustment to decompose the causal effect into two partial effects, according to the mediators between item features and user feedback.
no code implementations • 16 May 2022 • Shibo Feng, Chunyan Miao, Ke Xu, Jiaxiang Wu, Pengcheng Wu, Yang Zhang, Peilin Zhao
The probability prediction of multivariate time series is a notoriously challenging but practical task.
1 code implementation • 13 May 2022 • Xiangnan He, Yang Zhang, Fuli Feng, Chonggang Song, Lingling Yi, Guohui Ling, Yongdong Zhang
We demonstrate DCR on the backbone model of neural factorization machine (NFM), showing that DCR leads to more accurate prediction of user preference with small inference time cost.
no code implementations • 30 Apr 2022 • Cheng-Te Li, Cheng Hsu, Yang Zhang
We propose a novel fairness-aware sequential recommendation task, in which a new metric, interaction fairness, is defined to estimate how recommended items are fairly interacted by users with different protected attribute groups.
1 code implementation • NAACL 2022 • Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, Yang Zhang, Shiyu Chang, Marin Soljačić, Shang-Wen Li, Wen-tau Yih, Yoon Kim, James Glass
We propose DiffCSE, an unsupervised contrastive learning framework for learning sentence embeddings.
Ranked #13 on Semantic Textual Similarity on STS16
1 code implementation • 20 Apr 2022 • Kaizhi Qian, Yang Zhang, Heting Gao, Junrui Ni, Cheng-I Lai, David Cox, Mark Hasegawa-Johnson, Shiyu Chang
Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to downstream tasks.
no code implementations • 14 Apr 2022 • Yun Shen, Yufei Han, Zhikun Zhang, Min Chen, Ting Yu, Michael Backes, Yang Zhang, Gianluca Stringhini
Previous security research efforts orbiting around graphs have been exclusively focusing on either (de-)anonymizing the graphs or understanding the security and privacy issues of graph neural networks.
no code implementations • 4 Apr 2022 • Ashkan Khakzar, Yawei Li, Yang Zhang, Mirac Sanisoglu, Seong Tae Kim, Mina Rezaei, Bernd Bischl, Nassir Navab
One challenging property lurking in medical datasets is the imbalanced data distribution, where the frequency of the samples between the different classes is not balanced.
1 code implementation • 29 Mar 2022 • Heting Gao, Junrui Ni, Kaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson
We show that WavPrompt is a few-shot learner that can perform speech understanding tasks better than a naive text baseline.
1 code implementation • 29 Mar 2022 • Junrui Ni, Liming Wang, Heting Gao, Kaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson
An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untranscribed speech waveforms in that language; 2) a collection of texts written in that language without access to any transcribed speech.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
1 code implementation • 29 Mar 2022 • Evelina Bakhturina, Yang Zhang, Boris Ginsburg
First, a non-deterministic WFST outputs all normalization candidates, and then a neural language model picks the best one -- similar to shallow fusion for automatic speech recognition.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
1 code implementation • 26 Mar 2022 • Chak Ho Chan, Kaizhi Qian, Yang Zhang, Mark Hasegawa-Johnson
SpeechSplit can perform aspect-specific voice conversion by disentangling speech into content, rhythm, pitch, and timbre using multiple autoencoders in an unsupervised manner.
1 code implementation • ICLR 2022 • Shunyu Yao, Mo Yu, Yang Zhang, Karthik R Narasimhan, Joshua B. Tenenbaum, Chuang Gan
In this work, we propose a novel way to establish such a link by corpus transfer, i. e. pretraining on a corpus of emergent language for downstream natural language tasks, which is in contrast to prior work that directly transfers speaker and listener parameters.
1 code implementation • ICLR 2022 • Shangyuan Tong, Timur Garipov, Yang Zhang, Shiyu Chang, Tommi S. Jaakkola
Furthermore, we show that our approach can be viewed as a limit of existing notions of alignment by increasing transportation assignment tolerance.
1 code implementation • 14 Feb 2022 • Haotong Qin, Xudong Ma, Yifu Ding, Xiaoyang Li, Yang Zhang, Yao Tian, Zejun Ma, Jie Luo, Xianglong Liu
Then, to allow the instant and adaptive accuracy-efficiency trade-offs at runtime, we also propose a Thinnable Binarization Architecture to further liberate the acceleration potential of the binarized network from the topology perspective.
1 code implementation • 10 Feb 2022 • Andrew Ma, Yang Zhang, Thomas Christensen, Hoi Chun Po, Li Jing, Liang Fu, Marin Soljačić
Topological materials present unconventional electronic properties that make them attractive for both basic science and next-generation technological applications.
no code implementations • 28 Jan 2022 • Haowei He, Jingzhao Zhang, Yanan Wang, Benben Jiang, Shaobo Huang, Chen Wang, Yang Zhang, Gengang Xiong, Xuebing Han, Dongxu Guo, Guannan He, Minggao Ouyang
In addition to demonstrating how existing deep learning algorithms can be applied to this task, we further develop an algorithm that exploits the data structure of battery systems.
1 code implementation • 27 Jan 2022 • Tianshuo Cong, Xinlei He, Yang Zhang
Recent research has shown that the machine learning model's copyright is threatened by model stealing attacks, which aim to train a surrogate model to mimic the behavior of a given model.
1 code implementation • CVPR 2023 • Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, Yang Zhang
Self-supervised representation learning techniques have been developing rapidly to make full use of unlabeled images.
no code implementations • 17 Jan 2022 • Tianyi Xie, Liucheng Liao, Cheng Bi, Benlai Tang, Xiang Yin, Jianfei Yang, Mingjie Wang, Jiali Yao, Yang Zhang, Zejun Ma
The task of few-shot visual dubbing focuses on synchronizing the lip movements with arbitrary speech input for any talking head video.
no code implementations • 11 Jan 2022 • Zhengying Liu, Adrien Pavao, Zhen Xu, Sergio Escalera, Fabio Ferreira, Isabelle Guyon, Sirui Hong, Frank Hutter, Rongrong Ji, Julio C. S. Jacques Junior, Ge Li, Marius Lindauer, Zhipeng Luo, Meysam Madadi, Thomas Nierhoff, Kangning Niu, Chunguang Pan, Danny Stoll, Sebastien Treguer, Jin Wang, Peng Wang, Chenglin Wu, Youcheng Xiong, Arbe r Zela, Yang Zhang
Code submissions were executed on hidden tasks, with limited time and computational resources, pushing solutions that get results quickly.
no code implementations • 5 Jan 2022 • Yang Zhang, Yang Yang, Chenyun Xiong, Guodong Sun, Yanwen Guo
Encoder-decoder models have been widely used in RGBD semantic segmentation, and most of them are designed via a two-stream network.
Ranked #13 on Semantic Segmentation on SUN-RGBD (using extra training data)
1 code implementation • 15 Dec 2021 • Yun Shen, Xinlei He, Yufei Han, Yang Zhang
Graph neural networks (GNNs), a new family of machine learning (ML) models, have been proposed to fully leverage graph data to build powerful applications.
1 code implementation • NeurIPS 2021 • Yang Zhang, Bo Tang, Qingyu Yang, Dou An, Hongyin Tang, Chenyang Xi, Xueying Li, Feiyu Xiong
Further, a novel offline reinforcement learning method and an off-policy evaluation algorithm are proposed for policy learning and policy evaluation, respectively.
no code implementations • 25 Nov 2021 • Yang Zhang, Qiang Ma
Third, we propose a dynamic context sampling strategy which captures the ``macro-scoped'' citing intents from a manuscript and empowers the citation embeddings to be content-dependent, which allow the algorithm to further improve the performances.
no code implementations • 20 Nov 2021 • Xuezhen Tu, Kun Zhu, Nguyen Cong Luong, Dusit Niyato, Yang Zhang, Juan Li
In this paper, we provide a comprehensive review for the economic and game theoretic approaches proposed in the literature to design various schemes for stimulating data owners to participate in FL training process.
1 code implementation • 15 Nov 2021 • Junhao Zhou, Yufei Chen, Chao Shen, Yang Zhang
In addition, we show that our attacks can be used to enhance the performance of membership inference against GANs.
1 code implementation • 11 Nov 2021 • Yang Liu, Yao Zhang, Yixin Wang, Feng Hou, Jin Yuan, Jiang Tian, Yang Zhang, Zhongchao shi, Jianping Fan, Zhiqiang He
Transformer, an attention-based encoder-decoder model, has already revolutionized the field of natural language processing (NLP).
no code implementations • 8 Nov 2021 • Ahmed Salem, Michael Backes, Yang Zhang
In this work, we propose a new training time attack against computer vision based machine learning models, namely model hijacking attack.
1 code implementation • 28 Oct 2021 • Moyun Liu, Youping Chen, Lei He, Yang Zhang, Jingming Xie
To further prove the ability of our method, we test it on public dataset MS COCO, and the results show that our LF-YOLO has a outstanding versatility detection performance.
1 code implementation • NeurIPS 2021 • Mo Yu, Yang Zhang, Shiyu Chang, Tommi S. Jaakkola
The selection mechanism is commonly integrated into the model itself by specifying a two-component cascaded system consisting of a rationale generator, which makes a binary selection of the input features (which is the rationale), and a predictor, which predicts the output based only on the selected features.
1 code implementation • 26 Oct 2021 • Yunjie Peng, Kang Ma, Yang Zhang, Zhiqiang He
Gait recognition captures gait patterns from the walking sequence of an individual for identification.
no code implementations • 26 Oct 2021 • Tianxu Li, Kun Zhu, Nguyen Cong Luong, Dusit Niyato, Qihui Wu, Yang Zhang, Bing Chen
Then, we review a number of applications of MARL to solve emerging issues in future Internet.
1 code implementation • NeurIPS 2021 • Yonggan Fu, Qixuan Yu, Yang Zhang, Shang Wu, Xu Ouyang, David Cox, Yingyan Lin
Deep Neural Networks (DNNs) are known to be vulnerable to adversarial attacks, i. e., an imperceptible perturbation to the input can mislead DNNs trained on clean images into making erroneous predictions.