no code implementations • CCL 2022 • Junjie Ye, Junjun Guo, Kaiwen Tan, Yan Xiang, Zhengtao Yu
“多模态神经机器翻译旨在利用视觉信息来提高文本翻译质量。传统多模态机器翻译将图像的全局语义信息融入到翻译模型, 而忽略了图像的细粒度信息对翻译质量的影响。对此, 该文提出一种基于图文细粒度对齐语义引导的多模态神经机器翻译方法, 该方法首先跨模态交互图文信息, 以提取图文细粒度对齐语义信息, 然后以图文细粒度对齐语义信息为枢纽, 采用门控机制将多模态细粒度信息对齐到文本信息上, 实现图文多模态特征融合。在多模态机器翻译基准数据集Multi30K 英语→德语、英语→法语以及英语→捷克语翻译任务上的实验结果表明, 论文提出方法的有效性, 并且优于大多数最先进的多模态机器翻译方法。”
no code implementations • COLING 2022 • Junjie Ye, Junjun Guo, Yan Xiang, Kaiwen Tan, Zhengtao Yu
This paper proposes a noise-robust multi-modal interactive fusion approach with cross-modal relation-aware mask mechanism for MNMT.
no code implementations • 6 Feb 2025 • Junjie Ye, David Paz, Hengyuan Zhang, Yuliang Guo, Xinyu Huang, Henrik I. Christensen, Yue Wang, Liu Ren
Topology reasoning is crucial for autonomous driving as it enables comprehensive understanding of connectivity and relationships between lanes and traffic elements.
no code implementations • 23 Jan 2025 • Junjie Ye, Peichang Zhang, Xiao-Peng Li, Lei Huang, Yuanwei Liu
A fluid reconfigurable intelligent surface (fRIS)-aided integrated sensing and communications (ISAC) system is proposed to enhance multi-target sensing and multi-user communication.
1 code implementation • 20 Jan 2025 • Siyu Yuan, Zehui Chen, Zhiheng Xi, Junjie Ye, Zhengyin Du, Jiecao Chen
To further explore the scalability of this self-improvement paradigm, we investigate iterative refinement of both error correction capabilities and dataset construction.
no code implementations • 5 Jan 2025 • Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen
Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models (LLMs).
1 code implementation • 20 Dec 2024 • Junjie Ye, Yilong Wu, Sixian Li, Yuming Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Peng Wang, Zhongchao shi, Jianping Fan, Zhengyin Du
Large language models (LLMs) achieve remarkable advancements by leveraging tools to interact with external environments, a critical step toward generalized AI.
no code implementations • 18 Dec 2024 • Jiageng Mao, Siheng Zhao, Siqi Song, Tianheng Shi, Junjie Ye, Mingtong Zhang, Haoran Geng, Jitendra Malik, Vitor Guizilini, Yue Wang
Scalable learning of humanoid robots is crucial for their deployment in real-world applications.
no code implementations • 24 Sep 2024 • Junjie Ye, Yuming Yang, Qi Zhang, Tao Gui, Xuanjing Huang, Peng Wang, Zhongchao shi, Jianping Fan
Large language models (LLMs) encode extensive world knowledge through pre-training on massive datasets, which can then be fine-tuned for the question-answering (QA) task.
1 code implementation • 31 Jul 2024 • Ming Zhang, Caishuang Huang, Yilong Wu, Shichun Liu, Huiyuan Zheng, Yurui Dong, Yujiong Shen, Shihan Dou, Jun Zhao, Junjie Ye, Qi Zhang, Tao Gui, Xuanjing Huang
Task-oriented dialogue (TOD) systems aim to efficiently handle task-oriented conversations, including information collection.
1 code implementation • 5 Jul 2024 • Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, Yue Wang
First, RAM extracts unified affordance at scale from diverse sources of demonstrations including robotic data, human-object interaction (HOI) data, and custom data to construct a comprehensive affordance memory.
1 code implementation • 26 Jun 2024 • Caishuang Huang, Wanxu Zhao, Rui Zheng, Huijie Lv, WenYu Zhan, Shihan Dou, Sixian Li, Xiao Wang, Enyu Zhou, Junjie Ye, Yuming Yang, Tao Gui, Qi Zhang, Xuanjing Huang
As the development of large language models (LLMs) rapidly advances, securing these models effectively without compromising their utility has become a pivotal area of research.
1 code implementation • 17 Jun 2024 • Yuming Yang, Wantong Zhao, Caishuang Huang, Junjie Ye, Xiao Wang, Huiyuan Zheng, Yang Nan, Yuran Wang, Xueying Xu, Kaixin Huang, Yunke Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
First, we detect inconsistent entity definitions across datasets and clarify them by distinguishable label names to construct a universal taxonomy of 400+ entity types.
no code implementations • 1 May 2024 • Shihan Dou, Yan Liu, Enyu Zhou, Tianlong Li, Haoxiang Jia, Limao Xiong, Xin Zhao, Junjie Ye, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
These two issues can be united as a challenge posed by the shifted distribution of the environment.
1 code implementation • 26 Mar 2024 • Junjie Ye, Lei Huang, Zhen Chen, Peichang Zhang, Mohamed Rihan
It is critical to design efficient beamforming in reconfigurable intelligent surface (RIS)-aided integrated sensing and communication (ISAC) systems for enhancing spectrum utilization.
1 code implementation • 27 Feb 2024 • RuiZhe Zhong, Junjie Ye, Zhentao Tang, Shixiong Kai, Mingxuan Yuan, Jianye Hao, Junchi Yan
First, we propose global circuit training to pre-train a graph auto-encoder that learns the global graph embedding from circuit netlist.
1 code implementation • 26 Feb 2024 • Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, Xuanjing Huang
Adversarial misuse, particularly through `jailbreaking' that circumvents a model's safety and ethical protocols, poses a significant challenge for Large Language Models (LLMs).
no code implementations • 22 Feb 2024 • Junjie Ye, Nuo Xu, Yikun Wang, Jie zhou, Qi Zhang, Tao Gui, Xuanjing Huang
To overcome the limitations of existing data augmentation methods that compromise semantic integrity and address the uncertainty inherent in LLM-generated text, we leverage the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels.
no code implementations • 17 Feb 2024 • Siyin Wang, ShiMin Li, Tianxiang Sun, Jinlan Fu, Qinyuan Cheng, Jiasheng Ye, Junjie Ye, Xipeng Qiu, Xuanjing Huang
HAG extends the current paradigm in the text generation process, highlighting the feasibility of endowing the LLMs with self-regulate decoding strategies.
1 code implementation • 16 Feb 2024 • Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, Xuanjing Huang
Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios.
1 code implementation • 30 Jan 2024 • Xiaoran Fan, Tao Ji, Changhao Jiang, Shuo Li, Senjie Jin, Sirui Song, Junke Wang, Boyang Hong, Lu Chen, Guodong Zheng, Ming Zhang, Caishuang Huang, Rui Zheng, Zhiheng Xi, Yuhao Zhou, Shihan Dou, Junjie Ye, Hang Yan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #120 on
Visual Question Answering
on MM-Vet
1 code implementation • 21 Jan 2024 • Songyang Gao, Qiming Ge, Wei Shen, Shihan Dou, Junjie Ye, Xiao Wang, Rui Zheng, Yicheng Zou, Zhi Chen, Hang Yan, Qi Zhang, Dahua Lin
This reliance limits the applicability of RLHF and hinders the development of professional assistants tailored to diverse human preferences.
1 code implementation • 16 Jan 2024 • Junjie Ye, Yilong Wu, Songyang Gao, Caishuang Huang, Sixian Li, Guanyu Li, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang
To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning.
1 code implementation • 1 Jan 2024 • Junjie Ye, Guanyu Li, Songyang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Tao Ji, Qi Zhang, Tao Gui, Xuanjing Huang
Existing evaluations of tool learning primarily focus on validating the alignment of selected tools for large language models (LLMs) with expected outcomes.
no code implementations • 19 Dec 2023 • Jinyi Liu, Zhi Wang, Yan Zheng, Jianye Hao, Chenjia Bai, Junjie Ye, Zhen Wang, Haiyin Piao, Yang Sun
In reinforcement learning, the optimism in the face of uncertainty (OFU) is a mainstream principle for directing exploration towards less explored areas, characterized by higher uncertainty.
no code implementations • 28 Nov 2023 • Junjie Ye, Mohamed Rihan, Peichang Zhang, Lei Huang, Stefano Buzzi, Zhen Chen
Energy efficiency (EE) is a challenging task in integrated sensing and communication (ISAC) systems, where high spectral efficiency and low energy consumption appear as conflicting requirements.
1 code implementation • 17 Nov 2023 • Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, Yue Wang
Human-level driving is an ultimate goal of autonomous driving.
1 code implementation • 14 Oct 2023 • Junjie Ye, Jie zhou, Junfeng Tian, Rui Wang, Qi Zhang, Tao Gui, Xuanjing Huang
Recently, Target-oriented Multimodal Sentiment Classification (TMSC) has gained significant attention among scholars.
1 code implementation • 2 Oct 2023 • Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, Yue Wang
In this paper, we propose a novel approach to motion planning that capitalizes on the strong reasoning capabilities and generalization potential inherent to Large Language Models (LLMs).
Ranked #1 on
Motion Planning
on nuScenes
no code implementations • 3 May 2023 • Junjie Ye, Jilin Zhao
In this study, we explore the potential of using a straightforward neural network inspired by the retina model to efficiently restore low-light images.
1 code implementation • 17 Apr 2023 • Xiao Wang, Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, Jihua Kang, Jingsheng Yang, Siyuan Li, Chunsai Du
Large language models have unlocked strong multi-task capabilities from reading instructive prompts.
Ranked #3 on
Zero-shot Named Entity Recognition (NER)
on CrossNER
(using extra training data)
1 code implementation • 20 Mar 2023 • Junjie Ye, Changhong Fu, Ziang Cao, Shan An, Guangze Zheng, Bowen Li
To realize reliable UAV tracking at night, a spatial-channel Transformer-based low-light enhancer (namely SCT), which is trained in a novel task-inspired manner, is proposed and plugged prior to tracking approaches.
no code implementations • 18 Mar 2023 • Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang shen, Jie zhou, Siming Chen, Tao Gui, Qi Zhang, Xuanjing Huang
GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities.
1 code implementation • 9 Mar 2023 • Qizhou Wang, Junjie Ye, Feng Liu, Quanyu Dai, Marcus Kalander, Tongliang Liu, Jianye Hao, Bo Han
It leads to a min-max learning scheme -- searching to synthesize OOD data that leads to worst judgments and learning from such OOD data for uniform performance in OOD detection.
1 code implementation • 8 Mar 2023 • Liangliang Yao, Changhong Fu, Sihang Li, Guangze Zheng, Junjie Ye
The proposed method designs a new task-specific object saliency mining network to refine the cross-correlation operation and effectively discriminate foreground and background information.
no code implementations • 4 Mar 2023 • Jiren Mai, Fei Zhang, Junjie Ye, Marcus Kalander, Xian Zhang, Wankou Yang, Tongliang Liu, Bo Han
Motivated by this simple but effective learning pattern, we propose a General-Specific Learning Mechanism (GSLM) to explicitly drive a coarse-grained CAM to a fine-grained pseudo mask.
no code implementations • 1 Mar 2023 • Xuanting Chen, Junjie Ye, Can Zu, Nuo Xu, Rui Zheng, Minlong Peng, Jie zhou, Tao Gui, Qi Zhang, Xuanjing Huang
The GPT-3. 5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities.
Natural Language Inference
Natural Language Understanding
+1
1 code implementation • 26 Nov 2022 • Guangze Zheng, Changhong Fu, Junjie Ye, Bowen Li, Geng Lu, Jia Pan
The key to the visual UAM approaching lies in object tracking, while current UAM tracking typically relies on costly model-based methods.
1 code implementation • ICCV 2023 • Bowen Li, Ziyuan Huang, Junjie Ye, Yiming Li, Sebastian Scherer, Hang Zhao, Changhong Fu
Visual object tracking is essential to intelligent robots.
no code implementations • COLING 2022 • Siyin Wang, Jie zhou, Changzhi Sun, Junjie Ye, Tao Gui, Qi Zhang, Xuanjing Huang
In this work, we propose a causal intervention model for Implicit Sentiment Analysis using Instrumental Variable (ISAIV).
1 code implementation • 14 Aug 2022 • Changhong Fu, Haolin Dong, Junjie Ye, Guangze Zheng, Sihang Li, Jilin Zhao
Pixel-level range mask is introduced to make HighlightNet more focused on the enhancement of the tracking object and regions without light sources.
1 code implementation • 1 Aug 2022 • Changhong Fu, Weiyu Peng, Sihang Li, Junjie Ye, Ziang Cao
Specifically, with local-modeling to global-search mechanism, the proposed tracker replaces the global encoder by a novel local-recognition encoder.
2 code implementations • 11 Jun 2022 • Xiong Peng, Feng Liu, Jingfen Zhang, Long Lan, Junjie Ye, Tongliang Liu, Bo Han
To defend against MI attacks, previous work utilizes a unilateral dependency optimization strategy, i. e., minimizing the dependency between inputs (i. e., features) and outputs (i. e., labels) during training the classifier.
1 code implementation • 9 May 2022 • Changhong Fu, Kunhan Lu, Guangze Zheng, Junjie Ye, Ziang Cao, Bowen Li, Geng Lu
Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications and attracted increasing attention in the field of intelligent transportation systems because of its versatility and effectiveness.
2 code implementations • CVPR 2022 • Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, Guang Chen
Previous advances in object tracking mostly reported on favorable illumination circumstances while neglecting performance at nighttime, which significantly impeded the development of related aerial robot applications.
1 code implementation • 3 Mar 2022 • Changhong Fu, Sihang Li, Xinnan Yuan, Junjie Ye, Ziang Cao, Fangqiang Ding
Therefore, to help increase awareness of the potential risk and the robustness of UAV tracking, this work proposes a novel adaptive adversarial attack approach, i. e., Ad$^2$Attack, against UAV object tracking.
no code implementations • 29 Sep 2021 • Jinyi Liu, Zhi Wang, Yan Zheng, Jianye Hao, Junjie Ye, Chenjia Bai, Pengyi Li
Many exploration strategies are built upon the optimism in the face of the uncertainty (OFU) principle for reinforcement learning.
no code implementations • 24 Aug 2021 • Shan An, Guangfu Che, Jinghao Guo, Haogang Zhu, Junjie Ye, Fangru Zhou, Zhaoqi Zhu, Dong Wei, Aishan Liu, Wei zhang
To this concern, this work proposes a real-time augmented reality virtual shoe try-on system for smartphones, namely ARShoe.
1 code implementation • ICCV 2021 • Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, Yiming Li
Most existing Siamese-based tracking methods execute the classification and regression of the target object based on the similarity maps.
1 code implementation • 30 Jul 2021 • Junjie Ye, Changhong Fu, Guangze Zheng, Ziang Cao, Bowen Li
Recent years have witnessed the fast evolution and promising performance of the convolutional neural network (CNN)-based trackers, which aim at imitating biological visual systems.
1 code implementation • 16 Jun 2021 • Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, Yiming Li
By virtue of the attention mechanism, we conduct a special attentional aggregation network (AAN) consisting of self-AAN and cross-AAN for raising the representation ability of features eventually.
1 code implementation • 15 Jun 2021 • Guangze Zheng, Changhong Fu, Junjie Ye, Fuling Lin, Fangqiang Ding
However, prevalent discriminative correlation filter (DCF) based trackers are insensitive to target mutations due to a predefined label, which concentrates on merely the centre of the training region.
1 code implementation • 4 Jun 2021 • Bowen Li, Changhong Fu, Fangqiang Ding, Junjie Ye, Fuling Lin
The target-aware mask can be applied to jointly train a target-focused filter that assists the context filter for robust tracking.
Ranked #30 on
Video Object Tracking
on NT-VOT211
2 code implementations • 8 Mar 2021 • Bowen Li, Yiming Li, Junjie Ye, Changhong Fu, Hang Zhao
As a crucial robotic perception capability, visual tracking has been intensively studied recently.
1 code implementation • 21 Jan 2021 • Bowen Li, Changhong Fu, Fangqiang Ding, Junjie Ye, Fuling Lin
However, prior tracking methods have merely focused on robust tracking in the well-illuminated scenes, while ignoring trackers' capabilities to be deployed in the dark.
no code implementations • ICLR 2021 • Pengfei Chen, Guangyong Chen, Junjie Ye, Jingwei Zhao, Pheng-Ann Heng
The noise in stochastic gradient descent (SGD) provides a crucial implicit regularization effect, previously studied in optimization by analyzing the dynamics of parameter updates.
1 code implementation • 19 Dec 2020 • Changhong Fu, Ziang Cao, Yiming Li, Junjie Ye, Chen Feng
In the domain of visual tracking, most deep learning-based trackers highlight the accuracy but casting aside efficiency.
1 code implementation • 10 Dec 2020 • Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, Pheng-Ann Heng
In this work, we present a theoretical hypothesis testing and prove that noise in real-world dataset is unlikely to be CCN, which confirms that label noise should depend on the instance and justifies the urgent need to go beyond the CCN assumption. The theoretical results motivate us to study the more general and practical-relevant instance-dependent noise (IDN).
Ranked #45 on
Image Classification
on Clothing1M
1 code implementation • 8 Dec 2020 • Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, Pheng-Ann Heng
For validation, we prove that a noisy validation set is reliable, addressing the critical demand of model selection in scenarios like hyperparameter-tuning and early stopping.