2 code implementations • 19 May 2023 • Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation for Large Language Models (HaluEval) benchmark, a large collection of generated and human-annotated hallucinated samples for evaluating the performance of LLMs in recognizing hallucination.
1 code implementation • 18 May 2023 • Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jingyuan Wang, Jian-Yun Nie, Ji-Rong Wen
In order to further improve the capacity of LLMs for knowledge-intensive tasks, we consider augmenting LLMs with the large-scale web using search engine.
no code implementations • 25 Apr 2023 • Junyi Li, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
In this way, conditional text generation can be cast as a glyph image generation task, and it is then natural to apply continuous diffusion models to discrete texts.
1 code implementation • 31 Mar 2023 • Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, YiFan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, Ji-Rong Wen
To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size.
1 code implementation • CVPR 2023 • Junyi Li, Zhilu Zhang, Xiaoyu Liu, Chaoyu Feng, Xiaotao Wang, Lei Lei, WangMeng Zuo
And we extend the blind-spot network to a blind-neighborhood network (BNN) for providing supervision on flat areas.
no code implementations • 13 Feb 2023 • Junyi Li, Feihu Huang, Heng Huang
Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms, yet it is underexplored in the Federated Learning setting.
no code implementations • 13 Feb 2023 • Junyi Li, Feihu Huang, Heng Huang
This matches the best known rate for first-order FL algorithms and \textbf{FedDA-MVR} is the first adaptive FL algorithm that achieves this rate.
1 code implementation • 26 Dec 2022 • Tianyi Tang, Junyi Li, Zhipeng Chen, Yiwen Hu, Zhuohao Yu, Wenxun Dai, Zican Dong, Xiaoxue Cheng, Yuhao Wang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
To facilitate research on text generation, this paper presents a comprehensive and unified library, TextBox 2. 0, focusing on the use of pre-trained language models (PLMs).
Ranked #1 on
Style Transfer
on GYAFC
no code implementations • 25 Oct 2022 • Junyi Li, Heng Huang
Therefore, Federated Recommender (FedRec) systems are proposed to mitigate privacy concerns to non-distributed recommender systems.
1 code implementation • 24 Oct 2022 • Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
However, NAR models usually generate texts of lower quality due to the absence of token dependency in the output text.
1 code implementation • 20 Oct 2022 • Marcos V. Conde, Radu Timofte, Yibin Huang, Jingyang Peng, Chang Chen, Cheng Li, Eduardo Pérez-Pellitero, Fenglong Song, Furui Bai, Shuai Liu, Chaoyu Feng, Xiaotao Wang, Lei Lei, Yu Zhu, Chenghua Li, Yingying Jiang, Yong A, Peisong Wang, Cong Leng, Jian Cheng, Xiaoyu Liu, Zhicun Yin, Zhilu Zhang, Junyi Li, Ming Liu, WangMeng Zuo, Jun Jiang, Jinha Kim, Yue Zhang, Beiji Zou, Zhikai Zong, Xiaoxiao Liu, Juan Marín Vega, Michael Sloth, Peter Schneider-Kamp, Richard Röttger, Furkan Kınlı, Barış Özcan, Furkan Kıraç, Li Leyi, SM Nadim Uddin, Dipon Kumar Ghosh, Yong Ju Jung
Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP).
2 code implementations • 24 Jun 2022 • Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen
Pre-trained language models (PLMs) have achieved remarkable success in natural language generation (NLG) tasks.
no code implementations • 11 Jun 2022 • Junyi Li, Jian Pei, Heng Huang
Bilevel optimization problem is a type of optimization problem with two levels of entangled problems.
1 code implementation • NAACL 2022 • Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, Wayne Xin Zhao
First, PTG learns a set of source prompts for various source generation tasks and then transfers these prompts as target prompts to perform target generation tasks.
1 code implementation • NAACL 2022 • Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Wayne Xin Zhao, Ji-Rong Wen
In this paper, we present a large-scale empirical study on general language ability evaluation of PLMs (ElitePLM).
no code implementations • 3 May 2022 • Junyi Li, Feihu Huang, Heng Huang
Specifically, we first propose the FedBiO, a deterministic gradient-based algorithm and we show it requires $O(\epsilon^{-2})$ number of iterations to reach an $\epsilon$-stationary point.
1 code implementation • Computational and Structural Biotechnology Journal 2022 • Chi Zhang, Hao Jiang, Weihuang Liu, Junyi Li, Shiming Tang, Mario Juhas, Yang Zhang.
Results To solve the out-of-focus issue in microscopy, we developed a Cycle Generative Adversarial Network (CycleGAN) based model and a multi-component weighted loss function.
2 code implementations • 20 Apr 2022 • Ren Yang, Radu Timofte, Meisong Zheng, Qunliang Xing, Minglang Qiao, Mai Xu, Lai Jiang, Huaida Liu, Ying Chen, Youcheng Ben, Xiao Zhou, Chen Fu, Pei Cheng, Gang Yu, Junyi Li, Renlong Wu, Zhilu Zhang, Wei Shang, Zhengyao Lv, Yunjin Chen, Mingcai Zhou, Dongwei Ren, Kai Zhang, WangMeng Zuo, Pavel Ostyakov, Vyal Dmitry, Shakarim Soltanayev, Chervontsev Sergey, Zhussip Magauiya, Xueyi Zou, Youliang Yan, Pablo Navarrete Michelini, Yunhua Lu, Diankai Zhang, Shaoli Liu, Si Gao, Biao Wu, Chengjian Zheng, Xiaofeng Zhang, Kaidi Lu, Ning Wang, Thuong Nguyen Canh, Thong Bach, Qing Wang, Xiaopeng Sun, Haoyu Ma, Shijie Zhao, Junlin Li, Liangbin Xie, Shuwei Shi, Yujiu Yang, Xintao Wang, Jinjin Gu, Chao Dong, Xiaodi Shi, Chunmei Nian, Dong Jiang, Jucai Lin, Zhihuai Xie, Mao Ye, Dengyan Luo, Liuhan Peng, Shengjie Chen, Qian Wang, Xin Liu, Boyang Liang, Hang Dong, Yuhao Huang, Kai Chen, Xingbei Guo, Yujing Sun, Huilei Wu, Pengxu Wei, Yulin Huang, Junying Chen, Ik Hyun Lee, Sunder Ali Khowaja, Jiseok Yoon
This challenge includes three tracks.
1 code implementation • 12 Apr 2022 • Junyi Li, Xiaohe Wu, Zhenxing Niu, WangMeng Zuo
However, BiRNN is intrinsically offline because it uses backward recurrent modules to propagate from the last to current frames, which causes high latency and large memory consumption.
no code implementations • 22 Mar 2022 • Sha Yuan, Shuai Zhao, Jiahong Leng, Zhao Xue, Hanyu Zhao, Peiyu Liu, Zheng Gong, Wayne Xin Zhao, Junyi Li, Jie Tang
The results show that WuDaoMM can be applied as an efficient dataset for VLPMs, especially for the model in text-to-image generation task.
no code implementations • 18 Feb 2022 • Yifan Du, Zikang Liu, Junyi Li, Wayne Xin Zhao
In this paper, we review the recent progress in Vision-Language Pre-Trained Models (VL-PTMs).
1 code implementation • COLING 2022 • Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen
Secondly, we use continuous inverse prompting to improve the process of natural language generation by modeling an inverse generation process from output to input, making the generated text more relevant to the inputs.
no code implementations • 14 Jan 2022 • Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
We begin with introducing three key aspects of applying PLMs to text generation: 1) how to encode the input into representations preserving input semantics which can be fused into PLMs; 2) how to design an effective PLM to serve as the generation model; and 3) how to effectively optimize PLMs given the reference text and to ensure that the generated texts satisfy special text properties.
no code implementations • 9 Dec 2021 • Junyi Li, Bin Gu, Heng Huang
Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.
no code implementations • 26 Jul 2021 • Feihu Huang, Junyi Li, Shangqian Gao, Heng Huang
Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) to solve deterministic bilevel problems, which achieves a lower computational complexity than the best known results.
no code implementations • 21 Jun 2021 • Feihu Huang, Junyi Li, Shangqian Gao
To fill this gap, in the paper, we propose a novel fast adaptive bilevel framework to solve stochastic bilevel optimization problems that the outer problem is possibly nonconvex and the inner problem is strongly convex.
no code implementations • 21 Jun 2021 • Feihu Huang, Junyi Li
In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many data mining and machine learning problems with a hierarchical structure such as distributionally robust FL and model-agnostic meta learning (MAML).
1 code implementation • NeurIPS 2021 • Feihu Huang, Junyi Li, Heng Huang
To fill this gap, we propose a faster and universal framework of adaptive gradients (i. e., SUPER-ADAM) by introducing a universal adaptive matrix that includes most existing adaptive gradient forms.
1 code implementation • Findings (ACL) 2021 • Junyi Li, Tianyi Tang, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan, Ji-Rong Wen
This paper studies how to automatically generate a natural language text that describes the facts in knowledge graph (KG).
no code implementations • 21 May 2021 • Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen
In this paper, we present an overview of the major advances achieved in the topic of PLMs for text generation.
no code implementations • 9 May 2021 • Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan, Ji-Rong Wen
For global coherence, we design a hierarchical self-attentive architecture with both subgraph- and node-level attention to enhance the correlations between subgraphs.
2 code implementations • 11 Mar 2021 • Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, Zongzheng Xi, Yueqian Yang, Anwen Hu, Jinming Zhao, Ruichen Li, Yida Zhao, Liang Zhang, Yuqing Song, Xin Hong, Wanqing Cui, Danyang Hou, Yingyan Li, Junyi Li, Peiyu Liu, Zheng Gong, Chuhao Jin, Yuchong Sun, ShiZhe Chen, Zhiwu Lu, Zhicheng Dou, Qin Jin, Yanyan Lan, Wayne Xin Zhao, Ruihua Song, Ji-Rong Wen
We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model.
Ranked #1 on
Image Retrieval
on RUC-CAS-WenLan
1 code implementation • ACL 2021 • Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen
In this paper, we release an open-source library, called TextBox, to provide a unified, modularized, and extensible text generation framework.
no code implementations • SEMEVAL 2020 • Junyi Li, Yuhang Wu, Bin Wang, Haiyan Ding
Our methods achieved a F1 score of 0. 85 in subtask A.
no code implementations • SEMEVAL 2020 • Junyi Li, Xiaobing Zhou, Zichen Zhang
We only participate in the English part of subtask A, which aims to identify offensive languages in English.
no code implementations • SEMEVAL 2020 • Junyi Li, Bin Wang, Haiyan Ding
This article describes the system submitted to SemEval 2020 Task 4: Commonsense Validation and Explanation.
no code implementations • NeurIPS Workshop CAP 2020 • Jiyang Zhang, Sheena Panthaplackel, Pengyu Nie, Junyi Li, Ray Mooney, Milos Gligoric
Object-oriented programming languages enable a hierarchical class structure, which provides rich contextual information to guide code comprehension and synthesis.
1 code implementation • 4 Oct 2020 • Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan, Ji-Rong Wen
First, based on graph capsules, we adaptively learn aspect capsules for inferring the aspect sequence.
no code implementations • 1 Sep 2020 • Junyi Li, Bin Gu, Heng Huang
In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.
no code implementations • 17 Jun 2020 • Junyi Li, Heng Huang
Due to the rising privacy demand in data mining, Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field.
no code implementations • ICLR 2019 • Junyi Li, Xitong Wang, Yaoyang Lin, Arunesh Sinha, Micheal P. Wellman
We propose an approach to generate realistic and high-fidelity stock market data based on generative adversarial networks (GANs).
3 code implementations • COLING 2020 • Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks.
4 code implementations • 28 Mar 2020 • Gaole He, Junyi Li, Wayne Xin Zhao, Peiju Liu, Ji-Rong Wen
Our generator is isolated from user interaction data, and serves to improve the performance of the discriminator.
no code implementations • WS 2019 • Junyi Li, Xiaobing Zhou, Yuhang Wu, Bin Wang
We participated in the BioNLP 2019 Open Shared Tasks: binary relation extraction of SeeDev task.
1 code implementation • ACL 2019 • Junyi Li, Wayne Xin Zhao, Ji-Rong Wen, Yang song
In this paper, we propose a novel review generation model by characterizing an elaborately designed aspect-aware coarse-to-fine generation process.
no code implementations • SEMEVAL 2019 • Junyi Li
In this paper, we describe a suggestion mining system that participated in SemEval 2019 Task 9, SubTask A - Suggestion Mining from Online Reviews and Forums.