WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models

1 code implementation • 13 Nov 2023 • Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, Juanzi Li

To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible trace for watermark detection.

Benchmarking Instruction Following

Know2BIO: A Comprehensive Dual-View Benchmark for Evolving Biomedical Knowledge Graphs

1 code implementation • 5 Oct 2023 • Yijia Xiao, Dylan Steinecke, Alexander Russell Pelletier, Yushi Bai, Peipei Ping, Wei Wang

Knowledge graphs (KGs) have emerged as a powerful framework for representing and integrating complex biomedical information.

Entity Alignment Knowledge Graphs +1

T$^3$Bench: Benchmarking Current Progress in Text-to-3D Generation

1 code implementation • 4 Oct 2023 • Yuze He, Yushi Bai, Matthieu Lin, Wang Zhao, Yubin Hu, Jenny Sheng, Ran Yi, Juanzi Li, Yong-Jin Liu

Recent methods in text-to-3D leverage powerful pretrained diffusion models to optimize NeRF.

Benchmarking Text to 3D

Large Language Models Can Be Good Privacy Protection Learners

no code implementations • 3 Oct 2023 • Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Haifeng Chen, Wei Wang, Wei Cheng

To address this challenge, we introduce Privacy Protection Language Models (PPLM), a novel paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding data privacy.

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

1 code implementation • 28 Aug 2023 • Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

In this paper, we introduce LongBench, the first bilingual, multi-task benchmark for long context understanding, enabling a more rigorous evaluation of long context understanding.

Code Completion Few-Shot Learning

KoLA: Carefully Benchmarking World Knowledge of Large Language Models

1 code implementation • 15 Jun 2023 • Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan YAO, Ning Ding, Lei Hou, Zhiyuan Liu, Bin Xu, Jie Tang, Juanzi Li

The unprecedented performance of large language models (LLMs) necessitates improvements in evaluations.

Benchmarking

Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization

1 code implementation • 19 Dec 2022 • Yushi Bai, Xin Lv, Juanzi Li, Lei Hou

QTO finds the optimal solution by a forward-backward propagation on the tree-like computation graph, i. e., query computation tree.

Complex Query Answering

SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning

1 code implementation • 17 Jan 2022 • Yushi Bai, Xin Lv, Juanzi Li, Lei Hou, Yincen Qu, Zelin Dai, Feiyu Xiong

Multi-hop knowledge graph (KG) reasoning has been widely studied in recent years to provide interpretable predictions on missing links with evidential paths.

Navigate Reinforcement Learning (RL)

Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones

1 code implementation • NeurIPS 2021 • Yushi Bai, Rex Ying, Hongyu Ren, Jure Leskovec

Here we present ConE (Cone Embedding), a KG embedding model that is able to simultaneously model multiple hierarchical as well as non-hierarchical relations in a knowledge graph.

Ancestor-descendant prediction Knowledge Graph Completion +1