Search Results for author: Jiajie Zhang

Found 22 papers, 15 papers with code

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

1 code implementation19 Dec 2024 Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

This paper introduces LongBench v2, a benchmark designed to assess the ability of LLMs to handle long-context problems requiring deep understanding and reasoning across real-world multitasks.

8k In-Context Learning +1

LongReward: Improving Long-context Large Language Models with AI Feedback

1 code implementation28 Oct 2024 Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

Though significant advancements have been achieved in developing long-context large language models (LLMs), the compromised quality of LLM-synthesized data for supervised fine-tuning (SFT) often affects the long-context performance of SFT models and leads to inherent limitations.

Offline RL Reinforcement Learning (RL)

Pre-training Distillation for Large Language Models: A Design Space Exploration

no code implementations21 Oct 2024 Hao Peng, Xin Lv, Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, Juanzi Li

Previous work applying KD in the field of large language models (LLMs) typically focused on the post-training phase, where the student LLM learns directly from instructions and corresponding responses generated by the teacher model.

Knowledge Distillation

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

1 code implementation4 Sep 2024 Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack of citations in their responses makes user verification difficult, leading to concerns about their trustworthiness due to their potential hallucinations.

Question Answering Sentence

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

3 code implementations13 Aug 2024 Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10, 000 words while maintaining output quality.

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs

1 code implementation11 Apr 2024 Chaoqun He, Renjie Luo, Shengding Hu, Yuanqian Zhao, Jie zhou, Hanghao Wu, Jiajie Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

The rapid development of LLMs calls for a lightweight and easy-to-use framework for swift evaluation deployment.

KB-Plugin: A Plug-and-play Framework for Large Language Models to Induce Programs over Low-resourced Knowledge Bases

1 code implementation2 Feb 2024 Jiajie Zhang, Shulin Cao, Linmei Hu, Ling Feng, Lei Hou, Juanzi Li

Secondly, KB-Plugin utilizes abundant annotated data from a rich-resourced KB to train another pluggable module, namely PI plugin, which can help the LLM extract question-relevant schema information from the schema plugin of any KB and utilize this information to induce programs over this KB.

Program induction Self-Supervised Learning

LongAlign: A Recipe for Long Context Alignment of Large Language Models

1 code implementation31 Jan 2024 Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li

Extending large language models to effectively handle long contexts requires instruction fine-tuning on input sequences of similar length.

Diversity Instruction Following

Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions

1 code implementation23 Nov 2023 Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Juanzi Li, Lei Hou

During reasoning, for leaf nodes, LLMs choose a more confident answer from Closed-book QA that employs parametric knowledge and Open-book QA that employs retrieved external knowledge, thus eliminating the negative retrieval problem.

Retrieval

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

3 code implementations28 Aug 2023 Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

In this paper, we introduce LongBench, the first bilingual, multi-task benchmark for long context understanding, enabling a more rigorous evaluation of long context understanding.

16k Code Completion +2

DeRisk: An Effective Deep Learning Framework for Credit Risk Prediction over Real-World Financial Data

no code implementations7 Aug 2023 Yancheng Liang, Jiajie Zhang, Hui Li, Xiaochen Liu, Yi Hu, Yong Wu, Jinyao Zhang, Yongyan Liu, Yi Wu

Despite the tremendous advances achieved over the past years by deep learning techniques, the latest risk prediction models for industrial applications still rely on highly handtuned stage-wised statistical learning tools, such as gradient boosting and random forest methods.

Deep Learning Prediction

Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering

no code implementations24 May 2023 Jiajie Zhang, Shulin Cao, Tingjia Zhang, Xin Lv, Jiaxin Shi, Qi Tian, Juanzi Li, Lei Hou

To facilitate reasoning, we propose a novel two-stage XQA framework, Reasoning over Hierarchical Question Decomposition Tree (RoHT).

Question Answering

Video Event Extraction via Tracking Visual States of Arguments

no code implementations3 Nov 2022 Guang Yang, Manling Li, Jiajie Zhang, Xudong Lin, Shih-Fu Chang, Heng Ji

Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles.

Event Extraction

Knowledge Inheritance for Pre-trained Language Models

2 code implementations NAACL 2022 Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie zhou

Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs.

Domain Adaptation Knowledge Distillation +2

Malicious Requests Detection with Improved Bidirectional Long Short-term Memory Neural Networks

no code implementations26 Oct 2020 Wenhao Li, Bincheng Zhang, Jiajie Zhang

Detecting and intercepting malicious requests are one of the most widely used ways against attacks in the network security.

Few-Shot Learning Metric Learning +1

Difference-in-Differences: Bridging Normalization and Disentanglement in PG-GAN

no code implementations16 Oct 2020 Xiao Liu, Jiajie Zhang, Siting Li, Zuotong Wu, Yang Yu

We discover that pixel normalization causes object entanglement by in-painting the area occupied by ablated objects.

counterfactual Disentanglement +1

HighEr-Resolution Network for Image Demosaicing and Enhancing

1 code implementation19 Nov 2019 Kangfu Mei, Juncheng Li, Jiajie Zhang, Hao-Yu Wu, Jie Li, Rui Huang

However, plenty of studies have shown that global information is crucial for image restoration tasks like image demosaicing and enhancing.

Demosaicking

Cannot find the paper you are looking for? You can Submit a new open access paper.