Search Results for author:
Found 52 papers, 39 papers with code
{\mathcal{P}^2}: A Plan-and-Pretrain Approach for Knowledge Graph-to-Text Generation
Text verbalization of knowledge graphs is an important problem with wide application to natural language generation (NLG) systems.
DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
DetectiveQA focuses on evaluating the long-context reasoning ability of LLMs, which not only requires a full understanding of context but also requires extracting important evidences from the context and reasoning according to extracted evidences to answer the given questions.
What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices
In order to achieve success in long context tasks, a large amount of work has been done to enhance the long context capabilities of the model through synthetic data.
OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection
While open-source LLMs offer solutions to these concerns, they typically underperform commercial models in RTL code generation tasks, primarily due to the scarcity of high-quality open-source RTL datasets.
Farewell to Length Extrapolation, a Training-Free Infinite Context with Finite Attention Scope
Furthermore, we have applied LongCache on mainstream LLMs, including LLaMA3 and Mistral-v0. 3, enabling them to support context lengths of at least 400K in Needle-In-A-Haystack tests.
Case2Code: Learning Inductive Reasoning with Synthetic Data
In this paper, we hope to focus on evaluating and teaching LLMs to conduct inductive reasoning, that is, LLMs are supposed to infer underlying rules by observing examples or sequential transformations.
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
This long-context capability allows IXC-2. 5 to excel in tasks requiring extensive input and output contexts.
Ranked #33 on
Visual Question Answering
on MM-Vet
On Affine Homotopy between Language Encoders
In this spirit, we study the properties of \emph{affine} alignment of language encoders and its implications on extrinsic similarity.
Synergetic Event Understanding: A Collaborative Approach to Cross-Document Event Coreference Resolution with Large Language Models
Experimental results demonstrate that our approach surpasses the performance of both the large and small language models individually, forming a complementary advantage.
AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data
Open-source Large Language Models (LLMs) and their specialized variants, particularly Code LLMs, have recently delivered impressive performance.
RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models
In RefChecker, an extractor generates claim-triplets from a response, which are then evaluated by a checker against a reference.
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
Recent advancements in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens
The rapid advancement of Large Language Models (LLMs) has introduced a new frontier in natural language processing, particularly in understanding and processing long-context information.
Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem
This paper presents a new method for evaluating LLM hallucination in Question Answering (QA) based on the unanswerable math word problem (MWP).
Data-freeWeight Compress and Denoise for Large Language Models
Significantly, our method is characterized by without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods.
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
Code Needs Comments: Enhancing Code LLMs with Comment Augmentation
The programming skill is one crucial ability for Large Language Models (LLMs), necessitating a deep understanding of programming languages (PLs) and their correlation with natural languages (NLs).
Identifying Semantic Induction Heads to Understand In-Context Learning
Although large language models (LLMs) have demonstrated remarkable performance, the lack of transparency in their inference logic raises concerns about their trustworthiness.
Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
To address the dropped tokens and padding, we propose the Rectify-Router, comprising the Intra-GPU Rectification and the Fill-in Rectification.
F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication
Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo.
Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus
Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields.
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
As large language models (LLMs) have shown effectiveness with different prompting methods, such as Chain of Thought, Program of Thought, we find that these methods have formed a great complementarity to each other on math reasoning tasks.
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding
Analogy-making between narratives is crucial for human reasoning.
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Our experiments with instruction-tuning and further pre-training demonstrate that AdaLomo achieves results on par with AdamW, while significantly reducing memory requirements, thereby lowering the hardware barrier to training large language models.
Full Parameter Fine-tuning for Large Language Models with Limited Resources
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training.
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Motivated by the fact that all event structures can be inferred from AMR, this work reformulates EAE as a link prediction problem on AMR graphs.
Abstract Meaning Representation
Event Argument Extraction
+2
Do Large Language Models Know What They Don't Know?
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks.
Exploiting Abstract Meaning Representation for Open-Domain Question Answering
The Open-Domain Question Answering (ODQA) task involves retrieving and subsequently generating answers from fine-grained relevant passages within a database.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
Evaluating Open-QA Evaluation
This study focuses on the evaluation of the Open Question Answering (Open-QA) task, which can directly estimate the factuality of large language models (LLMs).
Word-Level Representation From Bytes For Language Modeling
Based on this observation, we rethink the existing character-aware method that takes character-level inputs but makes word-level sequence modeling and prediction.
RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees
RLET iteratively performs single step reasoning with sentence selection and deduction generation modules, from which the training signal is accumulated across the tree with elaborately designed aligned reward function that is consistent with the evaluation.
DORE: Document Ordered Relation Extraction based on Generative Framework
In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models.
What Dense Graph Do You Need for Self-Attention?
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities.
Dialogue Meaning Representation for Task-Oriented Dialogue Systems
We propose Dialogue Meaning Representation (DMR), a pliable and easily extendable representation for task-oriented dialogue.
Towards Collaborative Question Answering: A Preliminary Study
Knowledge and expertise in the real-world can be disjointedly owned.
A Unified Generative Framework for Various NER Subtasks
To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
Ranked #11 on
Nested Named Entity Recognition
on GENIA
Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.
GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation
With a human-annotated test set, we provide this new benchmark dataset for future research on unsupervised text generation from knowledge graphs.
Ranked #1 on
Unsupervised KG-to-Text Generation
on GenWiki (Fine)
CoLAKE: Contextualized Language and Knowledge Embedding
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained entity embeddings, which limits the performance gains of these models.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Due to the difficulty and high cost of data collection, the supervised data available in the two fields are usually on the magnitude of tens of thousands, for example, 18K in the WebNLG~2017 dataset after preprocessing, which is far fewer than the millions of data for other tasks such as machine translation.
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Adversarial attacks for discrete data (such as texts) have been proved significantly more challenging than continuous data (such as images) since it is difficult to generate adversarial samples with gradient-based methods.
Multi-Scale Self-Attention for Text Classification
In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules.
Joint Parsing and Generation for Abstractive Summarization
If generating a word can introduce an erroneous relation to the summary, the behavior must be discouraged.
Ranked #27 on
Text Summarization
on GigaWord
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
The Transformer model is widely successful on many natural language processing tasks.
Ranked #1 on
Machine Translation
on IWSLT2015 Chinese-English
Star-Transformer
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data.
Ranked #13 on
Sentiment Analysis
on SST-5 Fine-grained classification
Named Entity Recognition (NER)
Natural Language Inference
+2
Top-Down Tree Structured Text Generation
Text generation is a fundamental building block in natural language processing tasks.
First Step toward Model-Free, Anonymous Object Tracking with Recurrent Neural Networks
In this paper, we propose and study a novel visual object tracking approach based on convolutional networks and recurrent networks.
