Search Results for author:
Found 16 papers, 8 papers with code
LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
Large Language Diffusion Models, or diffusion LLMs, have emerged as a significant focus in NLP research, with substantial effort directed toward understanding their scalability and downstream task performance.
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache
Large Language Models struggle with memory demands from the growing Key-Value (KV) cache as context lengths increase.
AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecture
Based on our findings, we derive a reference framework and provide architectural guidelines by mapping it onto the ISO 23247 reference architecture for digital twins.
Thus Spake Long-Context Large Language Model
Moreover, the research on long-context LLMs has expanded from length extrapolation to a comprehensive focus on architecture, infrastructure, training, and evaluation technologies.
Capturing Human Cognitive Styles with Language: Towards an Experimental Evaluation Paradigm
While NLP models often seek to capture cognitive states via language, the validity of predicted states is determined by comparing them to annotations created without access the cognitive states of the authors.
VideoRoPE: What Makes for Good Video Rotary Position Embedding?
As part of our analysis, we introduce a challenging V-NIAH-D (Visual Needle-In-A-Haystack with Distractors) task, which adds periodic distractors into V-NIAH.
LongSafetyBench: Long-Context LLMs Struggle with Safety Issues
With the development of large language models (LLMs), the sequence length of these models continues to increase, drawing significant attention to long-context language models.
DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
DetectiveQA focuses on evaluating the long-context reasoning ability of LLMs, which not only requires a full understanding of context but also requires extracting important evidences from the context and reasoning according to extracted evidences to answer the given questions.
ReAttention: Training-Free Infinite Context with Finite Attention Scope
The long-context capability of the Large Language Models (LLM) has made significant breakthroughs, but the maximum supported context length remains a critical bottleneck limiting their practical applications.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo.
Scaling Laws of RoPE-based Extrapolation
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
Transfer and Active Learning for Dissonance Detection: Addressing the Rare-Class Challenge
While transformer-based systems have enabled greater accuracies with fewer training examples, data acquisition obstacles still persist for rare-class tasks -- when the class label is very infrequent (e. g. < 5% of samples).
Active Learning
Implicit Discourse Relation Classification
+1
Numerology Selection for OFDM Systems Based on Deep Neural Networks
In this letter, we propose a deep neutral network (DNN) approach for numerology selection of OFDM systems.
Anomaly Detection in a Digital Video Broadcasting System Using Timed Automata
This paper focuses on detecting anomalies in a digital video broadcasting (DVB) system from providers' perspective.
