Search Results for author:
Found 18 papers, 5 papers with code
Scaling Law for Quantization-Aware Training
Through 268 QAT experiments, we show that quantization error decreases as model size increases, but rises with more training tokens and coarser quantization granularity.
Efficient Pretraining Length Scaling
Recent advances in large language models have demonstrated the effectiveness of length scaling during post-training, yet its potential in pre-training remains underexplored.
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
Frac-Connections: Fractional Extension of Hyper-Connections
Residual connections are central to modern deep learning architectures, enabling the training of very deep networks by mitigating gradient vanishing.
HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
These findings highlight the potential of HybridNorm as a more stable and effective technique for improving the training and performance of deep transformer models.
Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
Training stability is a persistent challenge in the pre-training of large language models (LLMs), particularly for architectures such as Post-Norm Transformers, which are prone to gradient explosion and dissipation.
SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models
The increasing application of multi-modal large language models (MLLMs) across various sectors have spotlighted the essence of their output reliability and accuracy, particularly their ability to produce content grounded in factual information (e. g. common and domain-specific knowledge).
Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
Tokenization is a fundamental component of large language models (LLMs), yet its influence on model scaling and performance is not fully explored.
Ultra-Sparse Memory Network
It is widely acknowledged that the performance of Transformer models is exponentially related to their number of parameters and computational complexity.
AlignXIE: Improving Multilingual Information Extraction by Cross-Lingual Alignment
Empirical evidence suggests that LLMs exhibit spontaneous cross-lingual alignment.
Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
Transformers have found extensive applications across various domains due to the powerful fitting capabilities.
Hyper-Connections
We present hyper-connections, a simple yet effective method that can serve as an alternative to residual connections.
Self-Improvement Programming for Temporal Knowledge Graph Question Answering
Temporal Knowledge Graph Question Answering (TKGQA) aims to answer questions with temporal intent over Temporal Knowledge Graphs (TKGs).
Ranked #1 on
Question Answering
on MultiTQ
KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction
After instruction tuning, KnowCoder further exhibits strong generalization ability on unseen schemas and achieves up to $\textbf{12. 5%}$ and $\textbf{21. 9%}$, compared to sota baselines, under the zero-shot setting and the low resource setting, respectively.
Ranked #1 on
UIE
on ACE 2005-EAE
Retrieval-Augmented Code Generation for Universal Information Extraction
Therefore, in this paper, we propose a universal retrieval-augmented code generation framework based on LLMs, called Code4UIE, for IE tasks.
HanoiT: Enhancing Context-aware Translation via Selective Context
Context-aware neural machine translation aims to use the document-level context to improve translation quality.
What is Event Knowledge Graph: A Survey
Besides entity-centric knowledge, usually organized as Knowledge Graph (KG), events are also an essential kind of knowledge in the world, which trigger the spring up of event-centric knowledge representation form like Event KG (EKG).
Event Coreference Resolution with their Paraphrases and Argument-aware Embeddings
To resolve event coreference, existing methods usually calculate the similarities between event mentions and between specific kinds of event arguments.
