Search Results for author:
Found 19 papers, 10 papers with code
Simple Radiology VLLM Test-time Scaling with Thought Graph Traversal
Test-time scaling offers a promising way to improve the reasoning performance of vision-language large models (VLLMs) without additional training.
Beyond Similarity: A Gradient-based Graph Method for Instruction Tuning Data Selection
Large language models (LLMs) have shown great potential across various industries due to their remarkable ability to generalize through instruction tuning.
Advancing Large Language Model Attribution through Self-Improving
Teaching large language models (LLMs) to generate text with citations to evidence sources can mitigate hallucinations and enhance verifiability in information-seeking systems.
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization
News summarization in today's global scene can be daunting with its flood of multilingual content and varied viewpoints from different sources.
Extending Context Window of Large Language Models from a Distributional Perspective
However, existing scaling methods often rely on empirical approaches and lack a profound understanding of the internal distribution within RoPE, resulting in suboptimal performance in extending the context window length.
CFSP: An Efficient Structured Pruning Framework for LLMs with Coarse-to-Fine Activation Information
To this end, we introduce an efficient structured pruning framework named CFSP, which leverages both Coarse (interblock) and Fine-grained (intrablock) activation information as an importance criterion to guide pruning.
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
New candidate tokens from the decoding process are then used to update the matrix.
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training
To address this problem, we propose the Make Some Noise (MSN) training framework as a replacement for the supervised fine-tuning stage of the large language model.
MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability
When encountering malicious instructions, the router will assign a higher weight to the safe LLM to ensure that responses are harmless.
Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance
The results show that our approach not only boosts the general reasoning performance of LLMs but also makes considerable strides towards their capacity for abstract reasoning, moving beyond simple memorization or imitation to a more nuanced understanding and application of generic facts.
How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers
Through analysis, we found the contribution ratio of Multi-Head Attention (a combination function) to pre-trained language modeling is a key factor affecting base capabilities.
Semi-Instruct: Bridging Natural-Instruct and Self-Instruct for Code Large Language Models
Presently, two dominant paradigms for collecting tuning data are natural-instruct (human-written) and self-instruct (automatically generated).
Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models.
SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models
Existing methods devise the learning module to acquire task-specific knowledge with parameter-efficient tuning (PET) block and the selection module to pick out the corresponding one for the testing input, aiming at handling the challenges of catastrophic forgetting and knowledge transfer in CL.
Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
Built upon the Transformer, large language models (LLMs) have captured worldwide attention due to their remarkable abilities.
SmartTrim: Adaptive Tokens and Attention Pruning for Efficient Vision-Language Models
Despite achieving remarkable performance on various vision-language tasks, Transformer-based Vision-Language Models (VLMs) suffer from redundancy in inputs and parameters, significantly hampering their efficiency in real-world applications.
XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters
In recent years, pre-trained language models have undergone rapid development with the emergence of large-scale models.
TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation
Knowledge graph embedding (KGE) aims to learn continuous vectors of relations and entities in knowledge graph.
Ranked #8 on
Link Property Prediction
on ogbl-wikikg2
Verification Code Recognition Based on Active and Deep Learning
However, the advantages of convolutional neural networks depend on the data used by the training classifier, particularly the size of the training set.
