Search Results for author:
Found 8 papers, 2 papers with code
FANformer: Improving Large Language Models Through Effective Periodicity Modeling
Our pretrained FANformer-1B exhibits marked improvements on downstream tasks compared to open-source LLMs with similar model parameters or training tokens.
Theoretical Proof that Auto-regressive Language Models Collapse when Real-world Data is a Finite Set
Auto-regressive language models (LMs) have been widely used to generate data in data-scarce domains to train new LMs, compensating for the scarcity of real-world data.
LoRA Dropout as a Sparsity Regularizer for Overfitting Control
We then demonstrate the theoretical mechanism of our LoRA Dropout mechanism from the perspective of sparsity regularization by providing a generalization error bound under this framework.
Exploring the Potential of Large Language Models in Graph Generation
In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments.
Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models
With an analysis of loss distributions of code tokens, we find that code tokens can be divided into two categories: challenging tokens that are difficult to predict and confident tokens that can be easily inferred.
DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Training
Moreover, our case study shows that our visual solution can well reflect the characteristics of various training scenarios, showing good potential of DVI as a debugging tool for analyzing deep learning training processes.
Massive Self-Assembly in Grid Environments
Self-assembly plays an essential role in many natural processes, involving the formation and evolution of living or non-living structures, and shows potential applications in many emerging domains.
Multiagent Systems Distributed, Parallel, and Cluster Computing Robotics
SpotTune: Leveraging Transient Resources for Cost-efficient Hyper-parameter Tuning in the Public Cloud
Hyper-parameter tuning (HPT) is crucial for many machine learning (ML) algorithms.
