Search Results for author:
Found 7 papers, 1 papers with code
Why language models collapse when trained on recursively generated text
We present a new finding: the performance of LMs gradually declines when trained on recursively generated text until they perform no better than a randomly initialized LM.
LoRA Dropout as a Sparsity Regularizer for Overfitting Control
We then demonstrate the theoretical mechanism of our LoRA Dropout mechanism from the perspective of sparsity regularization by providing a generalization error bound under this framework.
Exploring the Potential of Large Language Models in Graph Generation
In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments.
Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models
With an analysis of loss distributions of code tokens, we find that code tokens can be divided into two categories: challenging tokens that are difficult to predict and confident tokens that can be easily inferred.
DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Training
Moreover, our case study shows that our visual solution can well reflect the characteristics of various training scenarios, showing good potential of DVI as a debugging tool for analyzing deep learning training processes.
Massive Self-Assembly in Grid Environments
Self-assembly plays an essential role in many natural processes, involving the formation and evolution of living or non-living structures, and shows potential applications in many emerging domains.
Multiagent Systems Distributed, Parallel, and Cluster Computing Robotics
SpotTune: Leveraging Transient Resources for Cost-efficient Hyper-parameter Tuning in the Public Cloud
Hyper-parameter tuning (HPT) is crucial for many machine learning (ML) algorithms.
