Search Results for author:
Found 4 papers, 3 papers with code
Tending Towards Stability: Convergence Challenges in Small Language Models
We find that nearly all layers in larger models stabilise early in training - within the first 20% - whereas layers in smaller models exhibit slower and less stable convergence, especially when their parameters have lower effective rank.
Causal Estimation of Memorisation Profiles
Understanding memorisation in language models has practical and societal implications, e. g., studying models' training dynamics or preventing copyright infringements.
AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets
By dynamically selecting different anchors at each iteration it promotes class balance and prevents overfitting the initial decision boundary, thus promoting the discovery of new clusters of minority instances.
Diable: Efficient Dialogue State Tracking as Operations on Tables
Sequence-to-sequence state-of-the-art systems for dialogue state tracking (DST) use the full dialogue history as input, represent the current state as a list with all the slots, and generate the entire state from scratch at each dialogue turn.
