Search Results for author:
Found 6 papers, 4 papers with code
Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
No submissions outperformed the baselines in the multimodal track.
Aioli: A Unified Optimization Framework for Language Model Data Mixing
Finally, we leverage the insights from our framework to derive a new online method named Aioli, which directly estimates the mixing law parameters throughout training and uses them to dynamically adjust proportions.
[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus
The big changes for this year's competition are as follows: First, we replace the loose track with a paper track, which allows (for example) non-model-based submissions, novel cognitively-inspired benchmarks, or analysis techniques.
Comparing Abstraction in Humans and Large Language Models Using Multimodal Serial Reproduction
To investigate the effect language on the formation of abstractions, we implement a novel multimodal serial reproduction framework by asking people who receive a visual stimulus to reproduce it in a linguistic format, and vice versa.
Latent State Models of Training Dynamics
We use the HMM representation to study phase transitions and identify latent "detour" states that slow down convergence.
Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
Co-training on these representations result in more human-like behavior in downstream meta-reinforcement learning agents than less abstract controls (synthetic language descriptions, program induction without learned primitives), suggesting that the abstraction supported by these representations is key.
