Search Results for author:
Found 6 papers, 1 papers with code
Comparing Plausibility Estimates in Base and Instruction-Tuned Large Language Models
Experiment 1 shows that, across model architectures and plausibility datasets, (i) log likelihood ($\textit{LL}$) scores are the most reliable indicator of sentence plausibility, with zero-shot prompting yielding inconsistent and typically poor results; (ii) $\textit{LL}$-based performance is still inferior to human performance; (iii) instruction-tuned models have worse $\textit{LL}$-based performance than base models.
Running cognitive evaluations on large language models: The do's and the don'ts
In this paper, I describe methodological considerations for studies that aim to evaluate the cognitive capacities of large language models (LLMs) using language-based behavioral assessments.
Dissociating language and thought in large language models
Large Language Models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split.
Event knowledge in large language models: the gap between the impossible and the unlikely
Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.
Beyond linear regression: mapping models in cognitive neuroscience should align with research goals
Moreover, we argue that, instead of categorically treating the mapping models as linear or nonlinear, we should instead aim to estimate the complexity of these models.
Probing artificial neural networks: insights from neuroscience
A major challenge in both neuroscience and machine learning is the development of useful tools for understanding complex information processing systems.
