Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset

1 code implementation • NeurIPS 2023 • Saeid Naeini, Raeid Saqur, Mozhgan Saeidi, John Giorgi, Babak Taati

In this paper we present the novel Only Connect Wall (OCW) dataset and report results from our evaluation of selected pre-trained language models and LLMs on creative problem solving tasks like grouping clue words by heterogeneous connections, and identifying correct open knowledge domain connections in respective groups.

Only Connect Walls Dataset Task 1 (Grouping) Only Connect Walls Dataset Task 2 (Connections)