1 code implementation • 3 Jul 2024 • Paul Pu Liang, Akshay Goindani, Talha Chafekar, Leena Mathur, Haofei Yu, Ruslan Salakhutdinov, Louis-Philippe Morency
Through comprehensive experiments across the 30 tasks in HEMM, we (1) identify key dataset dimensions (e. g., basic skills, information flows, and use cases) that pose challenges to today's models, and (2) distill performance trends regarding how different modeling dimensions (e. g., scale, pre-training data, multimodal alignment, pre-training, and instruction tuning objectives) influence performance.
no code implementations • 6 Dec 2023 • Talha Chafekar, Aafiya Hussain, Grishma Sharma, Deepak Sharma
There has been a lot of work in question generation where different methods to provide target answers as input, have been employed.