Search Results for author:
Found 6 papers, 4 papers with code
Do pretrained transformers infer telicity like humans?
Pretrained transformer-based language models achieve state-of-the-art performance in many NLP tasks, but it is an open question whether the knowledge acquired by the models during pretraining resembles the linguistic knowledge of humans.
SEA-LION: Southeast Asian Languages in One Network
Recently, Large Language Models (LLMs) have dominated much of the artificial intelligence scene with their ability to process and generate natural languages.
SEA-HELM: Southeast Asian Holistic Evaluation of Language Models
Though existing LLM benchmarks are capable of evaluating specific capabilities of LLMs in English as well as in various mid- to low-resource languages, including those in the Southeast Asian (SEA) region, a comprehensive and authentic evaluation suite for the SEA languages has not been developed thus far.
Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
Cultural biases in multilingual datasets pose significant challenges for their effectiveness as global benchmarks.
Kalahi: A handcrafted, grassroots cultural LLM evaluation suite for Filipino
Multilingual large language models (LLMs) today may not necessarily provide culturally appropriate and relevant responses to its Filipino users.
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models
As GPT-4 is purportedly one of the best-performing multilingual LLMs at the moment, we use it as a yardstick to gauge the capabilities of LLMs in the context of SEA languages.
