Search Results for author:
Found 11 papers, 4 papers with code
Multilingual Text Summarization on Financial Documents
The language models were fine-tuned on a financial document collection of three languages (English, Spanish, and Greek) and aim to identify the beginning of the summary narrative part of the document.
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage.
Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
Cultural biases in multilingual datasets pose significant challenges for their effectiveness as global benchmarks.
INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge
The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal value of generative AI tools in many communities.
Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants
We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses.
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Large language models (LLMs) can potentially democratize access to medical knowledge.
Ranked #1 on
Multiple Choice Question Answering (MCQA)
on MedMCQA
(Dev Set (Acc-%) metric)
CRAB: Assessing the Strength of Causal Relationships Between Real-world Events
In this work, we present CRAB, a new Causal Reasoning Assessment Benchmark designed to evaluate causal understanding of events in real-world narratives.
Beyond S-curves: Recurrent Neural Networks for Technology Forecasting
Our autoencoder approach improves the MAPE by 13. 5% on average over the second-best result.
Interpreting Language Models Through Knowledge Graph Extraction
In this paper, we compare BERT-based language models through snapshots of acquired knowledge at sequential stages of the training process.
On Representation Learning for Scientific News Articles Using Heterogeneous Knowledge Graphs
In this work, we present a methodology for creating scientific news article representations by modeling the directed graph between the scientific news articles and the cited scientific publications.
SciLens News Platform: A System for Real-Time Evaluation of News Articles
We demonstrate the SciLens News Platform, a novel system for evaluating the quality of news articles.
