Search Results for author:
Found 8 papers, 5 papers with code
A global analysis of metrics used for measuring performance in natural language processing
Our results suggest that the large majority of natural language processing metrics currently used have properties that may result in an inadequate reflection of a models' performance.
Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Benchmarks are crucial to measuring and steering progress in artificial intelligence (AI).
Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
Furthermore, there is a lack of systematized meta-information that allows clinical AI researchers to quickly determine accessibility, scope, content and other characteristics of datasets and benchmark datasets relevant to the clinical domain.
A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks
Research in artificial intelligence (AI) is addressing a growing number of tasks through a rapidly growing number of models and methodologies.
Neural sentence embedding models for semantic similarity estimation in the biomedical domain
BACKGROUND: In this study, we investigated the efficacy of current state-of-the-art neural sentence embedding models for semantic similarity estimation of sentences from biomedical literature.
Ranked #1 on
Sentence Embeddings For Biomedical Texts
on BIOSSES
GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain
However, in-domain pretraining seems not to be sufficient; novel pretraining and few-shot learning strategies are required in the biomedical NLP domain.
Deep learning models are not robust against noise in clinical text
The proposed perturbation methods can be used in performance evaluation tests to assess how robustly clinical NLP models can operate on noisy data, in real-world settings.
A critical analysis of metrics used for measuring progress in artificial intelligence
Our results suggest that the large majority of metrics currently used have properties that may result in an inadequate reflection of a models' performance.
