Repetition and reproduction of preclinical medical studies: taking a leaf from the plant sciences with consideration of generalised systematic errors

Reproduction of pre-clinical results has a high failure rate. The fundamental methodology including replication ("protocol") for hypothesis testing/validation to a state allowing inference, varies within medical and plant sciences with little justification. Here, five protocols are distinguished which deal differently with systematic/random errors and vary considerably in result veracity. Aim: to compare prevalence of protocols (defined in text). Medical/plant science articles from 2017/2019 were surveyed: 713 random articles assessed for eligibility for counts: first (with p-values): 1) non-replicated; 2) global; 3) triple-result protocols; second: 4) replication-error protocol; 5) meta-analyses. Inclusion criteria: human/plant/fungal studies with categorical groups. Exclusion criteria: phased clinical trials, pilot studies, cases, reviews, technology, rare subjects, -omic studies. Abbreviated PICOS question: which protocol was evident for a main result with categorically distinct group difference(s) ? Electronic sources: Journal Citation Reports 2017/2019, Google. Triplication prevalence differed dramatically between sciences (both years p<10-16; cluster-adjusted chi-squared tests): From 320 studies (80/science/year): in 2017, 53 (66%, 95% confidence interval (C.I.) 56%:77%) and in 2019, 48 (60%, C.I. 49%:71%) plant studies had triple-result or triplicated global protocols, compared with, in both years, 4 (5%, C.I. 0.19%:9.8%) medical studies. Plant sciences had a higher prevalence of protocols more likely to counter generalised systematic errors (the most likely cause of false positives) and random error than non-replicated protocols, without suffering from serious flaws found with random-Institutes protocols. It is suggested that a triple-result (organised-reproduction) protocol, with Institute consortia, is likely to solve most problems connected with the replicability crisis.

PDF Abstract

Datasets


  Add Datasets introduced or used in this paper

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods


No methods listed for this paper. Add relevant methods here