1. The most highly cited articles in evolutionary psychiatry demonstrated overall good quality, particularly in defining research questions with precision.

2. Important conceptual errors were nevertheless common, with half of the top-cited articles failing to consider alternative hypotheses or presenting unbalanced evidence.

3. No significant relationship was found between article quality, error quantity, and citation count, suggesting citation frequency is driven by factors other than scientific rigor.

4. The rating system derived from Nesse’s quality and error criteria showed reliable application, supporting its potential utility as a standardised evaluative tool for the field.

5. Findings highlight a need for consensus on quality standards in evolutionary psychiatry, as well as improved methodological rigour in specifying and evaluating evolutionary hypotheses.

The seminal text, “Why We Get Sick” argued that evolutionary theory is useful in understanding traits that leave members of a species vulnerable to disease [1]. Evolutionary medicine and evolutionary psychiatry subsequently emerged as new disciplines, with specialized conferences, textbooks, and journals [2]. A recent review of evolutionary psychiatry updated the range of possible explanations for vulnerability to mental disorders including mismatch of evolutionary history with current environments and evolutionary tradeoffs that have both benefits and costs [3]. Such explanations complement proximal biological explanations that describe mechanisms underlying disease with more distal evolutionary accounts of the origins of vulnerability to failure [3].

Evolutionary medicine and psychiatry have grappled with the conceptual questions of what makes for a high quality publication in the field, and of what conceptual errors continue to be made by contributors [4]. It has been suggested that good articles in the field define their questions with precision and consider multiple possible explanations of observed phenomena [5]. In contrast, errors include viewing disorders as adaptations, and providing explanations based on what is good for the species [4]. The difficulty in delineating normality from psychopathology is a particular problem in psychiatry and evolutionary psychiatry, where the drawing of a line between normal and excessive activation of an adaptive defence often becomes blurry [6]. This article is an effort towards providing some degree of quality standardization for the field.

We developed a measure of the overall quality and error number in evolutionary psychiatry articles, based on Nesse’s work on how to test evolutionary hypotheses, and on errors in the field [4]. Nesse’s work on quality standards is in turn based on extensive engagement with and review of the field. We applied these measures to the most highly cited articles in the field. We assessed article quality and error quantity in the most cited articles on evolutionary psychiatry and examined the relationship of these measures to citation frequency.

The main findings of this research were (1) the most highly cited articles have good quality on average, (2) the most highly cited articles make important errors, (3) there was no significant relationship between article quality or error quantity and citation number.

Overall, the top 20 mostly highly cited articles achieved a mean on the low end of the “good” rating for each of the 4 positive scoring metrics (3.6–3.9; with good considered 3.4–4.2). For example, the TQS had a rating of 3.64 (SD 0.6). The included articles scored a mean of 3.85 on F1 (Define the object of explanation with great specificity), indicating that this area represents a strength of the included articles. Conversely, F3 (Make explicit predictions from each possible hypothesis) had a mean of 3.60, while F2 (Specify all possible alternative hypotheses for why the trait is apparently suboptimal) and F4 (Use all available evidence to test the predictions from all alternative hypotheses) had the lowest mean score of included articles (3.55), indicating that these are areas that future articles can improve upon.

Of the errors made, the most common were failing to consider all of the alternative hypotheses and presenting all the evidence in favour of a pet hypothesis and all the evidence against other hypotheses, instead of offering a balanced consideration of all evidence for and against all hypotheses. 50% of the assessed articles made one or both of these errors.

None of the top 20 articles most highly cited articles made the errors of FN5 (Thinking that evidence for learning influencing a trait indicates that no evolutionary explanation is needed) or FN6 (Thinking that evidence for environmental or cultural differences in a trait is evidence against evolutionary influences). These errors may, however, be more prevalent in less cited evolutionary psychiatry articles.

Article quality and error quantity were not significantly associated with citation number. This is consistent with previous work in many fields suggesting that quality and methodological rigor are not necessarily correlated with citation number [27, 28]. In neuroscience, orthopaedics and plastic surgery citation numbers are influenced by factors including age of the article, study design, level of evidence, conflict of interest disclosures, and number of authors, but our study was not designed to investigate such influence [27, 29, 30].

This article indicates that the quality standards developed by Nesse can be assessed reliably, and suggests that their routine application could assist the field. Nesse’s quality standards have face validity insofar as they align with general scientific procedures, and our assessment measure now adds procedural validity insofar as it potentially allows different articles to be rated in a standard way. We cannot provide external validity of this measure against a pre-existing measure as this is the first such measure in the field, and indeed the question of how best to define the optimal “gold standard” for assessing the quality of work in evolutionary psychiatry is perhaps an “essentially contested” question that deserves ongoing attention.

Several additional limitations deserve emphasis. First, we assessed only the top 20 highly cited articles, which limits generalizability to the broader field of evolutionary psychiatry; it is possible that less highly cited articles would receive quite different quality and error scores. Second, we relied solely on Scopus for citation numbers; different databases provide different citation numbers, and may have yielded a different selection of articles. Third, some of the scoring items, particularly FN 1–4 and Overall Impression, involve a degree of subjective judgment; more objective anchoring of scores may be possible in the future. Fourth, we used one particular weighting system for summing error quantity; other weightings could be more useful and may be worth investigating. Fifth, the raters are both physicians rather than experts in evolutionary theory or medicine, and the high kappas obtained may not be generalizable to other raters. Sixth, we relied heavily on the work of Nesse; additional or different criteria may also be useful.

Our results suggest that the most highly cited articles in evolutionary psychiatry have both strengths and weaknesses. Strengths include defining the research question with precision and predicting the answers based on a specific hypothesis. Weaknesses include failing to consider and specify alternate hypotheses, and presenting evidence in favour of a pet hypothesis while not using all available evidence. The lack of an association of article quality and error quantity with citation number suggests that other factors influence such citations. While the field has significant promise, further work is needed to achieve consensus on how best to assess the quality of evolutionary psychiatry articles and on what errors should be avoided.

Data is available from the authors.

DJS conceptualized the research, DJS, CB, CR designed the research study. CB and CR evaluated the articles, with oversight from DJS. CB analyzed the data. All authors contributed to editorial changes in the manuscript. All authors read and approved the final manuscript. All authors have participated sufficiently in the work and agreed to be accountable for all aspects of the work.

Not applicable.

Not applicable.

DJS is supported by the South African Medical Research Council.

The authors declare no conflict of interest.