[Citation misuse and its effects on public health]. / Le mésusage des citations et ses conséquences en médecine.

In order to effectively contribute to scientific knowledge, biomedical observations have to be validated and debated by scientists in the relevant field. Along this debate that mainly takes place in the scientific literature, citation of previous studies plays a major role. However, only a few academic studies have quantitatively evaluated the suitability and accuracy of scientific citations. Here we review these academic studies. Two types of misuse have been pointed out: Citation bias and citation distortion. First, scientific citations favor positive results and those supporting authors' conclusion. Second, many statements linked to a reference actually misrepresent the referenced findings. About 10% of all citations in biomedicine are strongly inaccurate and misleading for the reader. Finally, we give two examples illustrating how some citation misuses do affect public health: The opioid crisis in the USA and the unjustified fostering of hydroxychloroquine for Covid-19 treatment in France.

TITLE: Le mésusage des citations et ses conséquences en médecine. ABSTRACT: Les observations biomédicales ne deviennent une source de connaissance qu'après un débat entre chercheurs. Au cours de ce débat, la citation des études antérieures tient un rôle majeur, mais les travaux académiques qui en évaluent l'usage sont rares. Ils ont cependant pu révéler deux types de problèmes : les biais de citation et les écarts de sens entre l'étude antérieure citée et ce qu'en dit l'article citant. Dans cette revue, nous synthétisons ces travaux et en dégageons les principales caractéristiques : les études favorables à la conclusion des auteurs citants sont plus souvent citées que celles qui les questionnent ; des écarts de sens majeurs affectent environ 10 % des citations. Nous illustrons par deux exemples les conséquences de ce mésusage des citations.

Messaging in Biological Psychiatry: Misrepresentations, Their Causes, and Potential Consequences.

Most experts in the field of psychiatry recognize that neuroscience advances have yet to be translated into clinical practice. The main message delivered to laypeople, however, is that mental disorders are brain diseases cured by scientifically designed medications. Here we describe how this misleading message is generated. We summarize the academic studies describing how biomedical observations are often misrepresented in the scientific literature through various forms of data embellishment, publication biases favoring initial and positive studies, improper interpretations, and exaggerated conclusions. These misrepresentations also affect biological psychiatry and are spread through mass media documents. Exacerbated competition, hyperspecialization, and the need to obtain funding for research projects might drive scientists to misrepresent their findings. Moreover, journalists are unaware that initial studies, even when positive and promising, are inherently uncertain. Journalists preferentially cover them and almost never inform the public when those studies are disconfirmed by subsequent research. This explains why reductionist theories about mental health often persist in mass media even though the scientific claims that have been put forward to support them have long been contradicted. These misrepresentations affect the care of patients. Indeed, studies show that a neuro-essentialist conceptualization of mental disorders negatively affects several aspects of stigmatization, reduces the chances of patients' healing, and overshadows psychotherapeutic and social approaches that have been found effective in alleviating mental suffering. Public information about mental health should avoid these reporting biases and give equal consideration to the biological, psychological, and social aspects of mental health.

Do newspapers preferentially cover biomedical studies involving national scientists?

News value theory rates geographical proximity as an important factor in the process of issue selection by journalists. But does this apply to science journalism? Previous observational studies investigating whether newspapers preferentially cover scientific studies involving national scientists have generated conflicting answers. Here we used a database of 123 biomedical studies, 113 of them involving at least one research team working in eight countries (Australia, Canada, France, Ireland, Japan, New Zealand, the United Kingdom, and the United States). We compiled all the newspaper articles covering these 123 studies and published in English, French, and Japanese languages. In all eight countries, we found that newspapers preferentially covered studies involving a national team. Moreover, these "national" studies on average gave rise to a larger number of newspaper articles than "foreign" studies. Finally, our study resolves the conflict with previous conclusions by providing an alternative interpretation of published observations.

Low statistical power in biomedical science: a review of three human research domains.

Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0-10% or 11-20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

Poor replication validity of biomedical association studies reported by newspapers.

OBJECTIVE: To investigate the replication validity of biomedical association studies covered by newspapers. METHODS: We used a database of 4723 primary studies included in 306 meta-analysis articles. These studies associated a risk factor with a disease in three biomedical domains, psychiatry, neurology and four somatic diseases. They were classified into a lifestyle category (e.g. smoking) and a non-lifestyle category (e.g. genetic risk). Using the database Dow Jones Factiva, we investigated the newspaper coverage of each study. Their replication validity was assessed using a comparison with their corresponding meta-analyses. RESULTS: Among the 5029 articles of our database, 156 primary studies (of which 63 were lifestyle studies) and 5 meta-analysis articles were reported in 1561 newspaper articles. The percentage of covered studies and the number of newspaper articles per study strongly increased with the impact factor of the journal that published each scientific study. Newspapers almost equally covered initial (5/39 12.8%) and subsequent (58/600 9.7%) lifestyle studies. In contrast, initial non-lifestyle studies were covered more often (48/366 13.1%) than subsequent ones (45/3718 1.2%). Newspapers never covered initial studies reporting null findings and rarely reported subsequent null observations. Only 48.7% of the 156 studies reported by newspapers were confirmed by the corresponding meta-analyses. Initial non-lifestyle studies were less often confirmed (16/48) than subsequent ones (29/45) and than lifestyle studies (31/63). Psychiatric studies covered by newspapers were less often confirmed (10/38) than the neurological (26/41) or somatic (40/77) ones. This is correlated to an even larger coverage of initial studies in psychiatry. Whereas 234 newspaper articles covered the 35 initial studies that were later disconfirmed, only four press articles covered a subsequent null finding and mentioned the refutation of an initial claim. CONCLUSION: Journalists preferentially cover initial findings although they are often contradicted by meta-analyses and rarely inform the public when they are disconfirmed.

Replication Validity of Initial Association Studies: A Comparison between Psychiatry, Neurology and Four Somatic Diseases.

CONTEXT: There are growing concerns about effect size inflation and replication validity of association studies, but few observational investigations have explored the extent of these problems. OBJECTIVE: Using meta-analyses to measure the reliability of initial studies and explore whether this varies across biomedical domains and study types (cognitive/behavioral, brain imaging, genetic and "others"). METHODS: We analyzed 663 meta-analyses describing associations between markers or risk factors and 12 pathologies within three biomedical domains (psychiatry, neurology and four somatic diseases). We collected the effect size, sample size, publication year and Impact Factor of initial studies, largest studies (i.e., with the largest sample size) and the corresponding meta-analyses. Initial studies were considered as replicated if they were in nominal agreement with meta-analyses and if their effect size inflation was below 100%. RESULTS: Nominal agreement between initial studies and meta-analyses regarding the presence of a significant effect was not better than chance in psychiatry, whereas it was somewhat better in neurology and somatic diseases. Whereas effect sizes reported by largest studies and meta-analyses were similar, most of those reported by initial studies were inflated. Among the 256 initial studies reporting a significant effect (p<0.05) and paired with significant meta-analyses, 97 effect sizes were inflated by more than 100%. Nominal agreement and effect size inflation varied with the biomedical domain and study type. Indeed, the replication rate of initial studies reporting a significant effect ranged from 6.3% for genetic studies in psychiatry to 86.4% for cognitive/behavioral studies. Comparison between eight subgroups shows that replication rate decreases with sample size and "true" effect size. We observed no evidence of association between replication rate and publication year or Impact Factor. CONCLUSION: The differences in reliability between biological psychiatry, neurology and somatic diseases suggest that there is room for improvement, at least in some subdomains.