[ad_1]
Characteristics of newspaper stories, original studies and subsequent studies
Figure 1 illustrates the details of the search. The eight newspapers selected were the New York Times (USA, quality), Washington Post (USA, quality), Daily Telegraph (UK, quality), Times (UK, quality), USA Today (USA, non-quality), Daily News (USA, non-quality), Daily Mail (UK, non-quality) and Daily Mirror (UK, non-quality). When searching for journal names in newspaper stories, we found 1298 newspaper stories, of which 344 described the effectiveness of or recommended certain treatments or preventive measures (kappa=0.73) (table 1). Online supplemental eTable 1 lists the names of 40 medical journals.
Flow chart of original study identification process.
Characteristics of included newspaper stories
A total of 344 newspaper stories were referred to in 319 scientific journal articles. After excluding duplicates, we identified 212 articles that mentioned the effectiveness of the recommended treatment or prevention. We excluded 48 articles because the research questions could not be identified. Finally, we identified 164 original articles and randomly selected 100 of them. These were cited in 158 newspaper articles. The journals in which the 100 original articles were published were as follows: New England Journal of Medicine (NEJM), 39; Journal of the American Medical Association (JAMA), 21; Lancet, 16; British Medical Journal (BMJ), 9; Archives of Internal Medicine, 8; Annals of Internal Medicine, 3; American Journal of Epidemiology, 1; American Journal of Public Health, 1; Infection Control and Hospital Epidemiology, 1; and Mayo Clinic Proceedings, 1. Approximately three-quarters of these articles were published in three major journals (NEJM, JAMA, Lancet).
Of the 100 articles, 58 were RCTs and 31 OSs. A few other designs corresponded to various ICD-10 categories. Of the 158 newspaper stories, two-thirds were in quality papers and the rest in non-quality.
For four of the 100 original studies, the newspapers stated their effectiveness, but the primary outcome of those studies did not indicate their effectiveness. Therefore, these were excluded from this study. In the remaining 96 studies, 104 effective treatments were identified. Subsequent studies on each treatment were searched. We identified relevant subsequent studies for 86 of these 104 treatments. The 18 others remained unchallenged (table 2). Of the 86 subsequent studies, 83 were SR (SR of RCTs, n=45; SR of OSs, n=23; SR of RCTs and OSs, n=15), followed by RCT (n=2) and OS (n=1). The PubMed search formulae are listed in online supplemental eTable 2.
Main analyses of the proportion of confirmed studies
Comparisons of original and subsequent studies
Table 2 shows the proportions of the confirmed studies. A total of 69% (59/86) (95% CI 58.1 to 77.5) of the original studies were confirmed in subsequent studies. Furthermore, of the 59 confirmed original studies, 16 were comparable to subsequent studies in terms of effect size. Among these 16, 13 were replicated and three reported effect sizes larger than the corresponding subsequent studies. Of these 16 studies, 11 compared SMDs. The median SMDs of the original and subsequent studies were 0.23 (0.18, 0.45) and 0.25 (0.15, 0.32), respectively (p=0.34, Wilcoxon signed-rank test). However, for the remaining 43 studies, strict comparisons of effect sizes were not possible because the outcomes were not fully matched between the original and subsequent studies. Details of the original and subsequent studies are presented in online supplemental eTable 3.
We conducted subgroup analyses on the proportions of confirmed studies for each research design in the original articles (online supplemental eTable 4). The proportions of confirmed OS and RCT studies (of which there was a relatively large number) were 61.3% (19/31) and 70.5% (31/44), respectively. Other designs included fewer studies, and we found no significant differences in the research design (p=0.74, χ2 test). For the ICD-10 categories, the differences according to disease were not significant (p=0.67, χ2 test). The proportion of confirmed studies cited in quality papers (56/88, 63.6%) was lower than that in non-quality papers (31/44, 70.5%); however, the difference was not statistically significant (p=0.42, χ2 test).
Example 1: contradicted
A prospective cohort study published in BMJ in 2000, covered by Daily Mail, suggested that drinking fluoridated water significantly reduced hip fractures.16 Neither the subsequent matching study, meta-analysis of 14 observational studies, nor the original study17 found any significant risk reduction in hip fractures.
Example 2: confirmed
One RCT published in the JAMA in 2000 and covered by the Washington Post suggested that sertraline was more effective than a placebo in patients with post-traumatic stress disorder (PTSD). The subsequent matching study was a meta-analysis comparing pharmacotherapies for PTSD, published in 2022.18 In the subgroup analysis, which included the original RCT, sertraline was compared with placebo. The authors concluded that sertraline was effective. Therefore, the effectiveness reported in the original study was confirmed in a subsequent study. Furthermore, the point estimate of the original study’s RR described in the subsequent study’s forest plot was 0.70, and the point estimate and 95% CI of the RR of the new article was 0.68 (0.56 to 0.81). After calculating the SMD from these values, the original study had an SMD of 0.26, and the new study had a value of 0.27 (95% CI 0.15 to 0.40). We categorised this finding as not only ‘confirmed’ but also ‘replicated’.
Example 3: unchallenged
Examples included in the unchallenged studies are as follows: Most studies have investigated unique interventions (eg, short nails for preventing infection, anti-digoxin fab for cardiac arrhythmia, horse chestnut seed extract for chronic venous insufficiency, beta-sheet breaker peptides for prion-related disorders, the Krukenberg procedure for double-hand amputees and yoga for carpal tunnel syndrome), and several studies have examined the effects of special drug use (eg, ondansetron for bulimia nervosa, growth hormone for Crohn’s disease and combination therapy with old antidepressants, nefazodone and psychotherapy for chronic depression). However, these findings are difficult to validate using well-designed studies. The details are shown in online supplemental eTable 3.
[ad_2]
Source link
