TY - JOUR
AU - Zhang, Boheng
AB - Abstract Context.— Study design and statistical analyses have improved in journals published in Western countries, but the type of research designs and statistical methods used in medical journals outside Western countries has not been assessed. Objectives.— To determine the frequency of research designs and statistical techniques used in Chinese medical journals, types of statistical errors present, and trends over a 10-year period. Design .— Evaluation of all original articles published in 5 leading journals in 1985 (N=640) and in 1995 (N=954). Main Outcome Measures.— Research designs and statistical methods. Results.— Compared with 1985, significant improvement was seen in 1995; the percentages of original articles reporting clinical trials, prospective studies, or basic science research increased from 18% to 31% (P<.001), the proportion of papers using statistical tests increased from 40% to 60% (P<.001), more sophisticated statistical methods were used, and of those articles using statistics, the proportion using appropriate methods increased from 22% to 46% (P<.001). In both years, the most commonly used statistical methods were t tests and contingency tables. The most common errors were presentation of P values without specifying the test used, use of multiple t tests instead of analysis of variance, and use of unpaired t tests when paired tests were required. Conclusions.— The use of statistical methods in Chinese medical journals research is improving, and by 1995, the frequency of using statistical methods in published articles was similar to the results determined in previous studies of journals. However, the lack of or inappropriate use of statistics remains a serious problem. STATISTICAL METHODS used in biomedical research articles have been increasingly scrutinized, especially in Western journals.1-10 However, they have not been studied in Chinese medical journals. With more than 400 medical journals being used in China by large numbers of clinicians and readers, it is important to know if the statistical methods used in Chinese medical journals are appropriate and to determine what improvements should be made in the tradition of self-evaluation and improvement. We reviewed original research articles published in 5 leading Chinese medical journals in 1985 and 1995. We determined the frequency of research designs and statistical techniques used and the common statistical errors in the research reports. We also compared the articles published in 1985 and 1995 to describe changes over time. Methods The 5 leading medical journals publishing clinical research in China were selected: Chinese Journal of Internal Medicine, Chinese Journal of Surgery, Chinese Journal of Obstetrics & Gynecology, Chinese Journal of Pediatrics, and Chinese Journal of Cardiology. These journals, sponsored by the Chinese Medical Association, represent the highest academic level in Chinese medical journals. All use peer reviewers for original articles. Two journals routinely involve a statistical reviewer for each original research article; the others involve a statistical reviewer when considering articles containing sophisticated statistical methods. We selected papers identified as original research articles in each journal's table of contents. All 954 original articles published in 1995 and all 640 in 1985 were reviewed. For each article, we recorded the research design and type of statistical methods used. We then judged whether statistical tests were necessary and, if used, whether appropriate. A structured form with a checklist was used. We categorized statistical methods by a modification of the method used by Emerson and Colditz,1 omitting cost-benefit analysis and sensitivity analysis because no articles used them. Few papers used the categories of discriminate analysis (1 paper); adjustment and standardization (2 papers); power and sample size calculations (1 paper); receiver operating characteristic curve analysis (2 papers); or mathematical modeling (2 papers), so we classified them as "others." If more than 1 statistical method was used in a paper, all were recorded; if the same statistical method was applied in the same paper repeatedly, it was recorded only once. We also recorded whether the authors specified which statistical methods they had used. All articles were reviewed by both authors. Discrepancies were resolved by discussion. Both authors assessed 100 articles independently; agreement on whether the statistics methods used were appropriate was 91% (κ=0.89). The frequency of each category of statistical test used in an article was calculated in 2 ways: as a simple percentage of the total number of articles and as a percentage of articles that used statistical methods. χ2 Tests were used to compare differences in use of statistical methods during the 10 years. Results Compared with 1985, the original research articles in 1995 consisted of more clinical trials, prospective studies, and basic science studies (31.1% in 1995 and 17.6% in 1985) (χ2=36.4; P <.001) and fewer case reports and retrospective studies (Table 1). For all study designs (except case reports), a higher percentage of articles contained statistical methods in 1995. Both in 1985 and 1995, clinical trials were most likely to use statistical tests, and case reports were least likely. Table 1.—Use of Statistical Methods in Original Articles According to Study Design, 1995 and 1985 Table 1.—Use of Statistical Methods in Original Articles According to Study Design, 1995 and 1985 View LargeDownload Of 954 original research articles published in 1995, 573 (60.1%) used statistical tests (Table 2). Of the 381 articles without statistical analysis, 93 articles needed but omitted statistical analysis, 71 were case reports, and 217 were descriptive studies using only descriptive statistics such as means, SDs, or percentages. Of 640 articles published in 1985, the proportion of the articles using statistical methods was 40.2%, significantly lower than in 1995 (χ2=60.9; P <.001). Of the 383 articles without statistical analysis, 60 articles needed but omitted statistical methods. Table 2.—Statistical Content of Original Articles, 1995 and 1985* Table 2.—Statistical Content of Original Articles, 1995 and 1985* View LargeDownload In both 1985 and 1995, the most commonly used statistical methods were t tests and contingency tables (Table 2). More sophisticated statistical methods were used in 1995 than in 1985. In 1995, 20 articles used multiple regression, while in 1985 only 5 articles used multiple regression. When presenting survival analysis, in 1995, 3 articles used the Cox regression model, while in 1985 none did and only 3 articles used simple survival analysis. The proportion of articles using more than 1 statistical method was higher in 1995, 260 (27.3%) of the 954 articles, than in 1985, 63 (9.8%) of the 640 articles (χ2=71.9; P<.001). More articles that used statistical methods used them appropriately in 1995 (46%) than in 1985 (22%) (χ2=45.3; P <.001) (Table 3). For inappropriate analyses, the most common error was presenting P values without specifying the statistical methods the authors used, especially in 1985. Because more statistical tests were used in 1995, the percentage of articles using incorrect methods actually increased, from 24% in 1985 to 36% in 1995. The most common errors were using t tests instead of analysis of variance (ANOVA) to compare means of more than 2 groups and using unpaired t tests instead of paired t tests. Table 3.—Appropriateness of Statistical Methods, 1995 and 1985* Table 3.—Appropriateness of Statistical Methods, 1995 and 1985* View LargeDownload In 1995, the percentage of articles using statistical methods varied across the 5 journals from 39% to 70%; in 4 journals, the percentage was more than 60%. The proportion of articles using inappropriate statistical methods varied from 39% to 71%, with 2 of the journals at about 70% and the other 3 journals at about 40%. Comment We found that 60% of original research articles published in 5 major Chinese medical journals in 1995 used statistical tests, a significant increase over the 40% in 1985. In both years, t test and contingency tables were used most frequently, followed by ANOVA and Pearson correlation. The study by Emerson and Colditz1 of 332 original research articles published in 1979 in The New England Journal of Medicine1 found that 73% used statistical methods. If we exclude the 72 case reports in 1995 in our study, the proportion of articles using statistical methods would be 65%, slightly lower than the study by Emerson and Colditz. Morris5 found that only 66 (23%) of 103 original articles published in an orthopedics journal in 1984 used statistical methods. Of 592 articles published in ophthalmic journals in 1990, 66% contained statistical techniques. Kanter and Taylor10 found that statistical methods were used in 48% of original research articles published in 1993 in Transfusion. Thus, use of statistical methods in leading Chinese medical journals appear to be at least as frequent as most journals around the world for which data are available. The rank order of frequency of specific statistical methods used in our study was similar to other studies.1,2,6,7,11 However, articles in Chinese medical journals tended to use t tests and contingency table analysis more often and sophisticated methods less often. We found substantial improvement in the use of statistical methods over time. Rosenfeld and Rockette11 and Juzych et al7 also found improvement when comparing articles in the 1990s with those in the 1970s and 1980s. Despite the improvement between 1985 and 1995, the proportion of articles using inappropriate statistical methods in 1995 was still high, and less than half, only 46%, used statistical methods correctly. Morris5 found that 46% of articles in the surgery journal published in 1984 used correct methods. The most common error in our study, presenting P values without specifying the test used, was also noted in other studies.2,5,8,10 In our study, the proportion of articles using nonparametric statistics and data transformation were only 1.7% and 0.8%, respectively, in 1995, much lower than other reports.1,2,11 Rosenfeld and Rockette11 documented increasing use of nonparametric statistics in articles from 0% in 1967 to 7.3% in 1989. The low proportion of articles with nonparametric statistics in Chinese medical journals may relate to the authors ignoring the assumptions underlying parametric statistics. The frequency of certain study designs changed over the decade, with clinical trials, prospective studies, and basic science studies becoming more frequent. Even so, in 1995 these types of designs accounted for 31% of all articles, and clinical trials made up only 11%. This finding contrasts with that of McDermott et al12 who found that in 1991 clinical trials made up 35% of published articles in JAMA, The Lancet, and The New England Journal of Medicine. This study covered all original research articles published in 2 different years over a decade, and the results give an estimate of the progress occurring in Chinese medical journals. However, since we studied only the 5 leading Chinese medical journals, it is likely that our results are better than the average level in Chinese medical journals. The quality of using statistical methods in Chinese medical research is gradually improving, and the frequency of using statistical tests in Chinese medical journals appears comparable to that in other parts of the world. However, the lack or inappropriate use of statistics remains a problem. The type of problems we found, such as omitting descriptions of statistical methods or giving P values without specifying the statistical method, suggests that educating medical journal editors in research, design, and biostatistics might improve the quality of journal articles published in Chinese medical journals. References 1. Emerson JD, Colditz GA. Use of statistical analysis in the New England Journal of Medicine. N Engl J Med.1983;309:709-713.Google Scholar 2. Schwart SJ, Sturr M, Golberg G. Statistical methods in rehabilitation literature: a survey of recent publications. Arch Phys Med Rehabil.1996;77:497-500.Google Scholar 3. Wainapel SF, Kayne HL. Statistical methods in rehabilitation research. Arch Phys Med Rehabil.1985;66:332-334.Google Scholar 4. Fletcher RH, Fletcher SW. Clinical research in general medical journals: a 30-year perspective. N Engl J Med.1979;301:180-183.Google Scholar 5. Morris RW. A statistical study of papers in the journal of bone and joint surgery (BR) 1984. J Bone Joint Surg Br.1988;70:242-246.Google Scholar 6. Feinstein AR. Clinical biostatistics, XXV: survey of statistical procedures in general medical journals. Clin Pharmacol Ther.1974;15:97-107.Google Scholar 7. Juzych MS, Shin DH, Seyedsadr M, Siegner SW, Juzych LA. Statistical techniques in ophthalmic journals. Arch Ophthalmol.1992;110:1225-1229.Google Scholar 8. Bailar JC, Mosteller F. Guidelines for statistical reporting in articles for medical journal: amplifications and explanations. Ann Intern Med.1988;108:266-273.Google Scholar 9. Avram MJ, Shanks CA, Dykes MHM, Ronai AK, Stiers WM. Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods. Anesth Analg.1985;64:607-611.Google Scholar 10. Kanter MH, Taylor JR. Accuracy of statistical methods in transfusion: a review of articles from July/August 1992 through June 1993. Transfusion.1994;34:697-701.Google Scholar 11. Rosenfeld RM, Rockette HE. Biostatistics in otolaryngology journals. Arch Otolaryngol Head Neck Surg.1991;117:1172-1176.Google Scholar 12. McDermott MM, Lefevre F, Feinglass J. et al. Changes in study design, gender issues, and other characteristics of clinical research published in three major medical journals from 1971 to 1991. J Gen Intern Med.1995;10:13-18.Google Scholar
TI - Research Design and Statistical Methods in Chinese Medical Journals
JF - JAMA
DO - 10.1001/jama.280.3.283
DA - 1998-07-15
UR - https://www.deepdyve.com/lp/american-medical-association/research-design-and-statistical-methods-in-chinese-medical-journals-dCdcVCz0yT
SP - 283
EP - 285
VL - 280
IS - 3
DP - DeepDyve
ER -