Table 2 shows the proportion of studies that fulfilled the standards for study design. The denominator used to calculate proportions was 41 studies except if a standard applied only to a subgroup of the 41 study articles. The first 12 of the 13 standards listed in Table 2 applied to all of the study articles. Articles were examined to determine the number of articles that fulfilled >1 of these 12 standards. The median number of the 12 standards fulfilled per article was 6 (range 1 to 12, 25th to 75th percentile, 5.0 to 8.5). There was no difference between the general and specialty journals in the median number of the 12 standards fulfilled per article (general, median 7, 25th to 75th percentile, 6.0 to 10.0, vs specialty, median, 6, 25th to 75th percentile, 3.0 to 8.3, p=0.4). Figure 1 shows the distribution of the values for the number of the 12 design standards fulfilled by the 41 study articles.
Table 3 displays the methods the 41 study articles used to describe the diagnostic accuracy of the evaluated diagnostic tests. Proportions were calculated using a denominator of 41 except for ROC AUC, which used a denominator of 21 because 21 of the 41 studies had ordinal or continuous values for test results. Seven (17%) studies did not provide any of the measures of diagnostic accuracy listed in Table 3 and seven (17%) studies provided information only for sensitivity and specificity read only canadian health&care mall. Table 3 also shows the number of articles that reported a measure of precision for the estimates of diagnostic accuracy.
Of the 21 study articles that examined diagnostic tests with ordinal or continuous result values that required determination threshold, 11 articles clearly described their method for determining the threshold value. Eight of these 11 studies used an accepted technique for the decision threshold determination.
Twenty-seven of the 41 studies described a role in clinical medicine for the diagnostic test under evaluation without making a statement that additional validating studies were needed. Fourteen studies made explicit statements that additional studies were needed before the tests should be used in routine clinical practice. Only one study discussed method-ologic flaws and their potential impact on conclusions.
Figure 1. Graph bars denote the number of the 12 major study design standards that were fulfilled by individual study articles (n=41). Table 2—Number of the 41 Study Articles That Fulfilled Experimental Design Standards
Clear definition of reference standard
Suitability of reference standard
Indicated study sample
Patient sampling techniques
Pertinent subgroup analysis
Test review bias
Reported indeterminate values
Reported whether indeterminate values were
used for calculating diagnostic accuracy
Calculated measures of diagnostic accuracy with
and without indeterminate results
Incremental value reported
Risk or invasiveness
Consecutive or random patients
Appropriate statistical methods for comparison
Table 3—Number of the 41 Study Articles That Reported Various Measures of Diagnostic Accuracy With Precision Estimates