Methodologic Standards for Diagnostic Test Research in Pulmonary Medicine: Results

In: Pulmonary Medicine

30 Jun 2014

Table 2 shows the proportion of studies that fulfilled the standards for study design. The denominator used to calculate proportions was 41 studies except if a standard applied only to a subgroup of the 41 study articles. The first 12 of the 13 standards listed in Table 2 applied to all of the study articles. Articles were examined to determine the number of articles that fulfilled >1 of these 12 standards. The median number of the 12 standards fulfilled per article was 6 (range 1 to 12, 25th to 75th percentile, 5.0 to 8.5). There was no difference between the general and specialty journals in the median number of the 12 standards fulfilled per article (general, median 7, 25th to 75th percentile, 6.0 to 10.0, vs specialty, median, 6, 25th to 75th percentile, 3.0 to 8.3, p=0.4). Figure 1 shows the distribution of the values for the number of the 12 design standards fulfilled by the 41 study articles.
Table 3 displays the methods the 41 study articles used to describe the diagnostic accuracy of the evaluated diagnostic tests. Proportions were calculated using a denominator of 41 except for ROC AUC, which used a denominator of 21 because 21 of the 41 studies had ordinal or continuous values for test results. Seven (17%) studies did not provide any of the measures of diagnostic accuracy listed in Table 3 and seven (17%) studies provided information only for sensitivity and specificity read only canadian health&care mall. Table 3 also shows the number of articles that reported a measure of precision for the estimates of diagnostic accuracy.
Of the 21 study articles that examined diagnostic tests with ordinal or continuous result values that required determination threshold, 11 articles clearly described their method for determining the threshold value. Eight of these 11 studies used an accepted technique for the decision threshold determination.
Twenty-seven of the 41 studies described a role in clinical medicine for the diagnostic test under evaluation without making a statement that additional validating studies were needed. Fourteen studies made explicit statements that additional studies were needed before the tests should be used in routine clinical practice. Only one study discussed method-ologic flaws and their potential impact on conclusions.

Figure 1. Graph bars denote the number of the 12 major study design standards that were fulfilled by individual study articles (n=41).

Figure 1. Graph bars denote the number of the 12 major study design standards that were fulfilled by individual study articles (n=41).
Table 2—Number of the 41 Study Articles That Fulfilled Experimental Design Standards

No. (%)
Clear definition of reference standard 31(76)
Suitability of reference standard 26 (63)
Spectrum composition 25 (61)
Indicated study sample 37(90)
Patient sampling techniques 22 (54)
Pertinent subgroup analysis 12 (29)
Test review bias 15 (37)
Workup bias 26 (63)
Reported indeterminate values 34 (83)
Reported whether indeterminate values were 14 (67)*
used for calculating diagnostic accuracy
Calculated measures of diagnostic accuracy with 3(14)*
and without indeterminate results
Test reproducibility 7(17)
Objective tests 4 (18)f
Subjective tests 3 (15)*
Test acceptability 6(15)
Incremental value reported 31(76)
Diagnostic accuracy 30 (73)
Relative cost 7(17)
Feasibility 8(20)
Risk or invasiveness 8(20)
Head-to-head comparisons
Comparisons performed 17(41)
Consecutive or random patients 15 (88)
Appropriate statistical methods for comparison 4 (24)

Table 3—Number of the 41 Study Articles That Reported Various Measures of Diagnostic Accuracy With Precision Estimates

No. (%)
Sensitivity and specificity 33 (80)
Estimate of precision 9 of 33
Predictive values 24 (59)
Estimate of precision 7 of 24
Likelihood ratios 4(10)
Estimate of precision 2 of 4
Odds ratios 0(0)
Estimate of precision 0 of 0
ROC AUC 4(19)*
Estimate of precision 2 of 4

About this blog

Blog invites submissions of review articles, reports on clinical techniques, case reports, conference summaries, and articles of opinion pertinent to the control of pain and anxiety in dentistry.