6  Diagnostic Test Evaluation

The Clinician’s Toolkit for Measuring How Well Tests Work

Diagnostic Statistics
Clinical Epidemiology
ROC Analysis

Lecture slides for this module: Open Slides

6.1 Learning Objectives

By the end of this module, you will be able to:

  1. Construct a 2×2 contingency table with the index test in rows and gold standard in columns
  2. Calculate sensitivity, specificity, PPV, and NPV from the 2×2 table
  3. Explain how prevalence transforms predictive values — the same test behaves differently in different populations
  4. Compute and interpret likelihood ratios (LR+ and LR−) as prevalence-independent measures of test performance
  5. Use the Fagan nomogram to move from pre-test to post-test probability
  6. Construct and interpret ROC curves and understand AUC
  7. Choose the optimal cutoff for a test based on clinical context
  8. Decide when to prioritise sensitivity vs specificity in test selection
  9. Apply these concepts to Indian clinical scenarios: cervical cancer screening, TB, and malaria
Builds on Module 4

This module assumes you understand conditional probability, Bayes’ theorem, and natural frequencies from Module 4: Probability and Clinical Reasoning. If concepts like “pre-test probability” or “likelihood ratio” feel unfamiliar, review Module 4 first.


6.2 Real-World Dilemma: Cervical Cancer Screening in Rural India

India bears nearly a quarter of the global cervical cancer burden. In many rural districts, Pap smear infrastructure is unavailable — cytology labs are hours away and results take weeks. So the government promotes VIA (Visual Inspection with Acetic Acid) as an alternative screening test.

A community health worker in a tribal district of Madhya Pradesh screens 1000 women with VIA. The results are compared against colposcopy-directed biopsy (the gold standard).

VIA test characteristics:

  • Sensitivity: ~80%
  • Specificity: ~85%

Pap smear test characteristics:

  • Sensitivity: ~60%
  • Specificity: ~95%

The district medical officer asks: “VIA is more sensitive, so isn’t it the better test?”

The answer is: it depends on what you need the test to do — and understanding that requires the full diagnostic test evaluation toolkit.

This module gives you that toolkit.


6.3 Part 1: The 2×2 Contingency Table

Every diagnostic test evaluation begins with comparing the index test (the test being evaluated) against a gold standard (the reference standard that defines “true” disease status).

The Standard Layout

The conventional arrangement places the index test result in rows and the gold standard in columns:

Table 6.1: Standard 2×2 Table Layout: Index Test in Rows, Gold Standard in Columns
Gold Standard
Disease Present Disease Absent Row Total
**Test Positive** True Positive (TP) False Positive (FP) TP + FP
**Test Negative** False Negative (FN) True Negative (TN) FN + TN
**Total** TP + FN FP + TN N

Key definitions:

  • True Positive (TP): Test says positive, patient truly has the disease
  • False Positive (FP): Test says positive, but patient does NOT have the disease (right upper corner — “false alarm”)
  • False Negative (FN): Test says negative, but patient truly HAS the disease (left lower corner — “missed case”)
  • True Negative (TN): Test says negative, patient truly does not have the disease

Worked Example: VIA Screening in Rural MP

Let’s construct the 2×2 table for the VIA cervical cancer screening scenario. Suppose in a population of 1000 women screened, cervical precancer/cancer prevalence is 5%.

Table 6.2: 2×2 Table: VIA Screening for Cervical Precancer (n = 1000, Prevalence = 5%)
Colposcopy-Directed Biopsy (Gold Standard)
CIN2+ Present (Biopsy) CIN2+ Absent (Biopsy) Row Total
**VIA Positive** 40 143 183
**VIA Negative** 10 807 817
**Total** 50 950 1000

Reading the table:

  • Of 50 women with CIN2+ (precancer), VIA correctly identified 40 (true positives) and missed 10 (false negatives)
  • Of 950 women without CIN2+, VIA correctly cleared 807 (true negatives) but falsely flagged 143 (false positives)

Visual: The 2×2 Table as an Icon Array

Figure 6.1: 1000 Women Screened with VIA: Each square is one woman. Green = correct result, Red = incorrect result. Note the large number of false positives (red, bottom-right cluster) even though specificity is 85%.

Common Mistake: Getting the 2×2 Layout Wrong

Many students place disease status in rows and test result in columns. While mathematically equivalent, the standard convention (as used in most textbooks, NEET PG, and USMLE) places the index test in rows and the gold standard (reference) in columns. This means:

  • FP is in the top-right cell (test positive, disease absent)
  • FN is in the bottom-left cell (test negative, disease present)

Getting this layout right is essential for exam questions.


6.4 Part 2: Sensitivity and Specificity — Intrinsic Properties of the Test

These two measures describe how well the test performs by itself — they don’t change when you take the test to a different population (unlike PPV and NPV, which do).

Sensitivity and Specificity

Sensitivity (True Positive Rate):

\[\text{Sensitivity} = \frac{TP}{TP + FN} = \frac{\text{True Positives}}{\text{All Diseased}}\]

“Of everyone who truly HAS the disease, what fraction does the test correctly identify?”

Specificity (True Negative Rate):

\[\text{Specificity} = \frac{TN}{FP + TN} = \frac{\text{True Negatives}}{\text{All Non-Diseased}}\]

“Of everyone who truly does NOT have the disease, what fraction does the test correctly clear?”

VIA vs Pap Smear: Head-to-Head

Figure 6.2: VIA vs Pap Smear: A Trade-Off. VIA catches more cases (higher sensitivity) but produces more false alarms (lower specificity). Pap misses more cases but has fewer false alarms.

Worked Calculations

VIA:

\[\text{Sensitivity} = \frac{40}{40 + 10} = \frac{40}{50} = 80\%\]

\[\text{Specificity} = \frac{807}{143 + 807} = \frac{807}{950} = 85\%\]

Pap smear:

\[\text{Sensitivity} = \frac{30}{30 + 20} = \frac{30}{50} = 60\%\]

\[\text{Specificity} = \frac{902}{48 + 902} = \frac{902}{950} = 95\%\]

The Clinical Mnemonics

  • SnNOutSensitive test, Negative result, rules Out disease. If a highly sensitive test is negative, you can confidently say the patient doesn’t have the disease.
  • SpPInSpecific test, Positive result, rules In disease. If a highly specific test is positive, you can confidently say the patient has the disease.

VIA is more useful for SnNOut (screening: don’t miss cases). Pap is more useful for SpPIn (confirmation: don’t overtreat).


6.5 Part 3: Predictive Values — How Prevalence Changes Everything

As you learned in Module 4, the same test behaves very differently depending on how common the disease is in the population being tested. Sensitivity and specificity are properties of the test. Predictive values are properties of the test in a specific population.

Positive and Negative Predictive Values

Positive Predictive Value (PPV):

\[\text{PPV} = \frac{TP}{TP + FP} = \frac{\text{True Positives}}{\text{All Positive Tests}}\]

“If the test is positive, what’s the probability the patient actually has the disease?”

Negative Predictive Value (NPV):

\[\text{NPV} = \frac{TN}{FN + TN} = \frac{\text{True Negatives}}{\text{All Negative Tests}}\]

“If the test is negative, what’s the probability the patient truly doesn’t have the disease?”

VIA Predictive Values at 5% Prevalence

\[\text{PPV}_{\text{VIA}} = \frac{40}{40 + 143} = \frac{40}{183} = 21.9\%\]

\[\text{NPV}_{\text{VIA}} = \frac{807}{10 + 807} = \frac{807}{817} = 98.8\%\]

Interpretation: Of every 100 women who screen VIA-positive, only about 22 actually have CIN2+. The remaining ~78 will undergo unnecessary colposcopy — anxiety, cost, and discomfort for nothing.

VIA vs Pap: Predictive Value Comparison

Table 6.3: Head-to-Head: VIA vs Pap Smear at 5% Prevalence
Metric VIA Pap Smear Clinical Implication
Sensitivity 80% 60% VIA catches more cases (better for screening)
Specificity 85% 95% Pap has fewer false alarms (better for confirmation)
PPV 21.9% 38.5% Pap: fewer unnecessary colposcopies
NPV 98.8% 97.8% Both excellent for ruling out
False Positives (per 1000) 143 48 VIA: 142 unnecessary referrals vs Pap: 48
False Negatives (per 1000) 10 20 VIA: 10 missed vs Pap: 20 missed

The Prevalence Effect: Same Tests, Different Populations

Figure 6.3: PPV vs Prevalence for VIA and Pap Smear. At low prevalence (screening camps), both tests have low PPV — but Pap’s higher specificity gives it an advantage. At higher prevalence (referral clinics), both perform well.

Why India Chose VIA for Community Screening

Despite its lower specificity (more false positives), VIA was chosen for the national cervical cancer screening programme because:

  1. Higher sensitivity catches more precancers — critical when follow-up is uncertain in rural settings
  2. “See and treat” approach — VIA-positive women can receive cryotherapy the same day, avoiding loss to follow-up
  3. No lab infrastructure needed — works with acetic acid, a speculum, and a trained health worker
  4. Lower specificity is acceptable when the cost of a false positive (an unnecessary colposcopy) is much less than the cost of a false negative (missed cancer progressing to invasive disease)

The test selection depends on the clinical context and healthcare system, not just the numbers.


6.6 Part 4: Likelihood Ratios — The Prevalence-Independent Power Tool

Sensitivity and specificity describe the test. PPV and NPV depend on prevalence. Is there a single measure that captures test performance and works across all populations?

Yes: Likelihood Ratios. As introduced in Module 4, they connect directly to Bayesian updating.

Likelihood Ratios

Positive Likelihood Ratio (LR+):

\[LR^+ = \frac{\text{Sensitivity}}{1 - \text{Specificity}} = \frac{TPR}{FPR}\]

“How many times more likely is a positive result in someone with disease vs. without?”

Negative Likelihood Ratio (LR−):

\[LR^- = \frac{1 - \text{Sensitivity}}{\text{Specificity}} = \frac{FNR}{TNR}\]

“How much less likely is a negative result in someone with disease vs. without?”

Interpretation Scale

Table 6.4: Likelihood Ratio Interpretation Scale
LR+ Value Shift in Probability Clinical Action
> 10 Large increase (~+45%) Near-diagnostic — strong rule-in
5 – 10 Moderate increase (~+30%) Often sufficient for treatment decisions
2 – 5 Small increase (~+15%) Helpful in combination with other evidence
1 – 2 Minimal change Rarely useful alone
1 No change (useless test) Discard the test

VIA and Pap Smear Likelihood Ratios

Test LR+ LR− Interpretation
VIA 5.3 0.24 Moderate rule-in (LR+ ~5), modest rule-out (LR− = 0.24)
Pap Smear 12 0.42 Strong rule-in (LR+ = 12), weak rule-out (LR− = 0.42)

Key insight: Pap has a much higher LR+ (12 vs 5.3) — so a positive Pap is more convincing than a positive VIA. But VIA has a better LR− (0.24 vs 0.42) — so a negative VIA is more reassuring than a negative Pap.

This quantifies the SnNOut/SpPIn trade-off precisely.


6.7 Part 5: The Fagan Nomogram — Visual Bayesian Updating

The Fagan nomogram is a graphical tool that lets you move from pre-test probability to post-test probability using likelihood ratios — without doing any calculation.

Figure 6.4: Pre-Test to Post-Test Probability for VIA and Pap Smear. Red lines = after positive test; Blue lines = after negative test. Pap’s positive result (LR+ = 12) shifts probability more dramatically than VIA’s (LR+ = 5.3).
Reading the Fagan Nomogram
  1. Start on the x-axis at your pre-test probability (clinical estimate before any test)
  2. Draw a vertical line up to the relevant curve (positive or negative result)
  3. Read across to the y-axis — that’s your post-test probability

At 5% pre-test probability:

  • VIA positive → post-test ~22% (needs confirmation)
  • Pap positive → post-test ~39% (stronger evidence, may warrant treatment)
  • VIA negative → post-test ~1.2% (good rule-out)
  • Pap negative → post-test ~2.2% (adequate rule-out, but worse than VIA)

6.8 Part 6: ROC Curves — Choosing the Best Cutoff

Many diagnostic tests don’t give a simple “positive/negative” — they produce a continuous measurement (blood glucose, tumour marker, antibody titre). We choose a cutoff to classify results as positive or negative. Different cutoffs give different sensitivity-specificity trade-offs.

The ROC (Receiver Operating Characteristic) curve plots sensitivity vs (1 − specificity) for every possible cutoff, showing you the complete trade-off landscape.

Building an ROC Curve: Fasting Glucose for Diabetes

Figure 6.5: ROC Curve for Fasting Blood Glucose as a Diabetes Screening Test. Each point is a different cutoff value. The curve shows the trade-off: lowering the cutoff catches more diabetics (higher sensitivity) but also more false positives (lower specificity).

Understanding the ROC Curve

Moving along the curve:

  • Top-left corner = perfect test (sensitivity = 100%, FPR = 0%)
  • Bottom-right = useless test (sensitivity = 0%, FPR = 100%)
  • Diagonal = no discrimination (random coin flip)
  • Closer to top-left = better discrimination

The two cutoff points:

  • Cutoff 100 mg/dL (IFG threshold): High sensitivity (90%), moderate specificity (75%) — good for screening (don’t miss pre-diabetics)
  • Cutoff 126 mg/dL (DM threshold): Lower sensitivity (72%), high specificity (99%) — good for diagnosis (don’t overdiagnose)

AUC: Summarising Overall Test Performance

The Area Under the ROC Curve (AUC) captures the test’s overall ability to discriminate between diseased and non-diseased individuals across all possible cutoffs.

Table 6.5: AUC Interpretation Scale
AUC Range Discrimination Clinical Analogy
0.90 – 1.00 Excellent Troponin for MI — near-perfect separation
0.80 – 0.90 Good GeneXpert for TB — reliably distinguishes
0.70 – 0.80 Fair CRP for infection — helpful but imperfect
0.60 – 0.70 Poor ESR for inflammation — barely better than chance
0.50 None (coin flip) Random guess — test is worthless

Practical meaning of AUC: AUC = 0.9 means if you randomly pick one diabetic and one non-diabetic person, there’s a 90% chance the diabetic has a higher fasting glucose.

Comparing Two Tests: ROC Curves Side by Side

Figure 6.6: Comparing Two Diagnostic Tests by ROC Curves. The curve closer to the top-left corner is the better test. The difference in AUC quantifies how much better.

6.9 Part 7: Choosing the Right Test — When Sensitivity vs Specificity Matters

Different clinical situations demand different test properties. The choice is not about which test is “better” in the abstract — it’s about which test fits the clinical question.

Table 6.6: When to Prioritise Sensitivity vs Specificity
Clinical Scenario Priority Why Indian Example
Community screening (healthy population) HIGH Sensitivity Don't miss cases — false negatives are costly VIA for cervical cancer; Rapid antigen for COVID
Confirmation after positive screen HIGH Specificity Reduce false positives before invasive treatment Colposcopy + biopsy after VIA+; RT-PCR after antigen+
Ruling out a dangerous condition HIGH Sensitivity False negative = patient sent home with dangerous disease D-dimer for PE; CT head for stroke
Disease very rare in population HIGH Specificity Low prevalence → FP vastly outnumber TP → need high Spec HIV screening in low-prevalence population
Disease very common (outbreak) HIGH Sensitivity High prevalence → TP outnumber FP → Sens determines yield Malaria RDT during monsoon outbreak
Treatment is toxic or irreversible HIGH Specificity False positive → unnecessary harm from treatment Starting chemotherapy based on biopsy

The Two-Step Strategy: Screen Then Confirm

Most diagnostic pathways use this approach:

  1. Step 1 — Screening test (high sensitivity, moderate specificity): Cast a wide net, catch as many cases as possible. Accept some false positives.
  2. Step 2 — Confirmatory test (high specificity, moderate sensitivity): Among those who screened positive, use a specific test to separate true positives from false positives.
Figure 6.7: The Two-Step Diagnostic Strategy: Screen with a sensitive test, then confirm with a specific test. This approach maximises detection while minimising unnecessary treatment.
Table 6.7: Two-Step Diagnostic Pathways Used in Indian Public Health
Pathway Screening Test Confirmatory Test Screen: Sens / Spec Confirm: Sens / Spec
Cervical Cancer VIA Colposcopy + Biopsy 80% / 85% 95% / 95%
Tuberculosis Symptom screening + sputum smear GeneXpert MTB/RIF + Culture 70% / 90% 98% / 99%
HIV Rapid antibody test Western blot / NAAT 99% / 99% 99.9% / 99.9%
Breast Cancer Clinical breast exam Mammography + Biopsy 60% / 95% 90% / 95%

6.10 Part 8: Indian Clinical Scenarios

Scenario 1: TB Diagnostics — GeneXpert in a Rural PHC

Setting: Primary health centre in Bihar; TB prevalence among symptomatic patients ~3%.

Test: GeneXpert MTB/RIF — Sensitivity: 98%, Specificity: 99%

Figure 6.8: GeneXpert for TB: Excellent discrimination. At 3% prevalence, PPV = 75% and NPV = 99.9%. This is what happens when you have very high specificity — PPV stays useful even at low prevalence.

Why GeneXpert works so well even at low prevalence: Its 99% specificity is the key. At 3% prevalence, there are 9,700 non-diseased people. With 99% specificity, only 97 test falsely positive. Compare this to a test with 90% specificity — that would give 970 false positives, destroying the PPV. High specificity protects PPV at low prevalence.

Scenario 2: Malaria RDT Across Seasons

Setting: Monsoon season in a tribal belt of Chhattisgarh — malaria prevalence ~25%. Dry season in the same region — prevalence drops to ~2%.

Test: Rapid Diagnostic Test for P. falciparum — Sensitivity: 85%, Specificity: 95%

Figure 6.9: Malaria RDT: Same test, different seasons, dramatically different PPV. During monsoon (25% prevalence), PPV = 85% — highly useful. During dry season (2%), PPV drops to 26% — most positives are false alarms.

Clinical action: During monsoon, a positive RDT is reliable enough to start treatment. During dry season, a positive RDT needs confirmation with peripheral smear microscopy.

Scenario 3: Cervical Cancer Screening — VIA in the “See and Treat” Model

Figure 6.10: The ‘See and Treat’ Model: VIA’s strength is not in PPV — it’s in the ability to immediately treat positives with cryotherapy on the same visit, preventing loss to follow-up.

The programme logic: In rural India where follow-up is unreliable, VIA + immediate cryotherapy catches 80% of precancers in a single visit. The 142 women who receive unnecessary cryotherapy (false positives) experience minimal harm — cryotherapy is a simple, safe outpatient procedure. But the 40 women whose precancers are caught and treated are spared invasive cervical cancer. The trade-off is overwhelmingly favourable.

This is a clear example of why test selection depends on the healthcare context, not just the statistical properties.


6.11 Summary and Key Takeaways

What this module added to Module 4’s foundation:

  1. The 2×2 table is the organising structure — index test in rows, gold standard in columns, FP in the top-right corner

  2. Sensitivity and specificity are intrinsic test properties that don’t change across populations. PPV and NPV do change with prevalence.

  3. Likelihood ratios are the prevalence-independent bridge — LR+ > 10 is near-diagnostic, LR− < 0.1 is near-exclusion

  4. ROC curves show the complete sensitivity-specificity trade-off for continuous tests. AUC summarises overall discrimination.

  5. Test selection depends on clinical context: screening → high sensitivity (SnNOut); confirmation → high specificity (SpPIn)

  6. Two-step strategy (screen then confirm) is the standard approach in public health and clinical practice

  7. Context matters enormously — VIA was chosen for India not because it has the best PPV, but because it fits a “see and treat” model that prevents loss to follow-up


6.12 Further Learning Resources

Video Lectures

StatQuest with Josh Starmer — “Sensitivity and Specificity” (YouTube) Clear visual explanation of the 2×2 table and its derivatives.

StatQuest — “ROC and AUC, Clearly Explained!” (YouTube) The definitive visual guide to ROC curves.

Zedstatistics (Andrew Mead) — “Diagnostic Test Evaluation” playlist (YouTube) Step-by-step walkthrough with clinical examples.

Textbooks

Sackett DL et al (2005). Clinical Epidemiology: A Basic Science for Clinical Medicine. 3rd ed. Ch 4: Diagnosis. Gold standard for diagnostic test evaluation in clinical context.

Kirkwood BR & Sterne JAC (2003). Essential Medical Statistics. 2nd ed. Ch 36: Diagnostic tests and screening. Clear statistical treatment with worked examples.

Indian Context

NTEP (National TB Elimination Program) Guidelines — GeneXpert sensitivity/specificity recommendations for Indian populations.

Operational Framework for Cervical Cancer Screening and Management (MOHFW, India) — VIA implementation guidelines, “see and treat” protocol.

NVBDCP Guidelines — Malaria RDT use across endemic and non-endemic zones.


6.13 Practice MCQs: NEET PG Level

Q1. In the standard 2×2 table for diagnostic test evaluation with the index test in rows and gold standard in columns, which cell represents false positives?


Q2. VIA screening for cervical cancer has 80% sensitivity and 85% specificity. Pap smear has 60% sensitivity and 95% specificity. In a population screening programme where follow-up is unreliable, which test is preferred and why?


Q3. A tuberculosis test has sensitivity 98% and specificity 99%. At 3% prevalence, 1,000 patients are tested. How many false positives will occur?


Q4. A malaria RDT has sensitivity 85% and specificity 95%. During monsoon (prevalence 25%), LR+ = 17. During dry season (prevalence 2%), LR+ is still 17. Why does PPV change so dramatically between seasons if LR+ stays the same?


Q5. An ROC curve for a new biomarker has AUC = 0.55. What does this mean clinically?


6.14 References