Study Design, Bias, and Confounding

Why Study Design Determines What You Can Conclude

Learning Objectives

  1. Describe the purpose, logic, and key features of cross-sectional, case-control, cohort, and RCT designs
  2. Choose the right design for a given research question
  3. Calculate prevalence, OR, RR, and AR from 2×2 tables
  4. Distinguish selection bias, information bias, and confounding
  5. Use DAGs to reason about confounding
  6. Evaluate the hierarchy of evidence

Clinical Hook: Does Wine Prevent Heart Disease?

A 10-year cohort study (n = 20,000) found:

Light wine drinkers had 30% lower MI risk vs non-drinkers.

Headlines: “Wine is Cardioprotective!”

But wait — wine drinkers also:

  • Had higher incomes (better healthcare access)
  • Exercised more regularly
  • Had lower smoking rates
  • Ate more fruits and vegetables

The wine–MI association may be entirely due to confounding by socioeconomic status and lifestyle.

This module teaches you to think critically about how a study was designed before trusting what it found.

Can You Fill In This Design?

Clue: This design takes a snapshot — everything is measured at one time point.

Answer: Cross-sectional study — population → classify exposed/non-exposed → assess outcome simultaneously.

Cross-Sectional Study

Purpose: Snapshot — measure exposure and outcome at the same time

How it works:

  1. Select a sample from the population
  2. Measure exposure and outcome simultaneously
  3. Compare prevalence of outcome in exposed vs non-exposed

Key features:

  • Quick, inexpensive
  • Cannot establish temporal sequence (chicken or egg?)
  • Best for: prevalence estimation, hypothesis generation

Measure: Prevalence, Prevalence Odds Ratio (POR)

Example: Measuring BP and salt intake in 500 adults at a health camp

Cross-Sectional: What Can You Measure?

Disease + Disease −
Exposed a b
Non-exposed c d

Prevalence = (a + c) / (a + b + c + d) — proportion with disease at that time point

Prevalence Odds Ratio = (a × d) / (b × c) — how much more common is disease among exposed?

Limitation: You measured exposure and outcome simultaneously — you can’t tell which came first. Association ≠ temporal sequence.

Can You Fill In This Design?

Clue: This design starts from the outcome and looks backward in time. Efficient for rare diseases.

Answer: Case-control study — select cases & controls → ascertain past exposure.

Case-Control Study

Purpose: Start with the disease and look backward for exposure

How it works:

  1. Identify people WITH the disease (cases)
  2. Select similar people WITHOUT the disease (controls)
  3. Compare past exposure in cases vs controls

Key features:

  • Efficient for rare diseases (don’t need to wait for events)
  • Relatively quick and cheap
  • Prone to recall bias (cases remember exposures better)
  • Cannot calculate incidence or RR directly

Measure: Odds Ratio (OR = ad/bc)

Example: 100 oral cancer patients + 200 controls — asked about paan chewing history

Case-Control: What Can You Measure?

Cases Controls
Exposed a b
Non-exposed c d

Odds Ratio = (a × d) / (b × c)

  • OR > 1 → exposure more common in cases (risk factor)
  • OR < 1 → exposure less common in cases (protective)
  • OR = 1 → no association

Why not RR? You selected cases and controls separately — you don’t know the total population at risk, so you can’t calculate incidence.

Can You Fill In This Design?

Clue: Everyone starts disease-free. You classify by exposure and follow forward over time.

Answer: Cohort study — disease-free population → classify exposed/non-exposed → follow up for outcome.

Cohort Study

Purpose: Start with exposure and follow forward to see who develops disease

How it works:

  1. Identify exposed and non-exposed groups (disease-free at start)
  2. Follow both groups over time
  3. Compare incidence of outcome in exposed vs non-exposed

Key features:

  • Establishes temporal sequence (exposure precedes outcome)
  • Can calculate incidence and RR directly
  • Efficient for rare exposures
  • Expensive and time-consuming (especially prospective)
  • Loss to follow-up is the main threat

Measure: Relative Risk (RR), Absolute Risk, Incidence

Example: Follow 5,000 factory workers (smokers vs non-smokers) for 10 years for lung disease

Cohort: What Can You Measure?

Disease + Disease − Total
Exposed a b a + b
Non-exposed c d c + d

Risk in exposed = a / (a + b)     Risk in non-exposed = c / (c + d)

Relative Risk (RR) = Risk in exposed / Risk in non-exposed

Absolute Risk Difference = Risk in exposed − Risk in non-exposed

Why cohort can do this but case-control can’t: You followed a defined population forward — so the denominators (a+b, c+d) represent the actual groups at risk.

Can You Fill In This Design?

Clue: Looks like a cohort — but the allocation to groups is done by the investigator, not nature. The red arrows are the giveaway.

Answer: Randomized Controlled Trial — eligible participants → randomize to treatment/control → follow up for outcome.

Randomized Controlled Trial (RCT)

Purpose: The gold standard — test whether an intervention causes the outcome

How it works:

  1. Randomly assign participants to treatment or control
  2. Apply intervention
  3. Follow up and compare outcomes

Key features:

  • Randomization distributes ALL confounders (known and unknown) equally
  • Only design that can establish causation
  • Highest internal validity
  • Expensive, time-consuming, not always ethical or feasible
  • Requires equipoise (genuine uncertainty about which is better)

Measure: RR, ARR, NNT

Example: Randomly assign 200 diabetics to yoga vs standard care for 6 months

RCT: What Can You Measure?

Event No Event Total
Treatment a b a + b
Control c d c + d

Control Event Rate (CER) = c / (c + d)     Treatment Event Rate (TER) = a / (a + b)

Absolute Risk Reduction (ARR) = CER − TER

Number Needed to Treat (NNT) = 1 / ARR — how many patients must be treated to prevent one event

Clinical meaning of NNT: If NNT = 20, you treat 20 patients to prevent 1 bad outcome. Lower NNT = more clinically impactful treatment.

Design Comparison at a Glance

Feature Cross-sectional Case-control Cohort RCT
Starting point Population Outcome Exposure Random assignment
Time direction Snapshot Backward Forward Forward
Measure Prevalence, POR OR RR, AR RR, ARR, NNT
Can show causation? No Suggests Suggests **Yes**
Best for rare... Neither **Disease** **Exposure** N/A
Time & cost Low Low High Very high
Main threat Temporal ambiguity Recall bias Loss to follow-up Ethical constraints

Key identifier: What is the starting point? Disease → case-control. Exposure → cohort. Random allocation → RCT.

How to Choose a Study Design

Research Question Best Design Why
What is the prevalence of diabetes in rural MP? Cross-sectional Snapshot of a population
Is gutka associated with oral cancer? Case-control Oral cancer is rare; start from cases
Does air pollution increase asthma in children? Cohort Follow exposed vs non-exposed over time
Does a new drug lower BP more than standard? RCT Need causal evidence for treatment
Is a screening test effective at reducing mortality? RCT (or large cohort) Lead-time bias threatens observational designs

The choice depends on: research question, outcome frequency, time, ethics, and resources.

Selection Bias

Systematic error in how participants are selected or retained.

Common types:

  • Berkson’s bias: Hospital controls overrepresent comorbidities
  • Volunteer bias: Participants differ from non-participants
  • Loss to follow-up: Differential dropout distorts results
  • Healthy worker effect: Workers are healthier than general population

Example:

Study of phone radiation and brain cancer using hospital controls.

  • Hospital controls may have headache disorders (more phone use)
  • Spurious protective association!
  • Fix: Use population-based controls

Key principle: Selection bias cannot be corrected after data collection. It must be prevented by design.

Information (Measurement) Bias

Systematic error in how exposure or outcome is measured.

Type What Happens Example
Recall bias Cases remember exposure better than controls Mothers of malformed babies recall drug use more
Observer bias Assessor knows exposure status Unblinded radiologist reads scans differently
Misclassification Wrong category assignment BP cuff too small → systematic overestimation
Hawthorne effect Behavior changes when observed Hand hygiene improves during audit

Solutions: Blinding, standardized instruments, objective measures, validated questionnaires

Lead-Time Bias

What Is Confounding?

A confounder is a variable that:

  1. Is associated with the exposure
  2. Is an independent cause of the outcome
  3. Is NOT on the causal pathway between exposure and outcome

Example: SES confounds the wine → heart disease association (see clinical hook)

The Wine Paradox DAG

After adjusting for SES and lifestyle, the wine–CVD association largely disappears.

Methods to Control Confounding

Stage Method How It Works
Design Randomization Distributes ALL confounders equally (known + unknown)
Design Restriction Limit to one stratum (e.g., only males)
Design Matching Select controls with same confounder values
Analysis Stratification Analyze within confounder strata separately
Analysis Multivariable regression Adjust for confounders in model

Only randomization controls for UNKNOWN confounders. All other methods require you to identify and measure confounders in advance.

Hierarchy of Evidence

Higher = stronger evidence for causation — but not always feasible or ethical!

Identify the Design — Vignette 1

A district hospital in Madhya Pradesh screens 800 pregnant women attending their first antenatal visit. Blood samples are drawn for haemoglobin, and a questionnaire records dietary iron intake. The researchers want to estimate anaemia prevalence and its association with diet.

Cross-sectional — Exposure (diet) and outcome (anaemia) measured at the same time point. No follow-up, no selection by disease. The giveaway: “screens at first visit” = single snapshot.

Identify the Design — Vignette 2

Researchers at a cancer centre identify 150 patients with gallbladder carcinoma and select 300 age- and sex-matched patients admitted for unrelated surgeries. Both groups are interviewed about typhoid history, use of mustard oil, and family history.

Case-control — Started from the disease (gallbladder cancer), selected controls, and looked backward at past exposures. The giveaway: “identify patients with disease” + “interviewed about past history.”

Identify the Design — Vignette 3

A research team enrols 3,000 non-diabetic adults aged 30–50 from an urban slum in Pune. At baseline, they record physical activity levels, BMI, and fasting glucose. They follow the cohort for 8 years, checking annually for new-onset type 2 diabetes.

Cohort — Everyone was disease-free at start, classified by exposure (physical activity), and followed forward for outcome. The giveaway: “non-diabetic at enrolment” + “follow for 8 years.”

Identify the Design — Vignette 4

Forty primary health centres in Chhattisgarh are randomly allocated: 20 receive a mHealth reminder system for DOTS adherence, 20 continue with usual care. After 12 months, TB treatment completion rates are compared.

Cluster RCT — The investigator randomly allocated the intervention. Groups were followed forward and outcomes compared. The giveaway: “randomly allocated” — nature didn’t decide, the researcher did. (Cluster because the unit of randomization is the PHC, not individual patients.)

The Sweden–India Paradox

Country Crude Death Rate Life Expectancy
India 6.4 70.8 years
Sweden 9.0 83.2 years

Sweden’s crude death rate is 41% higher — yet Swedes live 12 years longer!

. . .

Why? Sweden has 29% of its population aged 60+, vs 18% in India. More elderly = more deaths = higher crude rate.

Age is confounding the comparison.

Direct Standardization

Question: What would each country’s death rate be if both had the same age structure?

Method: Apply each country’s age-specific rates to a common standard population.

Age Group Standard Weight (Wᵢ) India ASDR (Rᵢ) Wᵢ × Rᵢ Sweden ASDR (Rᵢ) Wᵢ × Rᵢ
0–14 0.264 2.8 0.739 0.3 0.079
15–44 0.378 1.5 0.567 0.4 0.151
45–59 0.172 5.2 0.894 2.8 0.482
60–74 0.118 22.0 2.596 12.0 1.416
75+ 0.068 80.0 5.440 72.0 4.896
Total (ASR) 10.24 7.02

After standardization: India (10.2) > Sweden (7.0) — the truth emerges!

\[\text{Age-Standardized Rate} = \sum_{i} W_i \times R_i\]

Kerala vs Bihar: Same Paradox Within India

Kerala Bihar
% Population 60+ 26% 13%
Crude Death Rate 7.4 5.8
Age-Standardized Rate 5.6 8.1
Life Expectancy 76.3 yrs 69.2 yrs

Kerala’s crude rate appears higher — but after standardization, Bihar’s rate is 45% higher.

Kerala is healthier at every age. It just has more elderly people (demographic transition).

Indirect Standardization & SMR

When to use: Small populations where age-specific rates are unstable (few deaths per group).

Method: Apply the standard population’s rates to your population’s age structure → get expected deaths.

\[\text{SMR} = \frac{\text{Observed deaths}}{\text{Expected deaths}} \times 100\]

Example: A thermal power plant town in Chhattisgarh

Observed Expected (based on India rates)
Total deaths 103 75
SMR 137

Interpretation: 37% excess mortality compared to national average, after accounting for age structure.

Critical rule: SMRs from different populations are NOT comparable to each other. Only compare each SMR to 100 (the reference).

Direct vs Indirect: Quick Guide

Direct Indirect
Apply Study rates → standard weights Standard rates → study weights
Need Reliable age-specific rates Only age structure + total deaths
Get Age-standardized rate SMR (ratio)
Compare? Yes — rates are directly comparable No — SMR only vs reference (100)
Best for Large populations Small populations

Remember: Standardized rates are fictional — they tell you what would happen under a hypothetical age structure. Use crude rates for healthcare planning; use standardized rates for fair comparisons.

Key Takeaways

  1. Cross-sectional = snapshot; quick but cannot establish causation
  2. Case-control = start from disease, look backward; efficient for rare diseases
  3. Cohort = start from exposure, follow forward; can calculate RR and incidence
  4. RCT = random allocation; the only design that establishes causation
  5. Selection bias must be prevented by design — cannot be fixed in analysis
  6. Confounders must satisfy three criteria; use DAGs to reason about them
  7. Never compare crude rates across populations with different age structures — use standardized rates
  8. Choose your design based on: research question, outcome frequency, time, ethics, and resources