Machine Learning in Medical Sciences

When Algorithms Meet Clinical Data

Learning Objectives

  1. Distinguish supervised from unsupervised learning
  2. Understand common algorithms: logistic regression, decision trees, random forests
  3. Explain the bias-variance trade-off and overfitting
  4. Interpret AUC, sensitivity, specificity as ML evaluation metrics
  5. Understand training, validation, and test sets
  6. Evaluate ML claims in medical literature critically
  7. Recognise ethical concerns of ML in healthcare

Clinical Hook: The SONA Score

The AIIMS Bhopal team built the SONA questionnaire for OSA screening (Joshi A, Goyal A, Pakhare A, 2025):

  1. Data: 1,015 adults (BLESS cohort), all with PSG
  2. Split: Training (n = 732) / Testing (n = 244)
  3. Models: Four predictive models compared
  4. Selection: Best model by AUC = 0.83
  5. Validation: Confirmed on held-out test set
  6. Deploy: Coefficients → simple point score

. . .

This IS machine learning — even though it used logistic regression. ML is not about fancy algorithms. It’s about the discipline of training on one dataset and testing on another.

What Is Machine Learning?

Algorithms that learn patterns from data to make predictions — without explicit rules.

Traditional Statistics Machine Learning
Goal Understand relationships (inference) Make accurate predictions
Model choice Theory-driven Data-driven
Interpretability Usually high Variable
Validation CI, p-values Train/test split, AUC
Overfitting Managed by theory Managed by held-out test data

Tip

Logistic regression is BOTH statistics and ML. The difference isn’t the math — it’s how you use it. Inference → statistics. Prediction with train/test → ML.

Supervised vs Unsupervised Learning

Supervised

Algorithm learns from labelled examples.

Task Output Example
Classification Category OSA yes/no
Regression Number Predict AHI

Unsupervised

Algorithm finds hidden structure — no labels.

Task What Example
Clustering Groups HTN subtypes
Dimensionality Compress Gene expression

Common Algorithms

Algorithm Interpretability Data needed Best for
Logistic regression ★★★★★ Small–medium Risk scores (SONA)
Decision tree ★★★★☆ Small–medium Clinical decision rules
Random forest ★★☆☆☆ Medium–large Tabular clinical data
Gradient boosting ★★☆☆☆ Medium–large Prediction competitions
Neural network ★☆☆☆☆ Very large Images, text, ECG signals

For clinical screening tools (like SONA), logistic regression is often the best choice — interpretable, works with smaller data, and coefficients convert to simple point scores.

Decision Tree — Intuitive Splits

The Bias-Variance Trade-off

The gap between training and test accuracy = overfitting. Training accuracy is advertising; test accuracy is truth.

Overfitting — The Central Problem

Signs: Training 98% → Test 65% | Too many features for sample size | Absurd predictions at extremes

Cross-Validation

Use when: Not enough data for a separate test set. More robust than a single split.

ROC Curve and AUC

AUC: 0.50 = coin flip | 0.70–0.79 = acceptable | 0.80–0.89 = excellent | 0.90+ = outstanding

Accuracy Is Misleading

Disease prevalence = 1%. Model predicts “no disease” for everyone.

Metric Value
Accuracy 99% ← looks great!
Sensitivity 0% ← catches NOTHING
Specificity 100%
PPV Undefined

Warning

Never evaluate a model on accuracy alone. With imbalanced classes, accuracy is meaningless. Always report sensitivity, specificity, PPV, NPV, and AUC.

ML in Indian Healthcare

Application Technology Impact
Diabetic retinopathy Deep learning on retinal images (Google + Aravind Eye Hospital) Screening where ophthalmologists are scarce
TB from chest X-ray CAD4TB, qXR (Qure.ai) — WHO recommended Active case-finding in high-burden settings
ECG interpretation AI on portable ECG (SanketLife) PHCs without cardiologists
OSA screening SONA score (AIIMS Bhopal) — logistic regression Primary care tool for resource-limited settings

Common thread: ML is most impactful where specialist access is limited — extending expert-level decisions to primary care.

Critical Appraisal of ML Papers

# Question Red flag
1 Was data split into training and testing? Only training accuracy reported
2 Was external validation performed? Tested only where it was developed
3 Were multiple metrics reported (not just accuracy)? Only accuracy or AUC
4 Was overfitting addressed? High accuracy + complex model + small sample
5 Were predictors available at time of decision? Predictors include future information (data leakage)
6 Is the model clinically useful (better than current practice)? Marginal improvement over simple clinical rules

Cardinal rule: If they only report training performance → the results are meaningless.

Ethical Concerns

Warning

  1. Algorithmic bias — model trained on private hospital data fails in government hospital (different demographics, severity)
  2. Black box problem — if you can’t explain why the model flagged a patient, should you trust it?
  3. Data quality — “ML on dirty data = confidently wrong answers”
  4. Temporal drift — disease patterns change; models need retraining
  5. Accountability — who is responsible when AI makes a wrong prediction?
  6. Equity — dermatology AI trained on light skin performs poorly on dark skin

Always validate across diverse populations before deployment.

Identify the Approach — Vignette 1

A district hospital has 5 years of lab data on 10,000 patients. They want to identify hidden patient subtypes among diabetics based on HbA1c trajectories, comorbidity patterns, and treatment responses — without predefining the groups.

Unsupervised learning (clustering) — There are no predefined labels; the goal is to discover natural groupings. K-means or hierarchical clustering could identify phenotypic subtypes of diabetes, potentially guiding personalised treatment strategies.

Identify the Approach — Vignette 2

A research team develops an AI system to detect pneumothorax on chest X-rays using 50,000 labelled images from a tertiary hospital. They report AUC = 0.97 on an internal test set. A PHC wants to deploy it.

Supervised learning (classification) — but needs external validation first. The AUC = 0.97 is on internal data. The PHC has different X-ray machines, image quality, and patient population. The model MUST be tested on PHC data before deployment. Internal AUC ≠ real-world performance.

Identify the Approach — Vignette 3

A researcher with data from 80 patients and 200 clinical variables builds a random forest model for predicting ICU mortality. Training accuracy is 96%. She has no separate test set.

This is almost certainly overfitted. 200 features on 80 patients (features > patients!) is a recipe for overfitting. The 96% training accuracy is meaningless without a test set. She should use cross-validation, reduce features (variable selection), or use a simpler model (logistic regression with ≤5 predictors).

Common Mistakes

Warning

  1. Testing on training data — inflates accuracy
  2. Reporting only accuracy — misleading with class imbalance
  3. Data leakage — including future/outcome data in predictors
  4. Too many features — 200 variables on 80 patients = overfitting
  5. No external validation — model may only work at your hospital
  6. Ignoring clinical context — AUC 0.85 with no clinical workflow is useless
  7. Claiming causation — ML finds associations, not causes
  8. Ignoring bias — training on one population, deploying on another

Key Takeaways

  1. ML = disciplined prediction using train/test separation — not magic
  2. Logistic regression IS ML — SONA is a perfect example
  3. Always ask for test set performance — training accuracy is advertising
  4. AUC alone is not enough — report sensitivity, specificity, calibration
  5. External validation is essential — internal performance ≠ real-world
  6. Interpretability matters — if you can’t explain it, clinicians won’t trust it
  7. Check for bias — was training data representative of deployment population?
  8. AI assists, humans decide — ML is a diligent junior colleague, not a replacement