Correlation and Regression

From Measuring Association to Predicting Outcomes

The Clinical Problem

A research team at JIPMER, Puducherry studies 200 adults at a health screening camp.

Questions that cascade:

  1. Is higher BMI associated with higher FBS? → Correlation
  2. Can we predict FBS from BMI? → Linear regression
  3. Does the association persist after adjusting for age and sex? → Multiple regression
  4. What if the outcome is diabetes (yes/no)? → Logistic regression

Roadmap

What Correlation Measures

In words: As one variable goes up, does the other tend to go up, go down, or show no pattern?

\[r = \frac{\text{Covariance}(x, y)}{SD_x \times SD_y}\]

Range Meaning
r = +1 Perfect positive linear
r = 0 No linear association
r = −1 Perfect negative linear

Two versions: Pearson (for continuous, linear) and Spearman (for ordinal, or when outliers/skew present)

BMI vs. FBS: Scatter Plot

r = 0.448 — moderate positive correlation

Pearson vs. Spearman

Feature Pearson (r) Spearman (rₛ)
Measures Linear association Monotonic association
Data Continuous, ~normal Ordinal or skewed
Outlier-sensitive? Yes No (uses ranks)
Use when Scatter looks linear, no extreme outliers Ranks, ordinal data, non-linear monotonic

Rule of thumb: If in doubt, compute both. If they agree, report Pearson. If they disagree substantially, investigate the scatter plot for non-linearity or outliers.

Correlation ≠ Causation

Ice cream sales ↑ and drowning deaths ↑

r ≈ 0.8 — very strong!

Should we ban ice cream?

. . .

No. Lurking variable = temperature

Sources of spurious association:

  • Confounding (third variable)
  • Reverse causality (y → x)
  • Selection bias
  • Measurement error

A strong r means variables move together — not that one causes the other.

Simple Linear Regression: The Model

In words: Predicted outcome = Baseline + (Rate of change × Predictor) + Error

\[y = \beta_0 + \beta_1 x + \varepsilon\]

  • β₀ (intercept): Predicted y when x = 0
  • β₁ (slope): Change in y for each 1-unit increase in x
  • ε: What the model can’t explain

Least squares finds β₀ and β₁ that minimise \(\sum(y_i - \hat{y}_i)^2\)

Regression: BMI → FBS

Slope: Each 1 kg/m² BMI increase → 1.9 mg/dL higher FBS

R²: Variance Explained

In words: What fraction of the total scatter in y is captured by the regression line?

\[R^2 = 1 - \frac{\sum(y_i - \hat{y}_i)^2}{\sum(y_i - \bar{y})^2}\]

For simple regression: \(R^2 = r^2\)

Our model: R² = 0.201 → BMI explains 20.1% of FBS variation

The remaining 79.9% is due to age, diet, genetics, measurement error, etc.

Confidence vs. Prediction Interval

Regression Assumptions: LINE

  1. Linearity — scatter plot shows a linear trend
  2. Independence — observations are independent
  3. Normality — residuals (not x!) are approximately normal
  4. Equal variance — residual spread is constant (homoscedasticity)

Check with diagnostic plots:

  • Residuals vs. Fitted → random cloud (not curved, not funnel)
  • Q-Q plot → points on diagonal
  • Scale-Location → flat trend
  • Cook’s distance → no single observation dominates

What to Do When Assumptions Fail

Problem Sign Fix
Non-linear Curved residuals Transform x or y; polynomial term
Non-normal residuals Q-Q deviates at tails Transform y; often OK if n > 30
Heteroscedasticity Fan-shaped residuals Log-transform y; robust SE
Influential outlier Cook’s distance > 0.5 Investigate; report with and without

Multiple Regression: Adjusting for Confounders

Question: Is the BMI–FBS association genuine, or confounded by age?

\[\text{FBS} = \beta_0 + \beta_1 \times BMI + \beta_2 \times Age + \beta_3 \times Sex + \varepsilon\]

In words: Each coefficient tells you the effect of that variable while all others are held fixed.

Model BMI slope 95% CI Adj R²
Simple (BMI only) 1.93 1.39 – 2.47 0.197
Adjusted (+ Age, Sex) 2.01 1.5 – 2.53 0.275

If the slope changes after adjustment → confounding was present.

Logistic Regression: Binary Outcomes

When the outcome is yes/no (diabetes, death, readmission), linear regression breaks down. Logistic regression models the log-odds:

\[\ln\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x\]

The odds ratio for a 1-unit increase in x:

\[OR = e^{\beta_1}\]

Connection to Module 10: The 2×2 table OR is a special case. Logistic regression is the multivariable extension.

Logistic Regression: Predicted Probabilities

The sigmoid curve — probability bounded between 0 and 1

Decision Framework

Scenario Method
Strength of association (two continuous)? Pearson r
Ordinal data or outliers? Spearman rₛ
Predict continuous outcome (one predictor)? Simple linear regression
Adjust for confounders? Multiple linear regression
Predict binary outcome (yes/no)? Logistic regression
Predict time-to-event? Cox regression (Module 12)

Common Pitfalls

Avoid

  • Assuming correlation proves causation
  • Using linear regression for binary outcomes
  • Ignoring residual plots
  • Extrapolating beyond the data range
  • Reporting only R² without checking assumptions

Remember

  • r measures linear association only
  • R² = r² for simple regression
  • Adjusted R² for comparing models
  • Logistic regression → OR, not risk ratio
  • Regression adjusts for measured confounders only

Key Takeaways

  1. Pearson r = linear association; Spearman rₛ = rank-based alternative
  2. Correlation ≠ causation — always consider confounding
  3. Simple regression: y = β₀ + β₁x; slope = change per unit; R² = variance explained
  4. Check LINE assumptions with diagnostic plots
  5. CI for the mean; PI for a new individual — always report uncertainty
  6. Multiple regression adjusts for confounders; use adjusted R²
  7. Logistic regression for binary outcomes → coefficients exponentiate to ORs
  8. No regression proves causation in observational data — only RCTs can