Session 1: Beyond Spreadsheets — Why R for HTA

The case for moving from Excel to R, told through the Reinhart-Rogoff cautionary tale

Session Overview

This opening session answers the most fundamental question of the workshop: why should HTA practitioners learn R when Excel and TreeAge already work?

We answer this through a real-world cautionary tale — the Reinhart-Rogoff incident — followed by a live demonstration of what R makes possible using an interactive Shiny app.

Session flow:

Slides: The Reinhart-Rogoff story (the spreadsheet that shook nations)
Live demo: Beyond Spreadsheets Shiny App — interactive PSA, Markov model, CEAC
Slides: Summary — what R makes possible, workshop roadmap

The Reinhart-Rogoff Incident: A Cautionary Tale

The Context

In 2008, the world was in the grip of the worst financial crisis since the Great Depression. Governments faced a critical question: should they increase public spending to stimulate the economy, or cut spending to reduce debt? The answer would affect the lives of hundreds of millions of people.

The Paper

In 2010, two Harvard economists — Carmen Reinhart and Kenneth Rogoff — published “Growth in a Time of Debt.” Their finding was dramatic: countries whose government debt exceeded 90% of GDP experienced negative average economic growth of −0.1%. The paper was cited over 3,000 times and directly influenced policy at the European Commission, UK Treasury, IMF, and U.S. Congress. Paul Ryan cited it in his Budget Committee proposal to justify austerity measures.

The Discovery

In 2013, Thomas Herndon — a 28-year-old PhD student at the University of Massachusetts Amherst — was assigned to replicate a famous economics paper. He chose Reinhart-Rogoff. He could not reproduce the results. He contacted the authors, who shared their Excel spreadsheet. What he found changed the global conversation.

The Three Errors

Error 1: Excel Row Exclusion. The AVERAGE formula in the spreadsheet covered rows 30–44 instead of 30–49. Five countries — Australia, Austria, Belgium, Canada, and Denmark — were simply left out of the calculation. This was a drag-select error, the kind that is invisible in Excel unless someone inspects every formula.

Error 2: Unconventional Weighting. New Zealand had a single year of high-debt, low-growth data (−7.6%). This one observation was given equal weight to the UK’s 19 years of data. A single outlier dragged the entire “above 90%” average into negative territory.

Error 3: Selective Data Exclusion. Post-war data from three countries was excluded without transparent justification.

The Corrected Result

When Herndon, Ash, and Pollin corrected all three errors, the average GDP growth for high-debt countries was +2.2% — positive, not negative. The entire foundation for austerity policy was based on a spreadsheet error that no peer reviewer could detect because nobody could audit the spreadsheet.

Five Lessons for HTA

Errors hide in plain sight — wrong cell ranges are invisible in Excel
Peer review failed — reviewers had no access to the underlying calculations
Replication was impossible — until the original spreadsheet was shared years later
Policy moved faster than science — the paper influenced billions before anyone checked the maths
Transparency is not optional — it is the only protection against consequential errors

Live Demo: The Shiny App

During the session, we open the Beyond Spreadsheets Shiny app for a live, interactive demonstration:

🔗 Open the Shiny App

The app has five tabs:

Tab 1: The Spreadsheet That Shook Nations

An interactive exploration of the Reinhart-Rogoff errors. Toggle which errors are present and see how the results change in real time — something impossible with the original Excel file.

Tab 2: PSA Demo

A full probabilistic sensitivity analysis in R, driven by three lines of code:

set.seed(42)
delta_cost <- rnorm(10000, mean_cost_b, sd_cost_b) - rnorm(10000, mean_cost_a, sd_cost_a)
delta_qaly <- rnorm(10000, mean_qaly_b, sd_qaly_b) - rnorm(10000, mean_qaly_a, sd_qaly_a)

The app shows the cost-effectiveness plane (with all four quadrants), a simulation table where every row is auditable, and a CEAC preview — all generated in under a second.

Try this in Excel: setting up 10,000 PSA iterations requires complex macros, runs slowly, and produces output that is nearly impossible to audit.

Tab 3: Markov Model

A three-state cohort Markov model (Healthy → Sick → Dead) with:

Interactive transition probability sliders (auto-adjusted so rows sum to 1)
Live state-transition diagram that updates with your parameters
Markov trace comparing Control and Treatment arms
Cost-effectiveness results with ICER interpretation

The core logic is a single line: trace[i,] <- trace[i-1,] %*% P

Tab 4: CEAC Explorer

A standalone cost-effectiveness acceptability curve showing how the probability of cost-effectiveness changes across willingness-to-pay thresholds. The Indian WTP benchmark of ₹2.5 lakh/QALY (1× GDP per capita) is highlighted.

Tab 5: R vs Excel

A side-by-side comparison of how the same HTA tasks look in Excel versus R — making the case for transparency, reproducibility, and scalability.

What R Makes Possible

R does not replace your HTA knowledge. It gives you a different medium to express that knowledge — one that is:

Reproducible. Your entire analysis is a script. Anyone can re-run it and get the same result. The set.seed() function ensures that even probabilistic analyses produce identical output across runs, machines, and collaborators.

Scalable. Running 10,000 PSA iterations takes a fraction of a second. Running 100,000 is equally easy. Excel slows to a crawl; R does not.

Transparent. A reviewer can read your code and see exactly what you did. No hidden cell references, no buried macros. Every assumption is explicit.

Version-controlled. With Git, every change is tracked, documented, and reversible. No more Model_v3_final_FINAL_revised.xlsx.

Free and extensible. No license fees, no vendor lock-in. Thousands of R packages exist for health economics (heemod, hesim, dampack, BCEA), survival analysis (flexsurv, survminer), Bayesian methods, and visualisation.

Workshop Roadmap

In this workshop, you will see R applied to real HTA problems across four model types:

Figure 1: Four HTA model types covered in this workshop

Day 1: Why R for HTA → R orientation → Decision trees (GDM diagnostic, DES vs BMS therapeutic)

Day 2: Markov cohort model (CKD) → Probabilistic sensitivity analysis → GenAI for HTA coding

Day 3: Partitioned survival model (Breast cancer) → PSA for PSM → Interactive Shiny apps → Wrap-up

What This Workshop Will and Won’t Do

This workshop will not turn you into R programmers in three days, ask you to memorise syntax, or expect you to write complex code from scratch.

This workshop will give you working templates you can adapt for your own studies, build your confidence in reading and modifying R code, show you how to use AI tools to accelerate your workflow, and help you understand when R is the right tool for your HTA work.

Key Takeaway

R is not replacing your HTA expertise — it is giving that expertise a more powerful medium. The clinical judgement, the model structuring, the parameter selection — that remains yours. R simply makes the execution more reproducible, transparent, and scalable.

Herndon, T., Ash, M., & Pollin, R. (2014). Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge Journal of Economics, 38(2), 257–279.
Reinhart, C. M., & Rogoff, K. S. (2010). Growth in a time of debt. American Economic Review, 100(2), 573–578.
Baio, G. (2012). Bayesian Methods in Health Economics. CRC Press.
Briggs, A., Sculpher, M., & Claxton, K. (2006). Decision Modelling for Health Economic Evaluation. Oxford University Press.
R-HTA Consortium: r-hta.org

--- title: "Session 1: Beyond Spreadsheets — Why R for HTA" subtitle: "The case for moving from Excel to R, told through the Reinhart-Rogoff cautionary tale" format: html: toc: true toc-depth: 3 code-fold: show code-tools: true --- ## Session Overview This opening session answers the most fundamental question of the workshop: **why should HTA practitioners learn R when Excel and TreeAge already work?** We answer this through a real-world cautionary tale — the Reinhart-Rogoff incident — followed by a live demonstration of what R makes possible using an interactive Shiny app. **Session flow:** 1. Slides: The Reinhart-Rogoff story (the spreadsheet that shook nations) 2. Live demo: [Beyond Spreadsheets Shiny App](https://019cf9b0-c0f4-4fd1-e29d-92143d6955b5.share.connect.posit.cloud/) — interactive PSA, Markov model, CEAC 3. Slides: Summary — what R makes possible, workshop roadmap ## The Reinhart-Rogoff Incident: A Cautionary Tale ### The Context In 2008, the world was in the grip of the worst financial crisis since the Great Depression. Governments faced a critical question: should they increase public spending to stimulate the economy, or cut spending to reduce debt? The answer would affect the lives of hundreds of millions of people. ### The Paper In 2010, two Harvard economists — Carmen Reinhart and Kenneth Rogoff — published "Growth in a Time of Debt." Their finding was dramatic: countries whose government debt exceeded 90% of GDP experienced **negative average economic growth** of −0.1%. The paper was cited over 3,000 times and directly influenced policy at the European Commission, UK Treasury, IMF, and U.S. Congress. Paul Ryan cited it in his Budget Committee proposal to justify austerity measures. ### The Discovery In 2013, Thomas Herndon — a 28-year-old PhD student at the University of Massachusetts Amherst — was assigned to replicate a famous economics paper. He chose Reinhart-Rogoff. He could not reproduce the results. He contacted the authors, who shared their Excel spreadsheet. What he found changed the global conversation. ### The Three Errors **Error 1: Excel Row Exclusion.** The `AVERAGE` formula in the spreadsheet covered rows 30–44 instead of 30–49. Five countries — Australia, Austria, Belgium, Canada, and Denmark — were simply left out of the calculation. This was a drag-select error, the kind that is invisible in Excel unless someone inspects every formula. **Error 2: Unconventional Weighting.** New Zealand had a single year of high-debt, low-growth data (−7.6%). This one observation was given equal weight to the UK's 19 years of data. A single outlier dragged the entire "above 90%" average into negative territory. **Error 3: Selective Data Exclusion.** Post-war data from three countries was excluded without transparent justification. ### The Corrected Result When Herndon, Ash, and Pollin corrected all three errors, the average GDP growth for high-debt countries was **+2.2%** — positive, not negative. The entire foundation for austerity policy was based on a spreadsheet error that no peer reviewer could detect because **nobody could audit the spreadsheet**. ### Five Lessons for HTA 1. **Errors hide in plain sight** — wrong cell ranges are invisible in Excel 2. **Peer review failed** — reviewers had no access to the underlying calculations 3. **Replication was impossible** — until the original spreadsheet was shared years later 4. **Policy moved faster than science** — the paper influenced billions before anyone checked the maths 5. **Transparency is not optional** — it is the only protection against consequential errors ## Live Demo: The Shiny App During the session, we open the **Beyond Spreadsheets** Shiny app for a live, interactive demonstration: 🔗 **[Open the Shiny App](https://019cf9b0-c0f4-4fd1-e29d-92143d6955b5.share.connect.posit.cloud/)** The app has five tabs: ### Tab 1: The Spreadsheet That Shook Nations An interactive exploration of the Reinhart-Rogoff errors. Toggle which errors are present and see how the results change in real time — something impossible with the original Excel file. ### Tab 2: PSA Demo A full probabilistic sensitivity analysis in R, driven by three lines of code: ```r set.seed(42) delta_cost <- rnorm(10000, mean_cost_b, sd_cost_b) - rnorm(10000, mean_cost_a, sd_cost_a) delta_qaly <- rnorm(10000, mean_qaly_b, sd_qaly_b) - rnorm(10000, mean_qaly_a, sd_qaly_a) ``` The app shows the cost-effectiveness plane (with all four quadrants), a simulation table where every row is auditable, and a CEAC preview — all generated in under a second. **Try this in Excel:** setting up 10,000 PSA iterations requires complex macros, runs slowly, and produces output that is nearly impossible to audit. ### Tab 3: Markov Model A three-state cohort Markov model (Healthy → Sick → Dead) with: - Interactive transition probability sliders (auto-adjusted so rows sum to 1) - Live state-transition diagram that updates with your parameters - Markov trace comparing Control and Treatment arms - Cost-effectiveness results with ICER interpretation The core logic is a single line: `trace[i,] <- trace[i-1,] %*% P` ### Tab 4: CEAC Explorer A standalone cost-effectiveness acceptability curve showing how the probability of cost-effectiveness changes across willingness-to-pay thresholds. The Indian WTP benchmark of ₹2.5 lakh/QALY (1× GDP per capita) is highlighted. ### Tab 5: R vs Excel A side-by-side comparison of how the same HTA tasks look in Excel versus R — making the case for transparency, reproducibility, and scalability. ## What R Makes Possible R does not replace your HTA knowledge. It gives you a **different medium** to express that knowledge — one that is: **Reproducible.** Your entire analysis is a script. Anyone can re-run it and get the same result. The `set.seed()` function ensures that even probabilistic analyses produce identical output across runs, machines, and collaborators. **Scalable.** Running 10,000 PSA iterations takes a fraction of a second. Running 100,000 is equally easy. Excel slows to a crawl; R does not. **Transparent.** A reviewer can read your code and see exactly what you did. No hidden cell references, no buried macros. Every assumption is explicit. **Version-controlled.** With Git, every change is tracked, documented, and reversible. No more `Model_v3_final_FINAL_revised.xlsx`. **Free and extensible.** No license fees, no vendor lock-in. Thousands of R packages exist for health economics (`heemod`, `hesim`, `dampack`, `BCEA`), survival analysis (`flexsurv`, `survminer`), Bayesian methods, and visualisation. ## Workshop Roadmap In this workshop, you will see R applied to real HTA problems across four model types: ```{r} #| label: fig-four-models #| echo: false #| eval: true #| fig-cap: "Four HTA model types covered in this workshop" library(DiagrammeR) grViz(" digraph four_models { graph [rankdir=TB, bgcolor='transparent', fontname='Helvetica'] node [shape=box, style='filled,rounded', fontname='Helvetica', fontsize=11, width=2.5] center [label='HTA Decision\nModels in R', shape=ellipse, fillcolor='#4e79a7', fontcolor='white', fontsize=13] dt_diag [label='Diagnostic\nDecision Tree\n(Session 3: GDM)', fillcolor='#59a14f', fontcolor='white'] dt_ther [label='Therapeutic\nDecision Tree\n(Session 4: DES vs BMS)', fillcolor='#f28e2b', fontcolor='white'] markov [label='Markov Cohort\nModel\n(Session 5: CKD)', fillcolor='#e15759', fontcolor='white'] psm [label='Partitioned Survival\nModel\n(Session 8: Breast Cancer)', fillcolor='#76b7b2', fontcolor='white'] center -> dt_diag center -> dt_ther center -> markov center -> psm } ") ``` **Day 1:** Why R for HTA → R orientation → Decision trees (GDM diagnostic, DES vs BMS therapeutic) **Day 2:** Markov cohort model (CKD) → Probabilistic sensitivity analysis → GenAI for HTA coding **Day 3:** Partitioned survival model (Breast cancer) → PSA for PSM → Interactive Shiny apps → Wrap-up ## What This Workshop Will and Won't Do This workshop **will not** turn you into R programmers in three days, ask you to memorise syntax, or expect you to write complex code from scratch. This workshop **will** give you working templates you can adapt for your own studies, build your confidence in reading and modifying R code, show you how to use AI tools to accelerate your workflow, and help you understand when R is the right tool for your HTA work. ## Key Takeaway R is not replacing your HTA expertise — it is giving that expertise a more powerful medium. The clinical judgement, the model structuring, the parameter selection — that remains yours. R simply makes the execution more reproducible, transparent, and scalable. ## Further Reading - Herndon, T., Ash, M., & Pollin, R. (2014). Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. *Cambridge Journal of Economics*, 38(2), 257–279. - Reinhart, C. M., & Rogoff, K. S. (2010). Growth in a time of debt. *American Economic Review*, 100(2), 573–578. - Baio, G. (2012). *Bayesian Methods in Health Economics*. CRC Press. - Briggs, A., Sculpher, M., & Claxton, K. (2006). *Decision Modelling for Health Economic Evaluation*. Oxford University Press. - R-HTA Consortium: [r-hta.org](https://r-hta.org)