The Challenge
Every health economic model that reaches a payer, a committee, or a publication has been reviewed by someone. Almost none have been independently re-executed. Manual model QC means a senior modeller working through someone else's workbook cell by cell, rebuilding the logic in their head, over days of concentrated effort.
The harder problem is reviewing a model you did not build. Before you can judge whether it is right, you have to work out what it does, and a large workbook does not explain itself. Following a single parameter from where it is defined to where it is actually used is where a reviewer loses the thread.
The Order Is the Point
Model QCR works in a fixed sequence, and that sequence is the reason its findings are checkable rather than opinions. It does not offer a view on your model until it has shown it can reproduce it.
- It reads the workbook. Every formula is parsed and traced back to the input cells it depends on.
- It builds a map of the model from those formulas: health states, treatment arms, transitions, costs, utilities and time structure. The map comes from what the formulas actually do, not from a diagram someone drew in the workbook.
- It proves the map against your workbook. The result cells you nominate, and every cell they depend on, are reproduced from the workbook's own formulas and compared with the values your workbook has stored.
- Only then does it review. Structural checks and behavioural tests run against a model it has already shown it can reproduce.
- It reports. A Word report carrying the findings, the evidence behind each one, and a description of the model itself.
A typical model takes about five minutes.
What Comes Back
The results screen opens on an overall status, two counters, and an executive summary. The counters report evaluated checks only. A check that could not be evaluated on your model is not counted as a pass, and it is not hidden either. Anything flagged for a second look is stated separately as Review.
Because Model QCR has built a map of the model, it can describe the model back to you: population, health states, treatment arms, efficacy logic, mortality, costs, utilities and time dependencies, each with the cells it came from. A schematic is drawn from the transition logic. If a state is declared but never reachable, it still appears, because the model declares it.
Behavioural testing runs a standard battery on every model, covering cohort conservation, monotonic death, valid probabilities, cost and QALY accumulation and discounting behaviour, then adds tests specific to your model. Every test states what it expected and what it observed, and expands to show which values were overridden.
The report is the deliverable, and it is written to be handed on. Findings at a Glance opens with anything flagged, followed by what passed. Appendices carry the scorecard, every structural check, behavioural coverage, every test, the cohort traces and the parameter registry.
You can also ask about the results. The conversation alongside them answers questions about the model, the checks and the findings, and cites the evidence behind each answer so you can open it.
See It Work
The recorded demonstration follows one model the whole way: a seventeen-sheet cohort state-transition model with five simulation-output sheets left out, assessed in four minutes thirty-five seconds. The run returns a REVIEW status, seventeen of twenty-one structural checks evaluated with four flagged for a second look, thirty of thirty behavioural tests passed, and a sixty-three page report.
The finding worth watching is structural rather than numerical. Two exacerbation states are declared in the model and never entered across all 101 cycles, in either arm. Nothing failed, and four things are still flagged.
The model in this demonstration is synthetic, built as an example for testing. Its intervention, its comparator and every parameter in it were constructed, so nothing on screen is a real product, a real price or a real result.
Download the user guide (PDF, 20 pages) for a step-by-step walkthrough of an assessment, from upload to report.
It Stops Rather Than Guessing
Model QCR does not issue a partial review. If it cannot establish that it has reproduced your model, it stops and names the reason on screen. Where several reasons apply, all of them are listed.
This is deliberate. A review of a model the tool could not reproduce would carry the authority of a measurement without being one, and that is worse than no review at all.
What Version 1 Does Not Do
- A first-pass validation, not a certification. Expert review of the findings is essential.
- PSA and DSA are not configured or run in version 1.
- VBA macros are not executed. A model that depends on macros for its results cannot be fully assessed.
- Your workbook must be saved with its calculated values in place, because every check compares Model QCR's own recomputation against the values your workbook has stored.
- Coverage depends on how the workbook is organised. A clearly structured, well-labelled model receives the most complete analysis.
Your workbook is never modified. Every check works on Model QCR's own reproduction of it.
What It Means for You
- The findings rest on a reproduction you can check. The tool demonstrates it can reproduce the model before it judges it, so a finding is traceable to the cells that produced it rather than resting on a reviewer's reading.
- About five minutes per model, against days of senior reviewer time for a manual pass.
- The model becomes accessible. The description, the schematic and the conversation exist because the tool built a map of the model, which is the hard part when the model is someone else's.
- Nothing is quietly excluded. Checks that could not be evaluated are reported as such rather than counted as passes, and sheets left out of the assessment are disclosed.
- Independence by construction. The review does not come from the team that built the model, which is what evidence assessment groups and internal governance need.
For HTA evidence assessment groups, pharma HEOR teams, and consultancies validating their own or a vendor's model: an independent first-pass validation that reproduces the model from its own formulas, reports what it could and could not evaluate, and hands you a report written to be passed on.