Table 6 Quality assessment results of economic modelling studies included in the systematic review
Quality Criterion | Philips et al’ economic modelling checklist questionsa | Response (√, x, NC or NA) | ||
|---|---|---|---|---|
Studies (author and year) | ||||
Hadzijahic et al. (2000) [29] | Harewood et al. (2002) [30] | Wallace et al. (2002) [31] | ||
S1 | Is there a clear statement of the decision problem? | √ | √ | √ |
Is the objective of the evaluation and model specified and consistent with the stated decision problem? | √ | √ | √ | |
Is the primary decision-maker specified? | NC | √ | √ | |
S2 | Is the perspective of the model stated clearly? | x | √ | √ |
Are the model inputs consistent with the stated perspective? | NC | √ | √ | |
Has the scope of the model been stated and justified? | √ | √ | √ | |
Are the outcomes of the model consistent with the perspective, scope and overall objective of the model? | √ | √ | √ | |
S3 | Is the structure of the model consistent with a coherent theory of the health condition under evaluation? | √ | √ | √ |
Are the sources of data used to develop the structure of the model specified? | √ | √ | √ | |
Are the causal relationships described by the model structure justified appropriately? | NA | NA | NA | |
S4 | Are the structural assumptions transparent and justified? | √ | √ | √ |
Are the structural assumptions reasonable given the overall objective, perspective and scope of the model? | √ | √ | √ | |
S5 | Is there a clear definition of the options under evaluation? | √ | √ | √ |
Have all feasible and practical options been evaluated? | √ | √ | √ | |
Is there justification for the exclusion of feasible options? | NA | NA | NA | |
S6 | Is the chosen model type appropriate given the decision problem and specified causal relationships within the model? | √ | √ | √ |
S7 | Is the time horizon of the model sufficient to reflect all important differences between options? | x | x | X |
Are the time horizon of the model, the duration of treatment and the duration of treatment effect described and justified? | x | x | X | |
S8 | Do the disease states (state transition model) or the pathways (decision tree model) reflect the underlying biological process of the disease in question and the impact of interventions? | √ | √ | √ |
S9 | Is the cycle length defined and justified in terms of the natural history of disease? | NA | NA | NA |
D1 | Are the data identification methods transparent and appropriate given the objectives of the model? | √ | NC | √ |
Where choices have been made between data sources, are these justified appropriately? | NA | √ | √ | |
Has particular attention been paid to identifying data for the important parameters in the model? | √ | √ | X | |
Has the quality of the data been assessed appropriately? | x | x | x | |
Where expert opinion has been used, are the methods described and justified? | NA | NA | x | |
D2 | Is the data modelling methodology based on justifiable statistical and epidemiological techniques? | √ | NC | √ |
D2a | Is the choice of baseline data described and justified? | √ | √ | √ |
Are transition probabilities calculated appropriately? | NA | NA | NA | |
Has a half-cycle correction been applied to both cost and outcome? | NA | NA | NA | |
If not, has this omission been justified? | NA | NA | NA | |
D2b | If relative treatment effects have been derived from trial data, have they been synthesised using appropriate techniques? | NA | NA | NA |
Have the methods and assumptions used to extrapolate short-term results to final outcomes been documented and justified? | NA | NA | NA | |
Have alternative assumptions been explored through sensitivity analysis? | √ | √ | √ | |
Have assumptions regarding the continuing effect of treatment once treatment is complete been documented and justified? | NA | NA | NA | |
Have alternative assumptions regarding the continuing effect of treatment been explored through sensitivity analysis? | NA | NA | NA | |
D2c | Are the costs incorporated into the model justified? | √ | √ | √ |
Has the source for all costs been described? | √ | √ | √ | |
Have discount rates been described and justified given the target decision-maker? | NC | NA | √ | |
D2d | Are the utilities incorporated into the model appropriate? | NA | NA | √ |
Is the source for the utility weights referenced? | NA | NA | X | |
Are the methods of derivation for the utility weights justified? | NA | NA | X | |
D3 | Have all data incorporated into the model been described and referenced in sufficient detail? | NC | √ | √ |
Has the use of mutually inconsistent data been justified (i.e. are assumptions and choices appropriate)? | NC | NC | √ | |
Is the process of data incorporation transparent? | √ | x | X | |
If data have been incorporated as distributions, has the choice of distribution for each parameter been described and justified? | NA | NA | NA | |
If data have been incorporated as distributions, is it clear that second order uncertainty is reflected? | NA | NA | NA | |
D4 | Have the four principal types of uncertainty been addressed? | x | x | X |
If not, has the omission of particular forms of uncertainty been justified? | x | x | X | |
D4a | Have methodological uncertainties been addressed by running alternative versions of the model with different methodological assumptions? | x | x | X |
D4b | Is there evidence that structural uncertainties have been addressed via sensitivity analysis? | x | x | X |
D4c | Has heterogeneity been dealt with by running the model separately for different subgroups? | x | x | x |
D4d | Are the methods of assessment of parameter uncertainty appropriate? | √ | √ | √ |
If data are incorporated as point estimates, are the ranges used for sensitivity analysis stated clearly and justified? | NC | √ | √ | |
C1 | Is there evidence that the mathematical logic of the model has been tested thoroughly before use? | x | x | x |
C2 | Are any counterintuitive results from the model explained and justified? | NA | NA | NA |
If the model has been calibrated against independent data, have any differences been explained and justified? | NA | NA | NA | |
Have the results of the model been compared with those of previous models and any differences in results explained? | x | x | x | |
Score, ratioâ„¢ (%) | 21/38 (55%) | 24/38 (63%) | 28/43 (65%) | |