Skip to main content

Advertisement

Springer Nature Link
Log in

Good Practice Guidelines for the use of Statistical Regression Models in Economic Evaluations

  • Practical Application
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Decision-analytic models (DAMs) used to evaluate the cost effectiveness of interventions are pivotal sources of evidence used in economic evaluations. Parameter estimates used in the DAMs are often based on the results of a regression analysis, but there is little guidance relating to these. This study had two objectives. The first was to identify the frequency of use of regression models in economic evaluations, the parameters they inform, and the amount of information reported to describe and support the analyses. The second objective was to provide guidance to improve practice in this area, based on the review. The review concentrated on a random sample of economic evaluations submitted to the UK National Institute for Health and Clinical Excellence (NICE) as part of its technology appraisal process. Based on these findings, recommendations for good practice were drafted, together with a checklist for critiquing reporting standards in this area. Based on the results of this review, statistical regression models are in widespread use in DAMs used to support economic evaluations, yet reporting of basic information, such as the sample size used and measures of uncertainty, is limited. Recommendations were formed about how reporting standards could be improved to better meet the needs of decision makers. These recommendations are summarised in a checklist, which may be used by both those conducting regression analyses and those critiquing them, to identify what should be reported when using the results of a regression analysis within a DAM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. National Institute for Health and Clinical Excellence. Published technology appraisals. http://www.nice.org.uk/Guidance/TA/Published. 16 Oct 2012.

  2. Kearns B, Ara R, Waillo A. NICE Decision Support Unit: regression methods. http://www.nicedsu.org.uk/Regression-methods(2391678).htm. 1-8-2013.

  3. Cox DR, Snell EJ. Applied statistics: principles and examples. London: Chapman & Hall; 1981.

    Google Scholar 

  4. Chatfield C. The initial examination of data. J R Stat Soc A. 1985;148(3):214–53.

    Article  Google Scholar 

  5. National Institute for Health and Clinical Excellence. Guide to the methods of technology appraisal (updated June 2008). 2008.

  6. Freeman JV, Walters SJ, Campbell MJ. How to display data. Oxford: BMJ Books; 2008.

    Google Scholar 

  7. Few S. Show me the numbers. California: Analytics Press; 2004.

    Google Scholar 

  8. Altman DG. Statistics and ethics in medical research. VI: presentation of results. Br Med J. 1980;281(6254):1542–4.

    Article  CAS  PubMed  Google Scholar 

  9. Hand DJ. Statistics and data mining: intersecting disciplines. ACM SIGKDD Explor Newslett. 1999;1(1):16–9.

    Article  Google Scholar 

  10. Good P, Hardin J. Common mistakes in statistics (and how to avoid them). New Jersey: Wiley; 2003.

    Book  Google Scholar 

  11. Harrell FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.

    Article  PubMed  Google Scholar 

  12. Royston P, Suaerbrei W. Multivariable model-building: a pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester: Wiley; 2008.

    Google Scholar 

  13. Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. New York: Wiley; 2002.

    Google Scholar 

  14. Briggs A, Clark T, Wolstenholme J, Clarke P. Missing … presumed at random: cost-analysis of incomplete data. Health Econ. 2003;12(5):377–92.

    Article  PubMed  Google Scholar 

  15. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis a report of the ISPOR-SMDM modeling good research practices task force working group-6. Med Decision Mak. 2012;32(5):722–32.

    Article  Google Scholar 

  16. Campbell MJ. Statistics at square two: understanding modern statistical applications in medicine. Oxford: BMJ Books; 2006

    Book  Google Scholar 

  17. Cooper NJ, Sutton AJ, Mugford M, Abrams KR. Use of Bayesian Markov Chain Monte Carlo methods to model cost-of-illness data. Med Decision Mak. 2003;23(1):38–53.

    Article  Google Scholar 

  18. Tufte E. Improving data analysis in political science. World Politics. 1969;21(4):641–54.

    Article  Google Scholar 

  19. Machin D, Campbell MJ, Walters SJ. Medical statistics. 4th ed. Chichester: Wiley; 2007.

    Google Scholar 

  20. Ara R, Brazier J. Predicting the short form-6D preference-based index using the eight mean short form-36 health dimension scores: estimating preference-based health-related utilities when patient level data are not available. Value Health. 2009;12(2):346–53.

    Article  PubMed  Google Scholar 

  21. Sullivan PW, Ghushchyan V. Preference-based EQ-5D index scores for chronic conditions in the United States. Med Decis Making. 2006;26(4):410–20.

    Article  PubMed  Google Scholar 

  22. Sullivan PW, Slejko JF, Sculpher MJ, Ghushchyan V. Catalogue of EQ-5D scores for the United Kingdom. Med Decis Making. 2011;31(6):800–4.

    Article  PubMed  Google Scholar 

  23. Dobrez D, Cella D, Pickard AS, Lai JS, Nickolov A. Estimation of patient preference-based utility weights from the functional assessment of cancer therapy-general. Value Health. 2007;10(4):266–72.

    Article  PubMed  Google Scholar 

  24. Briggs A, Sculpher M, Claxton K. Decision modelling for health economic evaluation. Oxford: Oxford University Press; 2006.

    Google Scholar 

  25. Stevenson M, Tappenden P, Squires H. Methods for handling uncertainty within pharmaceutical funding decisions. Int J Syst Sci 2012. Available from: http://www.tandfonline.com/doi/abs/10.1080/00207721.2012.723056?journalcode=tsys20#.ubqbazqn0cB

  26. Strong M, Oakley JE, Chilcott J. Managing structural uncertainty in health economic decision models: a discrepancy approach. J R Stat Soc Ser C. 2012;61(1):25–45.

    Article  Google Scholar 

  27. Husereau D, Drummond D, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. Pharmacoeconomics. 2013;31(5):361–7.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This article is based on a report that was funded by NICE through its Decision Support Unit. The views, and any errors or omissions, expressed in this article are of the authors only.

BK drafted the manuscript and takes responsibility as the overall guarantor of the content. BK, RA and AW conceived and planned the work. All authors contributed to drafting the checklist and revising the manuscript for important intellectual content. All authors have given their approval for the final version to be published.

The authors acknowledge Paul Tappenden, who contributed to the work, but did not meet the criteria for authorship.

AM is a member of the technology appraisal committee at NICE. BK, RA, AW, MHA, KA and MC have no other potential conflicts of interest. AM’s contribution was made under the terms of a Career Development research training fellowship issued by the National Institute for Health Research (NIHR; grant CDF-2009-02-21). The views expressed in this publication are those of the authors and not necessarily those of the National Health Service, The NIHR or the Department of Health.

Author information

Authors and Affiliations

  1. School of Health and Related Research, University of Sheffield, Sheffield, S1 4DA, UK

    Ben Kearns, Roberta Ara, Allan Wailoo, Monica Hernández Alava & Mike Campbell

  2. Centre for Health Economics, University of York, York, YO10 5DD, UK

    Andrea Manca

  3. School of Medicine, University of Leicester, Leicester, LE1 7RH, UK

    Keith Abrams

Authors
  1. Ben Kearns
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roberta Ara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Allan Wailoo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea Manca
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Monica Hernández Alava
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Keith Abrams
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mike Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Kearns.

Appendix (Checklist)

Appendix (Checklist)

1.1 Proposed Checklist for Statistical Regression Analyses

1.1.1 Pre-modelling Considerations

  1. 1.

    Have the objectives of the analysis been stated?

  2. 2.

    Has the need for a de novo regression analysis been justified?

  3. 3.

    Has the source of the data used been stated? This would include synopses of key study features such as socio-demographic/clinical characteristics and the data collection method.

  4. 4.

    Has the total sample size available been reported?

  5. 5.

    Are sufficient explanations of all variables used provided?

  6. 6.

    Are sufficient numerical and/or graphical summaries provided?

  7. 7.

    Has the quality of data (missing values, outliers, possible bias, etc.) been described?

  8. 8.

    Has the type/method of regression model(s) considered been stated/justified?

  9. 9.

    Have any modelling assumptions been stated?

  10. 10.

    Is a convincing rationale given for the inclusion of explanatory variables?

1.1.2 Arriving at the Final Model

  1. 11.

    Are sufficient details about the computational methods used provided?

  2. 12.

    If more than one model was considered, has justification been given for why the preferred model has been selected?

  3. 13.

    Has the choice of covariates been justified?

  4. 14.

    Is the sample size reported for every model presented?

  5. 15.

    Has the handling of missing values (if any) been described?

1.1.3 Presentation of the Final Model

  1. 16.

    Are the coefficient estimates provided?

  2. 17.

    Are appropriate measures of uncertainty and significance provided?

1.1.4 Validating the Final Model

  1. 18.

    Are summary measures of goodness of fit presented?

  2. 19.

    Are details of the results of a residual analysis provided?

  3. 20.

    Has the model been validated on external (or quasi-external) data?

  4. 21.

    Is the plausibility of the modelled predictions and/or coefficients discussed?

  5. 22.

    Are the results compared to the literature and/or other data?

1.1.5 Acknowledging and Propagating Uncertainty in the Analysis

  1. 23.

    Has the method for handling parameter uncertainty been reported?

  2. 24.

    Is sufficient detail given for how parameter uncertainty was handled (e.g. if a variance–covariance matrix is used, is this available in some form?)

  3. 25.

    Is parameter uncertainty appropriately reflected in the DAM?

  4. 26.

    Has any structural (model) uncertainty been explored (in the DAM)?

  5. 27.

    Have the model’s limitations been discussed (and explored if possible)?

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kearns, B., Ara, R., Wailoo, A. et al. Good Practice Guidelines for the use of Statistical Regression Models in Economic Evaluations. PharmacoEconomics 31, 643–652 (2013). https://doi.org/10.1007/s40273-013-0069-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-013-0069-y

Keywords