Background
Precision is a recognised requirement of patient-reported outcome measures but no previous studies of the precision of methods for obtaining health state values from the general public, based on specific health state descriptions or vignettes, have been carried out. The methodological requirements of policy makers internationally is driving growth in the use of methods to obtain utilities from the general public to inform cost per quality-adjusted life-year (QALY) analyses of health technologies being considered for adoption by health systems.
Methods
The precision of five comparisons of the outcomes of treatments, based on health state descriptions, was assessed against the results of clinical trials which showed a statistically and clinically significant improvement using an internet panel of members of the UK general public. Health states were developed to depict the baseline and post-treatment states from these exemplar clinical trials. Preferences for health states were obtained using bottom-up titrated standard gamble over the internet, and differences between summary health state values corresponding to the treatment and comparator groups within each exemplar study were compared. Results are considered in the context of various estimates for the minimally important difference in utility values.
Results
Participation among members of the internet panel in the five exemplars ranged from 27 to 59. In four of the five exemplars, the utility-based estimates of treatment benefit showed significant differences between groups and were greater than an assumed minimally important difference of 0.1. Mean utility differences between groups were: 0.23 (computerised cognitive behavioural therapy for depression, P < 0.001), 0.11 (hip resurfacing for hip osteoarthritis, P < 0.001), 0.0005 (cognitive behavioural therapy for insomnia, P = 0.98), 0.15 (pulmonary rehabilitation for COPD, P < 0.001) and 0.11 (infliximab for Crohn’s disease, P < 0.001). The confidence intervals around the estimates of utility-based treatment effect in three of the five examples did not exclude the possibility of a difference smaller than a minimally important difference of 0.1. Recent empirical evidence suggests a lower minimally important difference (0.03) may be more appropriate, in which case our results provide further reassurance of preservation of precision in health state description and valuation.
Conclusions
The precision of estimates of treatment effects based on preference data obtained from disease-specific measurements in clinically significant studies of health technologies was acceptable using an internet-based panel of members of the general public and the standard gamble. Definition of the minimally important difference in utility estimates is required to adequately assess precision and should be the subject of further research.
