Objective
In the absence of population-based information, distribution-based meaningful change metrics have been previously found to perform similarly. Yet, it is unknown how a Bayesian approach derived from Posterior Predictive Distribution (PPD) of anticipated changes would compare against existing metrics.
Methods
PPD defines meaningful change as change scores that exceed the amount expected from the posterior predictive distribution given a previous score. The PPD adjusts for common statistical phenomena that arise in a pre-test–post-test setting, such as regression to the mean and post-test drift. The PPD was compared to Reliable Change Index (RCI) and Gulliksen–Lord–Novick (GLN) methods using published real-world data and simulated hypothetical data, respectively.
Results
Real-world data showed that the methods made similar classifications when the measurement reliability was above 0.80. When reliability was low at 0.50 and thus more susceptible to regression to the mean effects, PPD and GLN were able to correct for it but not the RCI. However, PPD was more conservative and sensitive to biased priors. The simulation study showed that the three methods performed similarly overall, but PPD was slightly better in detecting prevalent changes, e.g., at time 2 (against RCI at p < 0.0001; against GLN at p < 0.0001) and time 3 (p = 0.024, p = 0.002).
Conclusions
When measurement reliability is high, as is frequent in HRQOL development efforts, the three methods performed similarly. At a cost of more conservative cutoffs and complex calculations, the Bayesian PPD nevertheless confers practical advantages when reliability is low. It may be worthy of further research and applications.
