Young children (6–7 years) can meaningfully participate in cognitive interviews assessing comprehensibility in health-related quality of life domains: a qualitative study

Systematic measurement of children’s health-related quality of life (HRQoL) and related constructs via patient reported outcome measures (PROMs) is essential for incorporating children’s own views and experiences into healthcare, research and policy [1‐4]. Children should be enabled to self-report via PROMs wherever possible [2, 5] and the development of children’s PROMs has received increasing attention over time [4, 6, 7]. During development and evaluation processes, content validity of PROMs needs to be demonstrated [5, 8]. Instruments should accurately capture constructs of health that are meaningful to the intended target population [5, 8‐10] and should be relevant, comprehensive, and comprehensible [11].

The need to establish content validity, particularly comprehensibility, is magnified when developing self-report PROMs for young children (i.e., ≤ 7 years) [12]; still-developing cognitive and linguistic skills can affect how young children understand and complete PROMs [12, 13]. Crucially, it cannot be assumed that young children will interpret PROM content consistently with adult developers [14, 15]. To account for childhood development, guidance recommends establishing instrument content validity in narrow age bands for all ages the PROM is intended for (i.e., it cannot be assumed what is comprehensible for an 11-year-old will also be comprehensible for a 6-year-old) [2, 5, 13].

Cognitive interviews (or cognitive ‘debriefing’ interviews) are recommended for establishing comprehensibility (as well as other aspects of content validity) during instrument development/evaluation for both adults [5, 9, 16, 17] and children [2]. Typically, target population members complete the PROM while simultaneously saying all their thoughts out loud (“think aloud”) and/or being asked direct questions (“verbal probes”) to establish their thinking during PROM completion [16‐18]. The International Society for Pharmacoeconomics Research (ISPOR) Good Practices Task Force recommends using the information to evaluate and address problems with instrument comprehensiveness and comprehensibility (e.g., of items, response options, recall period, instructions, etc.) [17]. ISPOR provides a set of guidelines for cognitive interview design and implementation with adults [17] but not with children, despite potential methodological challenges associated with including children in qualitative PROM development often being recognised [2, 13].

The abstract nature of cognitive interviews raises concerns that they may be too challenging for young children [13, 19]; they are considered challenging even for adults [20, 21]. Young children can struggle with abstract, hypothetical questions and may struggle to verbally explain their thinking or elaborate on how they responded to PROM items [13]. Further, they may not fully understand their role as ‘evaluators’, instead focussing on completing the PROM [2], and may not have long-enough attention spans to evaluate an entire instrument [3, 13, 22, 23].

Identifying ways to address these challenges can be difficult; unlike the ISPOR guidance for adults [17] there are no standard guidelines for cognitive interviewing with young children, and general recommendations for qualitative PROM development with children primarily focus on older children and adolescents [2, 13, 19]. Additionally, a recent review found very few published examples of children’s (4–7 years) involvement in cognitive interviews as part of PROM development/evaluation, most of which contained little reporting detail explaining how young children were enabled to participate [24]. It is unclear whether young children’s limited involvement is because they are incapable of participating, or because methods of interviewing have not been suitably adapted to enable their participation [24].

In summary, the feasibility of cognitive interviewing with young children to support evaluation of PROM comprehensibility is unknown [24], potentially impacting how content validity can be established for PROMs developed for this age group. This study aimed to establish the feasibility of cognitive interviewing with children aged 6–7 years for comprehensibility evaluations. Reporting follows the Standards for Reporting Qualitative Research (SRQR) [25] (Online Resource (OR) 1: Supplement 1).

This study demonstrates new evidence for the feasibility of cognitive interviewing to evaluate PROM item comprehensibility with children aged 6–7 years. Evidence for alignment between intended and interpreted item meaning was clearly operationalised and identified in children’s responses to verbal probes; data was sufficient for evaluating item comprehensibility and for identifying items requiring further testing and/or refinement. Furthermore, children’s responses were often highly detailed, illustrating their understanding of items through both de-contextual and contextual explanations, and where interpretations aligned with intended meaning, children did not require extensive probing.

This study addresses gaps in previous literature and directly challenges the view that children ≤ 7 years cannot participate in cognitive interviews or will only provide limited data. Guidance for involving young children is scarce and there exists only a small number of published examples of children ≤ 7 years having been involved in cognitive interviews as part of PROM development [24]. Suggestions of the youngest age from which children can participate in cognitive interviews vary from potentially 5 years [2], to definitive statements that children younger than 8 cannot self-report via questionnaires so would not be involved in cognitive interviews [34]. It has also been proposed that children aged 6–8 years likely cannot elaborate on their thoughts and will only respond with ‘yes/no’ answers in cognitive interviews [13]. This study provides novel feasibility evidence that children aged 6–7 years old can provide comprehensibility data in PROM development and evaluation.

By implementing the Comprehensibility Continuum (CC), this study is the first of its kind to clearly report how evidence for alignment between intended and interpreted item meaning was defined. Previous studies with young children have not elaborated on how evidence for item comprehensibility was identified, as is typical within current approaches to cognitive interview analysis [32]. For example, Tomlinson et al. rated 4–7-year-olds’ interpretations of items from “completely incorrect” to “completely correct” but did not describe how judgements were made regarding the ‘correctness’ of the child’s interpretation [35]. Using rigorous analysis methods in this study further demonstrates the feasibility of cognitive interviewing with 6–7-year-olds.

The findings of this study have significance when considering the development/evaluation of children’s PROMs globally. To be considered meaningful measures in clinical trials, regulatory bodies, such as the Food and Drug Administration (FDA), require evidence of PROM content validity via qualitative development processes, including cognitive interviews, in narrow age bands [5, 8, 17]. In demonstrating the feasibility of gathering meaningful comprehensibility data, this study also supports the development of self-report instruments as are preferred over proxy-reports by the FDA when measuring non-observable states (e.g., pain intensity) [2, 5]. Findings also have implications for PROM implementation in clinical practice and healthcare systems where concerns regarding comprehensibility have been identified as a barrier to routine implementation of children’s instruments [36].

This study clearly describes how cognitive interview techniques were implemented to enable useful data to be gathered from young children. Although necessary for the development of best practices [27, 37], comprehensive reporting of cognitive interview methods used with young children is limited in previous studies [24]. There is concern that young children’s attention spans are too short for cognitive interviewing [13]; here it is demonstrated that 6–7-year-olds could discuss six items in depth for up to 20 minutes, which could help the design of future studies. Although using multiple probes may increase confidence in understanding children’s interpretations of items, the current study found extensive probing was not necessary for children to explain interpretations, contrary to some existing suggestions [13]. The adapted techniques (i.e., ‘Teach Teddy’) may be useful for future studies; data elicited from these techniques was comparable to data elicited from the standard ISPOR probe. In particular, using physical props may be advantageous in situations where young children need support maintaining attention on the interview task [13, 28, 30].

This study is not without limitations. Only item comprehensibility was investigated but multiple PROM components (e.g., response items, recall period, instructions) and aspects of content validity (e.g., comprehensiveness, relevance) should also be evaluated in cognitive interviews during PROM development/evaluation [16, 17]. Verbal probing techniques used in the current study could potentially be used to evaluate comprehensibility of other PROM components, and the principle of adapting interview tasks to be concrete could be applied to evaluating comprehensiveness and relevance. Further testing is required to establish the feasibility of cognitive interviewing with young children to evaluate all PROM components and aspects of content validity, and to confirm the applicability of findings to the development/evaluation of ‘real’ PROMs.

Only children aged 6–7 years participated in the study, meaning the feasibility of including children from 4- and 5-years remains unclear. However, chronological age does not equal ability [12, 13], and results were comparable across 6- and 7-year-olds. It is therefore possible that children aged 4–5 years can also participate in cognitive interviews, warranting further research. This study was conducted with a sample of children from the UK where formal schooling begins earlier (usually at 4-years-old) than is typical in most countries [38]. The transferability of findings to other countries, and to other languages, thus requires further testing. It is possible that type of health condition may have influenced a child’s engagement with the interview. As an initial study exploring the feasibility of cognitive interviewing with a small sample size, it was not considered appropriate or practical to formally explore potential effect of health condition on interview engagement; this would require further research. However, the quality and usefulness of children’s responses to verbal probes was consistent across the sample, with all except one item discussion providing the information necessary for comprehensibility evaluation. These initial findings are encouraging in suggesting young children with a range of health conditions can, in principle, participate in cognitive interviews.

Cognitive interviewing to evaluate PROM item comprehensibility was feasible with children aged 6–7 years. Clear evidence for feasibility was demonstrated via the novel Comprehensibility Continuum. Although including younger age groups in cognitive interviews may require different methodological considerations, interview methods used in this study were not vastly different from those typically used with older children and adults. PROM developers therefore have a responsibility to include young children in cognitive interviews wherever possible, ultimately to ensure the systematic and accurate measurement of children’s health outcomes.