Spirituality1 and religion (S/R) are important aspects of life for many people around the world. Globally, approximately 84% of the population identifies as religious (Pew Research Center, 2017). Notwithstanding decreasing affiliations with many organized religions in the U.S., 73% of Americans still endorse religion as important in their lives, 87% of the population believe in God or a Higher Power, and 58% engage in prayer frequently (Gallup, 2020; Pew Research Center, 2015). A robust empirical literature has revealed diverse ways in which S/R may strengthen well-being, reduce risk for mental and physical health conditions, and support healing and recovery when such issues occur (for a review, see Koenig et al., 2023). Specifically, S/R can support formation of beliefs, practices, and relationships that promote adaptive coping and meaning making (Pargament et al., 2013; Park, 2013), belonging and social connections (VanderWeele, 2017a), attachment security with God or a Higher Power (Davis et al., 2021), and healthy brain development among high-risk groups (e.g., thickening of cortical regions; Miller et al., 2014). However, a growing number of scientific studies also indicate persons with depressive disorders and other health-related conditions may struggle with their S/R in ways that hinder recovery and possible responsiveness to mental health treatment (Bockrath et al., 2022; Currier et al., 2024; Pargament & Exline, 2022). As such, the purpose of this study was to develop and evaluate a five-item version of Exline et al.’s (2014) Religious and Spiritual Struggles Scale (RSS) that could support clinical research and practice for spiritual struggles.
Defining Spiritual Struggles
Pargament and Exline (2022) define spiritual struggles as “experiences of tension, conflict, or strain that center on whatever people view as sacred.” Whether in the form of painful emotions (e.g., guilt, anger) or enduring internal conflict, Exline (2013) stated spiritual struggle “implies that something in a person’s current belief, practice, or experience is causing or perpetuating distress” (p. 459). These struggles usually fall into three categories: supernatural, interpersonal, or intra-personal. Particularly for persons whose meaning frameworks have been strongly shaped by a theistic religious tradition, spiritual struggles might entail feeling angry, neglected, or abandoned by God. Others may struggle in relationships with fellow religious adherents and/or leaders within their own tradition or another tradition. Lastly, intra-personal struggles may emerge from doubting core doctrines or teachings from one’s S/R, not living congruently with perceived moral standards, or a painful void of ultimate meaning about the deeper purpose of life in general and/or the significance of one’s life in particular. Spiritual struggles have been documented among persons with varying intersectional identities and realities with respect to age (Krause et al., 2017), gender (Exline et al., 2021; Martoyo et al., 2019), race and ethnicity (Krause et al., 2018), and S/R background (including secular or atheist; Abu-Raiya et al., 2015, 2016; Mercadante, 2020).
Research has also found that spiritual struggles are particularly relevant for mental health care (Pargament & Exline, 2022). Just as depression and other health-related conditions might impair functioning in relational and vocational domains, 50–90% of persons seeking mental health care are somehow struggling with their S/R (Damen et al., 2021; Currier et al., 2019a; Leavitt-Alcántara et al., 2023; Rosmarin et al., 2014). Meta-analytic evidence from 32 longitudinal studies revealed spiritual struggles were prospectively linked with severity of mental health challenges over time (Bockrath et al., 2022). Focusing on 1,227 adults engaging in spiritually integrated psychotherapies, Currier et al. (2024) similarly found those who were struggling with their S/R were generally more psychologically distressed throughout treatment, experienced reduction of depression and other symptoms as concomitant spiritual distress was reduced, and needed longer to achieve clinical improvement. Of the studies that directly tested temporal associations between spiritual struggles and mental health symptoms in clinical samples, findings supported a Primary Struggles Model in which the severity of spiritual struggles was predictive of worse outcomes over time, or a Complex Struggles Model in which reciprocal associations between spiritual struggles and mental health outcomes occur over time (Cowden et al., 2021; Currier et al., 2015; Pargament & Lomax, 2013). Overall, these results indicate contending with spiritual struggles can add a distinct burden for persons seeking mental health care in ways that warrant routine assessment and possible attention in treatment.
Measuring Spiritual Struggles
Exline et al.’s (2014) Religious and Spiritual Struggles Scale (RSS) has emerged as a well-established and particularly useful, reliable, and valid tool for assessing spiritual struggles. When considering the spectrum of S/R diversity in the U.S., the RSS has been used and adapted with a range of groups including Christians (Exline et al., 2014), Muslims (Abu-Raiya et al., 2015), Jews (Abu-Raiya et al., 2016), as well as atheists and secular persons (Sedlar et al., 2018). The measure has also consistently demonstrated concurrent and predictive validity through its associations with symptoms of depression, anxiety, and other mental health conditions (e.g., Bockrath et al., 2022; Cowden et al., 2021; Currier et al., 2019a). Building on previous measures for assessing generalized distress related to S/R (e.g., Exline et al.’s (2000) Religious Comfort and Strain Scale, Pargament et al.’s (2011) Brief RCOPE), the original 26-item RSS captures six of the common ways in which people struggle with S/R (i.e., Divine, Demonic, Interpersonal, Moral, Ultimate Meaning, Doubting). A 14-item version has also been evaluated by Exline et al. (2022). Factor analytic work on the longer version supported a general spiritual struggle factor as well as a micro-struggle approach in which the six subscales can be used separately to assess distinct forms of struggle (Exline et al., 2014; Stauner et al., 2016). Depending on one’s aims and objectives, researchers and clinicians might utilize a total score on the RSS to gauge the overall severity of spiritual struggles or focus more squarely on specific forms of struggle in Exline et al.’s (2014) framework with greatest relevance for their work.
Notwithstanding the psychometric strengths of existing versions of the RSS, a short form of this instrument is needed to advance clinical research and practice for spiritual struggles. From a research standpoint, epidemiologic studies and designs that repeatedly solicit responses over time-limited periods (e.g., ecological momentary assessment) necessitate concise measurement. Attrition is also less likely to occur in longitudinal studies that focus on a range of variables when follow-up assessments are not time-consuming and burdensome. Given the range of information that clinicians need to gather in an initial assessment, a briefer version of the RSS could also be included more easily in an intake questionnaire or other routine procedures in their settings. A short form of the RSS that captures the distinct forms of struggle in Exline et al.’s (2014) framework might also be used in routine outcome monitoring procedures to holistically track patients’ progress in psychotherapy or other clinical interventions. Further, Stauner et al. (2016) noted items of the 26-item version of the RSS could be modeled with a bifactor structure (Reise, 2012), such that the varying types of struggle shared a high degree of common variance with a general factor (e.g., the shared variance of the six subscales on the RSS). Thus, according to scale development researchers (e.g., Stucky & Edelen, 2014), the possibility emerges for creating a briefer version of the RSS assessing overall severity of spiritual struggles that also represents the culturally and clinically relevant types of struggle that might contibute to the etiology of and recovery from clinical problems that lead people to seek mental health care (Pargament & Exline, 2022).
Overview of RSS-5 Development and Evaluation
The RSS-5 was developed via three sequential investigations to address these concerns. First, a set of five items capturing divine, interpersonal, moral, doubt, and ultimate meaning struggles was identified from existing datasets in which the original version of Exline et al.’s (2014) was utilized. Focusing on 711 participants in the first study who exceeded the clinical cutoff for depression symptoms on the Patient Health Questionnaire (PHQ-8; Kroenke et al., 2009), bifactor modeling was utilized to identify the most representative and superior-performing items for each form of struggle along with analyses to test internal consistency and convergent validity with the longer version of the RSS and severity of depression symptoms. Drawing upon on the selected items, we then implemented this five-item version of the RSS in an online study with undergraduate students (Study 2) and routine intake procedures in an integrated behavioral health outpatient clinic (Study 3). The purpose of these latter studies was to further examine factorial validity and convergent/divergent validity of the RSS-5 with fuller set of measures assessing psychological and spiritual factors that could be relevant for persons struggling with their S/R. Given the relevance for mental health care (Bockrath et al., 2022; Pargament & Exline, 2022), the end-goal of these three investigations was to produce a valid, structurally sound, and brief form of the RSS that could advance clinical research and practice on spiritual struggles.
Several practical and methodological considerations for this process should be highlighted. We omitted the demonic struggles scale on the original version of Exline et al.’s (2014) measure. Given the avoidance of S/R in many mental health and healthcare settings (Saunders et al., 2010), we were concerned that potentially skeptical clinicians and researchers would be hesitant to implement items inquiring about supernatural evil or demonic activity. Demonic struggles also occur less often than other the forms of struggle, particularly among secular and non-religious persons, and do not predict mental health outcomes to the same degree (Pargament & Exline, 2022). In combination with preferences for measures with 2–5 items in many clinical or applied contexts, we prioritized Exline et al.’s other forms of struggle. In so doing, we attempted to identify a general factor across the RSS subscales by sampling items from the five domains. Historically, researchers had encountered difficulty in developing a brief measure of spiritual distress with adequate utility and psychometric strengths to justify use in clinical settings (e.g., King et al., 2017). Hence, rather than clinicians developing new items for their settings or choosing existing items from the RSS or another scale, a psychometrically validated brief version of the RSS would be ideal. Last, we targeted depression symptoms in Study 1 due to high prevalence in clinical samples and co-occurrence with other psychological and physical health issues that often lead people to seek care (NIMH, 2021). In so doing, we did not intend to create a scale that would only apply to depressed persons, but used a clinical cutoff on the PHQ-8 (Kroenke et al., 2009) to identify as large cross-section of participants from our prior studies with RSS who had a probable need for treatment to identify the final items.
Study 1
Methods
Participants and procedures
The first sample consisted of participants from seven previous studies using Exline et al.’s (2014) measure. Specifically, these studies included adults hospitalized in an acute psychiatric stabilization program in a large behavioral health center in the U.S. Midwest (Currier, Foster et al., 2019), veterans seeking outpatient mental health care at a Veterans Affairs medical center on the U.S. Gulf Coast (Raines et al., 2017), veterans in a long-term transitional living program (Currier, Fadoir et al., 2019), men in a residential treatment program for substance use disorders (Currier, Fadoir et al., 2019), student service members and veterans from two universities located on the U.S. Gulf Coast (Currier, McDermott, McCormick et al., 2018), another group of undergraduate students from universities on the U.S. Gulf Coast (Currier, McDermott, Hawkins et al., 2018), and an online sample of veterans who completed one or more war-zone deployments (Currier et al., 2015). Of the 2,127 participants across the seven studies, 711 met or exceeded the clinical cutoff score of 10 for moderately severe depression symptoms on the PHQ-8 (Kroenke et al., 2009). The average age of selected participants was 36 years old (SD = 15.5). Please refer to Table 1 for the other demographic and S/R background features of this aggregated sample and Supplement 1 for more details about the datasets on which Study 1 was based.
Table 1
Demographic of spiritual/religious backgrounds of study participants
Factor | Study 1 (N = 711) | Study 2 (N = 303) | Study 3 (N = 121) |
|---|---|---|---|
Gender: | |||
Male | 420 (59.1%) | 142 (46.9%) | 65 (53.7%) |
Female | 287 (40.4%) | 160 (52.8%) | 53 (43.8%) |
Transgender | – | – | 3 (2.5%) |
Race/Ethnicity: | |||
Black | 135 (19.0%) | 134 (44.2%) | 25 (20.7%) |
White | 469 (66.0%) | 66 (21.8%) | 92 (76.0%) |
Hispanic/Latino(a) | 50 (7.0%) | 34 (11.2%) | 5 (4.1%) |
Native American | 9 (1.3%) | – | 1 (0.8%) |
Asian American | 23 (1.7%) | 48 (15.8%) | 1 (0.8%) |
Multi-Racial | 25 (3.5%) | 12 (4.0%) | – |
Other background | 9 (1.3%) | 7 (2.3%) | 2 (1.7%) |
Religious Affiliation: | |||
Christian | 401 (56.4%) | 183 (60.4%) | 75 (62.0%) |
Atheist/None | 101 (14.2%) | 38 (12.5%) | 11 (9.1%) |
Jewish | 2 (0.3%) | 1 (0.3%) | 1 (0.8%) |
Muslim | 5 (0.7%) | 34 (11.2%) | 2 (1.7%) |
Hindu | – | 13 (4.3%) | – |
Other Religion | 68 (9.6%) | 2 (0.7%) | 11 (9.1%) |
Spiritual/Religious Identity: | |||
Spiritual but not Religious | 176 (24.8%) | 31 (10.2%) | – |
Religious not Spiritual | 42 (5.9%) | – | – |
Both Spiritual and Religious | 230 (32.3%) | – | – |
Neither Spiritual or Religious | 84 (11.8%) | – | – |
Spiritual/Religious Endorsement: | |||
Spiritual or Religious | – | – | 76 (62.8%) |
Connected with S/R community | – | – | 42 (5.8%) |
S/R contributed to problems | – | – | 230 (31.7%) |
S/R source of strength | – | – | 86 (11.8%) |
Suicide Risk: | |||
Any thoughts of suicide (past 30 days) | – | – | 38 (31.4%) |
Attempt suicide (past 30 days) | – | – | 1 (0.8%) |
Measures
Exline et al.’s (2014) Religious and Spiritual Struggles Scale (RSS) was used to assess ways that participants were struggling with their S/R. Namely, 22 of the 26 RSS items were used to gauge struggles with God or the divine (5 items; e.g., “Questioned God’s love for me”), morality (4 items; e.g., “Felt guilty for not living up to my moral standards), absence of ultimate meaning (4 items; e.g., “Had concerns about whether there is any ultimate purpose to life or existence), interpersonal struggles (5 items; e.g., “Had conflicts with other people about religious/spiritual matters”), and religious doubting (4 items; e.g., “Worried about whether my beliefs about religion/spirituality were correct”). Each study began with this statement to address anticipated S/R diversity among the participants:
“It is not uncommon for people to struggle in their spirituality or faith at different points in life. Below you will find questions that ask about different types of ways that people might struggle in this area. In responding to items, please feel free to substitute an alternate word that captures whatever “God” means to you. For example, you may see God as a Higher Power, Divine Being, Great Spirit, Nature, a Positive Energy Providence, Fate, etc.”
Items were rated on a five-point scale from 1 = Not at all to 5 = A great deal, such that higher total scores indicated greater struggles with S/R.
The Patient Health Questionnaire (PHQ-8; Kroenke et al., 2009) was used across studies to assess eight symptoms of major depressive disorder (e.g., anhedonia, depressed mood). The PHQ-8 is a widely used instrument that asks respondents to rate the frequency of symptoms over the past month, with scores ranging from 0 = not at all to 3 = nearly every day. A score of 10 or above indicates symptoms that are moderately severe or worse in nature (Kroenke et al., 2009).
Analysis Plan
We used a combination of empirical and theory-driven approaches to identify representative, psychometrically strong items from the five subscales of the RSS used across the seven studies (Divine, Interpersonal, Doubt, Moral, Ultimate Meaning). Specifically, summative self-report instruments with a total score and positively correlated subscale scores are often more unidimensional than multidimensional (Rodriguez et al., 2016). Although the RSS can be used as a multidimensional measure (Exline et al., 2014), many researchers implement the measure in a unidimensional fashion in which they solely report on the total score. A short form representing the total RSS score might be especially useful for research and clinical practice. Moreover, if an instrument is unidimensional, items that load highly on the general factor can be selected based on empirical criteria to represent a close approximation of the total score (Stucky & Edelen, 2014). Therefore, we used bifactor modeling via confirmatory factor analysis (CFA) to examine the dimensionality and the reliability of the fuller-length RSS and inform our item retention criteria for the short form. This is possible because a bifactor model partitions variance between a general factor representing the common variation among all items and specific factors representing the unique contributions of subscale factors when controlling for the general factor.
Following the guidelines proposed by Rodriguez and colleagues (2016), we selected the following ancillary bifactor indices to identify items for a shorter (but still internally consistent) version of the instrument: explained common variance (ECV); item-explained common variance (IECV); percent of uncontaminated correlations (PUC); and variations of coefficient omega such as omega (ω), omega hierarchical (ωH), and omega hierarchical subscale (ωHS). The coefficient ω represents a model-based estimation of the internal reliability of the common variance among all items. The two variants of ω represent variance attributable to the general factor alone (ωH) and variance attributable to specific factors alone (ωHS). In contrast, ECV is a model-based index of dimensionality and represents the proportion of all common variance explained by the general factor. ECV interacts with PUC (i.e., percentage of covariance terms in the covariance matrix, which only reflects variance from the general factor) to inform decisions about dimensionality of an instrument. When PUC values are lower than 0.80, general ECV values are greater than 0.60, and OmegaH greater than 0.70, multidimensionality is not severe enough to disqualify the interpretation of the instrument as primarily unidimensional (Reise et al., 2013). Additionally, if an instrument is essentially unidimensional, retaining items with variance primarily explained by the general factor by examining IECV values can yield a shorter, unidimensional version of an instrument’s total score (Stucky & Edelen, 2014).
Stauner et al. (2016) conducted the only bifactor representation of the RSS to date; in so doing, they found mixed evidence for multidimensionality but did not calculate ancillary bifactor indices of dimensionality or reliability. Thus, rather than relying solely on information from the bifactor model, we combined our empirically-driven item retention criteria with scientific and theoretical work on the RSS to ensure appropriate representation of the construct. Additionally, we used commonly reported fit indices recommended by Kline (2023) and these cut values to evaluate the overall fit of our model to the data: comparative fit Index (CFI) and Tucker-Lewis index (TLI; values approaching or exceeding 0.95 indicate acceptable fit); root mean square error of approximation (RMSEA; values close to 0.01, 0.05, and 0.08 represent excellent, good, and marginal fit respectively); and the standardized root mean residual (SRMR; values less than 0.08 indicate acceptable fit).
Results
Bifactor Modeling
A correlated factors model with five interrelated RSS factors provided an acceptable fit to the data, Χ2 (199, N = 711) = 555.31, p < 0.001, CFI = 0.951, TLI = 0.943, RMSEA = 0.050 (90% CI = 0.045, 0.055), and SRMR = 0.038. A bifactor model also provided an acceptable fit, Χ2 (187, N = 711), = 508.85, p < 0.001, CFI = 0.956, TLI = 0.945, RMSEA = 0.049 (90% CI = 0.044, 0.054), and SRMR = 0.032. The bifactor model was a significantly better fit than the correlated factor model, scaled ΔΧ2 (12) = 43.61, p < 0.001. Thus, a bifactor model appeared to be an appropriate representation of the RSS on par with the correlated factor model originally specified by Exline and colleagues (2014). Table 2 outlines results for comparisons between the varying models for the RSS.
Table 2
Model comparisons of RSS
Model | χ2 Value | df of χ2 | N | p-value of χ2 | CFI | TLI | RMSEA | 90% CI of RMSEA | SRMR |
|---|---|---|---|---|---|---|---|---|---|
(1) Correlated 5-factor RSS | 555.305 | 199 | 711 | < .001 | .951 | .943 | .05 | [.045, .055] | .038 |
(2) Bifactor model | 508.847 | 187 | 711 | < .001 | .956 | .945 | .049 | [.044, .054] | .032 |
(3) Unidimensional model | 2702.788 | 209 | 711 | < .001 | .657 | .621 | .130 | [.125, .134] | .096 |
(4) Second order model | 563.986 | 204 | 711 | < .001 | .95 | .944 | .04 | [.045, .055] | .032 |
A closer examination of the ancillary bifactor indices suggested that the RSS can be condensed into a shorter, unidimensional measure. First, the general factor appeared to be relatively strong; the ECV for the RSS general factor was 0.59, suggesting that more than half of all the common variance among RSS items was explained by the RSS general factor. Second, the PUC was high (PUC = 0.84), indicating that approximately 84% of all correlations in the model matrix reflected variance attributable to the RSS general factor. Likewise, IECV values for items ranged from 0.30 to 0.81, with a majority (68%) evidencing IECV values greater than 0.50. Thus, the majority of variance in most RSS items was explained by a general RSS factor. Regarding reliability, coefficient ω values suggested excellent internal consistency for the RSS general factor (0.96) and values ranging from 0.85 (moral struggles) to 0.90 (divine struggles) for the specific factors. According to the ωH coefficient, a majority (84%) of reliable variation in the RSS raw total score may be attributable to the general factor. Indeed, after partitioning out the variance attributable to the general factor, only between 27% (divine struggles) and 52% (interpersonal struggles) of the reliable variation was left explained by the specific factors, as evidenced by the ωHs model coefficients.
Item Retention
Although results suggested the RSS could be shortened into a brief unidimensional screening measure, ancillary bifactor indices suggested multidimensionality that would preclude viewing the RSS as a solely unidimensional instrument. In particular, only one item measuring S/R doubts or questions evidenced IECV values above Stucky and Edelen’s (2014) validated criteria for a reliable indicator of a general factor. In other words, the RSS had sufficient multidimensionality to preclude only selecting items with the highest loadings on the general factor to develop a shorter, unidimensional form. Accordingly, we drew upon previous research and theoretical conceptualizations for distinct forms of struggle assessed on the RSS to cover the content domain (Exline et al., 2014).
To this end, we selected five items (one from each of the five RSS subscales) to develop the RSS-5. Table 3 displays the unidimensional and correlated factor loadings, as well as the IECV values from the bifactor model for all RSS items. In accordance with our bifactor model results, we included two items capturing doubt and divine struggles that were highly representative of the general RSS factor: “Felt troubled by doubts or questions about spirituality/religion” (doubt-related struggle, Item 24) and “Felt as though God had abandoned me” (divine struggle, item 19). In keeping with Exline et al.’s (2014) original findings and goals to develop a multidimensional instrument, we also retained three items assessing meaning, interpersonal, and moral struggles that loaded highly on the unidimensional factor and evidenced the strongest loadings on their respective correlated factors: “Questioned whether life really matters” (ultimate meaning struggle, item 7), “Felt rejected or misunderstood by religious/spiritual people” (interpersonal struggle, item 17), and “Worried that my actions were morally or spiritually wrong; moral struggle, item 14).
Table 3
RSS Unidimensional factor loadings, IECV values, and model fit statistics
RSS Item | Unidimensional Factor Loadings | Correlated Factor Loadings | IECV |
|---|---|---|---|
1. Felt guilty for not living up to morals (moral) | .49 | .46 | .52 |
2. Felt angry at God (divine) | .69 | .40 | .72 |
3. Concerns about ultimate meaning (ultimate meaning) | .65 | .37 | .76 |
4. Felt hurt/mistreated/offended by S/R people (interpersonal) | .50 | .62 | .38 |
5. Struggled to figure out what I really believe (doubt) | .74 | .38 | .79 |
7. Questioned whether life really matters (ultimate meaning) | .67 | .62 | .52 |
8. Felt torn between what I wanted and morals (moral) | .56 | .56 | .48 |
9. Questioned God’s love for me (divine) | .74 | .38 | .77 |
10. Had conflicts with others about S/R (interpersonal) | .49 | .53 | .43 |
12. Felt as though life had no deeper meaning (ultimate meaning) | .66 | .58 | .53 |
13. Felt angry at organized religion (interpersonal) | .41 | .58 | .30 |
14. Worried my actions were morally wrong (moral) | .55 | .61 | .41 |
15. Felt confused about S/R beliefs (doubt) | .76 | .41 | .78 |
16. Felt as though God was punishing me (divine) | .71 | .38 | .76 |
17. Felt rejected/misunderstood by S/R people (interpersonal) | .59 | .57 | .51 |
19. Felt as though God had abandoned me (divine) | .76 | .55 | .61 |
20. Worried whether S/R beliefs were correct (doubt) | .62 | .46 | .61 |
21. Wrestled with attempts to follow morals (moral) | .60 | .56 | .52 |
22. Questioned if life will make any difference (ultimate meaning) | .66 | .56 | .56 |
23. Felt as though God had let me down (divine) | .72 | .53 | .61 |
24. Felt troubled by S/R doubts or questions (doubt) | .77 | .37 | .81 |
26. Felt that others were looking down on me (interpersonal) | .48 | .55 | .42 |
Internal Consistency
Cronbach’s alpha was 0.77 for the RSS-5 items in the aggregated sample (see Table 4).
Table 4
Descriptive statistics of study measures and bivariate correlations for study 1, 2, and 3 samples
Cronbach’s alpha | Mean (SD) | Correlation with RSS-5 Total Score | |
|---|---|---|---|
Study 1 Sample (N = 711) | |||
RSS-5 total score | 0.77 | 12.08 (5.10) | –- |
Depression (PHQ-8) | 0.74 | 16.17 (4.49) | 0.40** |
RSS total score (without demonic) | 0.94 | 53.60 (19.96) | 0.95** |
Study 2 (N = 303) | |||
RSS-5 total score | 0.82 | 9.19 (4.43) | |
Depression (CES-D) | 0.91 | 39.14 (11.54) | 0.50** |
Anxiety (STAI-S) | 0.94 | 40.88 (12.40) | 0.43** |
Meaning- Search (MLQ-Search) | 0.84 | 25.72 (6.21) | 0.18* |
Meaning- Presence (MLQ-Presence) | 0.88 | 24.22 (6.66) | −0.39** |
Life Satisfaction (SWLS) | 0.82 | 21.03 (6.27) | −0.33** |
Flourishing Scale | 0.87 | 88.24 (18.84) | −0.43** |
Study 3 (N = 121) | |||
RSS-5 total score | 0.85 | 9.86 (5.03) | –- |
Psychological Distress (CORE-10) | 0.90 | 18.35 (9.65) | 0.61** |
Depression (PHQ-2) | 0.91 | 3.15 (2.22) | 0.44** |
Anxiety (GAD-2) | 0.87 | 3.55 (2.13) | 0.51** |
Suicidal thoughts past month | 1.67 (1.52) | 0.49** | |
Suicide attempts past month | 1.01 (0.20) | ||
S/R affiliation | 0.14 | ||
Connection with S/R community | −0.06 | ||
S/R contributing to problems | 0.33** | ||
S/R is a strength | 0.07 | ||
Interest in integrating S/R in treatment | 0.22* | ||
Convergent Validity
Scores on the RSS-5 were moderately positively correlated with the PHQ-8 and strongly positively correlated with the longer version of the RSS (see Table 4).
Discussion
Results of this first study provide initial evidence for the factor structure, reliability, and validity of the RSS-5. Drawing upon an aggregated sample of adults from seven studies who exceeded the clinical cutoff for depression symptoms on the PHQ-8 (see Supplement 1 for the details), bifactor modeling results supported a general factor that accounted for 58.6% of the variance in RSS scores. However, consistent with Exline et al.’s (2014) results, we also found sufficient multidimensionality, such that a unidimensional short form based solely on items with the strongest loadings on the general factor would not fully capture all of the different forms of spiritual struggles. Indeed, the general factor of the longer RSS appeared to primarily measure doubt-related struggle in our sample, according to the IECV values. Therefore, five items were selected that loaded highly on the divine, interpersonal, doubt, moral, and ultimate meaning struggles, all of which also loaded highly onto the general factor. In turn, the selected RSS-5 items demonstrated acceptable internal consistency and convergent validity with the overall severity of depression symptoms. Further, scores on the RSS-5 correlated with longer version of the RSS at 0.95, suggesting a high degree of overlap between the two forms of the RSS.
Study 2
Methods
Participants and Procedures
The second sample consisted of undergraduates enrolled in a public institution in the southeastern U.S. who completed the RSS-5 as part of a general survey to receive course credit through the university’s online research platform between January and April 2023. The survey also included measures assessing sociodemographic factors, personality factors, and psychological well-being. Participants were excluded if less than 50% of the survey was completed, they finished in less than 5 min, or they failed more than half of validity checks. The final sample included 303 participants with an average age of 22 years (SD = 4.9). Please refer to Table 1 for other demographic and S/R features of the participants.
Measures
In addition to the RSS-5 developed in Study 1, participants completed S/R background items and the below measures of mental health symptoms and well-being (see Table 1).
Depression
The CES-D (Radloff, 1977) is a 20-item measure that assesses the extent to which symptoms of depression have been experienced over the past week. Response options range from 0 = Rarely or none of the time to 3 = Most or all of the time; total scores range from 0 to 60, with a score of 16 or higher indicating severity of depressive symptoms in the clinical range that indicate a probable need for a clinical intervention.
Anxiety
The STAI-S (Spielberger, 1983) is another self-report measure that consists of 20 items assessing anxiety symptoms at the moment of administration. Responses are scored on a 4-point scale ranging from 1 = Not at all to 4 = Very much so; total scores of 40 and higher are considered to indicate probable clinical levels of anxiety symptomatology that might indicate a need for treatment.
Meaning
The Meaning in Life Questionnaire (Steger et al., 2006) is a 10-item measure assessing the presence of meaning (five items) and search for meaning in life (five items). Item responses range from 1 = “Absolutely Untrue” to 7 = “Absolutely True.” Example items include: “I understand my life’s meaning” (presence of meaning item) and “I am looking for something that makes my life feel meaningful” (search for meaning in life item).
Life Satisfaction
Satisfaction with Life Scale (SWLS; Diener et al., 1985) is a 5-item measure assessing the extent of one’s life satisfaction. Example items include: “In most ways my life is close to my ideal” and “If I could live my life over, I would change almost nothing.” Items are rated on a 7-point scale ranging from 1 = “Strongly disagree” to 7 = “Strongly agree.” Total scores above 20 indicate some extent of life satisfaction (Diener et al., 1985).
Flourishing
The Flourishing Measure (VanderWeele, 2017b) is a 12-item measure that assesses happiness and life satisfaction (Items 1–2; e.g., “Overall, how satisfied are you with life as a whole these days?”), mental and physical health (Items 3–4; e.g., “In general, how would you rate your physical health?”), meaning and purpose (Items 5–6; e.g., “Overall, to what extent do you feel the things you do in your life are worthwhile?), character and virtue (Items 7–8; e.g., “I always act to promote good in all circumstances, even in difficult and challenging situations), close social relationships (Items 9–10; e.g., “I am content with my friendships and relationships), and financial and material stability (Items 11–12; “How often do you worry about being able to meet normal monthly living expenses?”). Item responses range from a 0–10 scale, with response anchors from “Extremely Unhappy” to “Extremely Happy,” “Strongly Disagree” to “Strongly Agree,” and “Poor” to “Excellent.”
Analysis Plan
We first confirmed the factor structure of the RSS-5 via structural equation modeling. Specifically, we specified a unidimensional factor structure and evaluated the model based on same indices of fit detailed in Study 1. Descriptive statistics of the RSS-5, internal consistency, and convergent validity were then calculated in this undergraduate sample.
Results
Preliminary Analysis
All missing items for demographic, psychological, and well-being measures accounted for less than 1% of the sample. Overall, 52.6% of participants scored in the clinical range for depression symptoms on the CES-D and 51.5% scored above the clinical cutoff for anxiety symptoms on the STAI-S.
Primary Analysis
Internal Consistency
The RSS-5 items yielded a Cronbach’s alpha of 0.82 in the second sample (see Table 4).
Factorial Validity
The unidimensional RSS-5 evidenced excellent fit to the data in the aggregated validation samples, Χ2 (5, N = 303) = 14.326, p = 0.0137, CFI = 0.966, TLI = 0.933, RMSEA = 0.078 (90% CI = 0.032, 0.128), and SRMR = 0.036. Of note, the RMSEA, which was the only index of fit that was marginal for the RSS-5, has been shown not to be as reliable in models with smaller degrees of freedom (Kenny & McCoach, 2003; Kenny et al., 2015).
Convergent/Divergent Validity
Bivariate correlations between participants’ RSS-5 scores and indices of negative and positive mental health are presented in Table 4. RSS-5 scores were strongly positively correlated with mental health symptoms on the CES-D and STAI-S. The RSS-5 was also positively linked with MLQ – search for meaning. In contrast, RSS-5 scores were moderately negatively correlated with responses on the MLQ – presence of meaning, SWLS, and Flourishing Scale.
Discussion
Focusing on an undergraduate sample, Study 2 provided additional important information about the factor structure, reliability, and construct validity of the RSS-5. CFA results supported the use of the RSS-5 as a unidimensional measure. Internal consistency was also again in a favorable range and remaining analyses supported construct validity of the measure. Specifically, in keeping with research with the original version of the RSS (e.g., Bockrath et al., 2022; Damen et al., 2021; Currier et al., 2019a; Leavitt-Alcantara et al., 2023; Murphy et al., 2016), RSS-5 scores were strongly positively associated with depression and anxiety symptoms; search for meaning was linked with RSS-5 scores at moderate level. In contrast, RSS-5 scores were highly inversely linked with perceived presence of meaning in life, life satisfaction, and flourishing. Additionally, it is important to note the high levels of distress in this undergraduate sample. There could be multiple factors to consider regarding these rates of distress, including sampling from a racially and ethnically diverse university, socio-economic status of students, or even cohort effects such as attending college following the height of the COVID pandemic.
Study 3
Methods
Participants and Procedures
The last sample consisted of 121 adults who completed the RSS-5 between December 2021 and December 2022 as part of routine intake procedures in an integrated behavioral health outpatient clinic that offers evidence-based psychotherapies, primary care, and peer support to persons who are struggling with mental health and/or substance use disorders. The clinic is certified as an outpatient substance abuse treatment program by the mental health department in the state in which it is located and received a Certified Community Behavioral Health Clinic (CCBHC) Expansion Grant from SAMHSA in August 2021. Although the organization specializes in caring for military service members, veterans, first responders, and their family members, the clinic also serves persons who did not serve in the military. The organization serves patients from religiously diverse backgrounds (including non-religious), such that clinicians are expected to honor patients’ preferences, needs, goals, and cultural beliefs and values in all cases. The average age of participants was 40 years old (SD = 14.4). Demographic and S/R backgrounds for the third sample are outlined in Table 1.
Measures
In addition to the RSS-5, participants completed the below assessments of mental health symptomatology and challenges in the intake questionnaire:
Spirituality
S/R background factors were asked on a yes/no response (0 = No, 1 = Yes): affiliation (“Do you view yourself as a religious and/or spiritual person?”), connection with a community (“Are you connected with a religious and/or spiritual community?”), problems (“Has your religious faith and/or spirituality contributed to some of your problems?”), source of strength (“Has your religious faith and/or spirituality been a source of strength in your life?”), and preferences for treatment (“Would you like to explore ways of including your faith and/or spirituality in your care?”).
Psychological Distress
The CORE-10 (Barkham et al., 2013) is a 10-item measure of common symptoms of psychological distress. Assessed on a five-point scale with anchor points of 0 = Not at all to 4 = Most or all of the time, items on this well-established instrument for tracking outcomes in psychotherapy capture symptoms of anxiety (e.g., “I have felt tense or anxious”), depression (e.g., “I have felt despairing or hopeless”), suicide risk (e.g., “I made plans to end my life”), and psychosocial and relational functioning (e.g., “I have felt able to cope when things go wrong, “I have felt that I have someone to turn to when needed” [reverse scored]). Scores of 10 or higher suggest clinical levels of psychological distress.
Depression
The PHQ-2 is an abbreviated version of the PHQ-8 that is composed of two items to screen for possible major depressive disorder (Kroenke et al., 2003). Items are scored from 0 = Never to 3 = Nearly Every Day, such that higher scores indicated greater depression symptoms. A total score of 3 or higher indicate a probable need for treatment.
Anxiety
The GAD-2 is an abbreviated version of the GAD-7 that consists of two items screening for generalized anxiety disorder. Items are scored from 0 = Never to 3 = Nearly Every Day, with a total of 3 indicating a probable need for treatment for anxiety (Kroenke et al., 2007).
Suicidality
Items from the revised version of Osman et al.’s (2001) Suicidal Behavior Questionnaire (SBQ-R) was used to assess frequency of suicidal ideation over past 30 days and the likelihood of attempting suicide in the future. The item assessing suicidal ideation was scored on a five-point scales in which 1 = “Never” and 5 = “Very Often” and attempt probability was assessed on a seven-point scale in which 0 = “Never” and 5 = “Very Likely.”
Analysis Plan
In keeping with the analytic approach in Study 2, we again confirmed the RSS-5 via CFA. Then, we calculated descriptive statistics of the RSS-5, internal consistency, convergent and incremental validity in this outpatient treatment-seeking sample.
Results
Preliminary Analysis
Initial frequency analyses revealed roughly 75% of participants scored above the clinical cutoff for psychological distress on the CORE-10, 44% of participants scored in the clinical range for depression symptoms on the PHQ-2, and 65% exceeded the threshold for anxiety symptoms on the GAD-2. Nearly 30% of the participants reported thoughts of suicide at least once in the past 30 days. The sample contained no missing values for demographic items and psychological measures at rates higher than 1% on all items.
Internal Consistency
The RSS-5 items yielded a Cronbach’s alpha of 0.85 in the third sample.
Factorial Validity
The unidimensional RSS-5 again evidenced excellent fit to the data in the clinical sample, Χ2 (5, N = 121) = 16.344, p = 0.006, CFI = 0.931, TLI = 0.862, RMSEA = 0.137 (90% CI = 0.067, 0.213), and SRMR = 0.045. Of note, the RMSEA, which was the only index of fit that was marginal for the RSS-5, has been shown not to be trustworthy (inflated) in models with small degrees of freedom (Kenny & McCoach, 2003; Kenny et al., 2015).
Convergent Validity
Bivariate correlations between participants’ RSS-5 scores and their S/R backgrounds and mental health symptoms are presented in Table 4. Scores on the RSS-5 were not associated with religious affiliation, connection with spiritual/religious community, or endorsement of S/R as a source of strength. In contrast, RSS-5 scores were moderately positively correlated with endorsement of S/R contributing to problems, and interest in including S/R in their care. In addition, RSS-5 scores were moderately to strongly correlated with psychological distress, as well as depression, anxiety, and suicidal ideation, all ps < 0.001 (See Table 4).
Incremental Validity
When controlling for scores on the PHQ-2 and GAD-2 in an initial step, we added RSS-5 scores in a second step to examine incremental validity in predicting suicidal ideation in the presence of depression and anxiety symptoms. In total, PHQ-2 and GAD-2 scores significantly explained patients’ differences in suicidal ideation in the first step, ∆R2 = 0.21, Fchange (2, 112) = 14.78, p < 0.001. The addition of RSS-5 scores also significantly increased the explained variance in the second study, ∆R2 = 0.15, Fchange (1, 111) = 26.49, p < 0.001.
Discussion
The RSS-5 again demonstrated factorial validity, internal consistency and evidence of construct validity in a clinical sample of adults seeking psychotherapy/counseling and/or primary care in an integrated behavioral health clinic. CFA findings supported a unidimensional factor structure and internal consistency was 0.85. RSS-5 scores were not associated with several S/R background factors (religious affiliation, connection with a spiritual/religious community, endorsement of S/R as a source of strength), possibly suggesting the distinctness of the measure from general indices of religiousness or spirituality S/R and applicability to many persons from across the spectrum of religiousness (including secular or non-religious). However, as one may anticipate, patients who endorsed S/R contributing to their problems and interest in exploring inclusion in their care generally had higher scores on the RSS-5. In keeping with Study 1 and 2, RSS-5 scores were also moderately to strongly correlated with greater mental health symptoms (psychological distress, depression, anxiety, and suicidal ideation) in this last sample. Consistent with the incremental validity of the original RSS in predicting suicide in other treatment-seeking samples (e.g., Raines et al., 2017), scores on the RSS-5 were also uniquely linked with suicidal ideation in the presence of depression and anxiety symptoms.
General Discussion
An amassing scientific literature highlights the importance of addressing spiritual struggles in clinical practice and research (Pargament & Exline, 2022). Notwithstanding the psychometric strengths of 14- and 26-item versions of the Religious and Spiritual Struggles Scale (RSS; Exline et al., (2014, 2022), a five-item version of the RSS might facilitate these advances. Overall, results from the three studies revealed the RSS-5 represents a structurally sound and reliable instrument for assessing the common forms of spiritual struggles from Exline et al.’s framework. Namely, even with reducing the measure to five items, internal consistency of the RSS-5 ranged from 0.77 to 0.85. In addition, scores on this short form overlapped highly with the original version of the RSS in Study 1 and were moderately to strongly associated with validated assessments of positive (well-being, flourishing, and perceived meaning in life) and negative (suicide ideation, depression and anxiety symptoms) mental health across the three studies. When accounting for scores on validated short forms of instruments assessing symptoms of major depressive disorder (PHQ-2) and generalized anxiety disorder (GAD-2), the RSS-5 also demonstrated incremental validity in predicting patients’ suicidal ideation over the past months in the clinical sample. In combination, these latter findings affirm the clinical relevance of spiritual struggles, assessed by the RSS-5, in varying indices of positive and negative mental health (e.g., Currier et al., 2015, 2018a, 2018b, 2019a, 2019b; Lemke et al., 2023; Wilt et al., 2017).
Although we supported a unidimensional factor structure for the RSS-5, there was also evidence of multidimensionality for selected items in Study 1 that aligned with psychometric findings for the original version (Exline et al., 2014; Stauner et al., 2016). From a practical view, clinicians and researchers might therefore use the total score of the RSS-5 to gauge the overall severity of spiritual distress with confidence that items in this short form also capture distinct ways that people who are experiencing mental health challenges often struggle with their S/R (divine, interpersonal, doubt, moral, and ultimate meaning struggles; Pargament & Exline, 2022). In keeping with Study 3, including the RSS-5 in an intake questionnaire could help clinicians to identify ways that patients could be struggling with their S/R. Namely, if patients endorse certain items as “Somewhat,” “Quite a bit,” or a “A great deal,” clinicians might inquire further about the specific forms of struggle and how these issues relate with the presenting problem(s) and are causing distress and maladaptive coping. However, whether focusing on longer versions or this short form, research will ideally identify a cutoff or threshold score on the RSS for determining clinical levels of spiritual distress to aid clinicians in case formulation and treatment planning. In doing so, emerging findings indeed suggest that many patients who are struggling with their S/R might benefit from clinicians offering a higher dosage of treatment and specifically attending to these issues in the therapeutic process (Currier et al., 2024; Pargament & Exline, 2022).
When spiritual struggles are co-occurring with other mental health challenges, clinicians and researchers might also use the RSS-5 in routine outcome monitoring (ROM). Research has supported the benefits of ROM (Barkham et al., 2023; Lambert et al., 2018). By tracking spiritual struggles and psychological outcomes on a session-to-session basis, practitioners of spiritually integrated psychotherapies might enhance engagement of patients who are struggling with their S/R and tailor treatment in ways that facilitate resolution of these issues and other clinical outcomes (e.g., reduce mental health symptoms, improve psychosocial functioning). When patients who are experiencing spiritual struggles are not improving, ROM might also signal a need for a change or modification of the treatment plan that might include a more deliberate exploration or processing of these issues, referral to a clergy person or other spiritual care professional, or other approaches (for a thorough discussion of clinical strategies, see Pargament & Exline, 2022). Relatedly, the RSS-5 might also support practice-based evidence research to identify interventions that might support transformation of spiritual struggles in psychotherapy. Notwithstanding evidence-based protocols for addressing spiritual struggles (e.g., Harris et al., 2018), such studies might guide clinicians to address spiritual struggles as a routine part of their practice. However, in addition to clarifying a clinical cutoff, research is also needed to determine clinically and reliable change on the RSS that might signify a successful treatment.
Without overlooking these possibilities, several limitations should be mentioned when considering implementing the RSS-5 in clinical practice and research. First, we utilized a cross-sectional design and cannot draw temporal inferences or make causal statements regarding spiritual struggles and outcomes in the three studies. Although research with the original RSS has documented temporal stability and precedence for spiritual struggles in predicting mental health symptoms over time (Bockrath et al., 2022), longitudinal studies are needed to determine test–retest reliability and predictive validity of the RSS-5. Second, we exclusively relied on self-report instruments for assessing depression and anxiety in ways that limit our ability to determine rates of diagnoses in the three samples and the sensitivity of the RSS-5 to detecting cases for these conditions. Research will ideally utilize clinical interviewing with a fuller range of mental health conditions to further evaluate construct validity of the RSS-5. Information on clinical diagnoses would also be needed for identifying a clinical cutoff and threshold for clinically significant change. Third, we also lacked data about the utility of the RSS-5 for evaluating treatment outcomes. Although the measure performed well as part of an intake questionnaire in the integrated behavioral health clinic in Study 3, we did not track patients’ health-related outcomes over time. As highlighted above, clinical research will ideally examine utility of the RSS-5 in routine outcome monitoring procedures in psychotherapy or other treatments. Fourth, demonic struggles were not included in our analyses. Though we do not consider this to be a significant weakness, it is a psychometric limitation of our findings. In reviewing Stauner et al., (2016), the bifactor results of Study 1 were similar with regard to item loadings on the general factor and specific factors, and we predict that demonic struggles would not have had a significant impact on the bifactor and item retention results of the RSS-5. In addition, while bifactor analysis is a strong tool for partitioning variance and understanding the reliability and dimensionality of an instrument, it is typically the least parsimonious model and can over-extract factors, so it may not be consistent with the underlying theory of a construct if the measure is not orthogonal (Bonifay et al., 2017). Looking ahead, research will ideally build on our psychometric findings for the RSS-5 with these methodological considerations in mind.
With these limitations in mind, the RSS-5 provides a brief, psychometrically-validated tool for assessing spiritual struggles in clinical practice and research. In conclusion, we supported a one-factor solution for the measure while detecting evidence for multidimensionality in Study 1 for selected items in ways that converged with previous research on the original RSS (Exline et al., 2014; Stauner et al., 2016). Taken together, the three studies also yielded evidence for internal consistency and validity of the RSS-5 (convergent, divergent, and incremental). Although more research is needed along the lines described above, the RSS-5 might represent a viable tool for assessing spiritual struggles in mental health care settings. Looking ahead, having a short form of the RSS will hopefully advance translational research on spiritual struggles that might facilitate innovation for clinicians to tailor evidence-based interventions and/or develop novel approaches for addressing these issues in their practice.
Declarations
Competing Interests
There are no financial or competing interests to report. Study data was reviewed/approved by the IRB from the University of South Alabama.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
