Differential Item Functioning for Gender and Age of the DSM-IV Borderline Personality Disorder Criteria in a Large Clinical Sample

Borderline personality disorder (BPD) is a debilitating condition characterized by a pattern of unstable relationships, disturbed identity, impulsivity, self-harm, and emotional dysregulation (American Psychiatric Association (APA), 2013). Despite its relatively low prevalence in the general population, estimated at around 1%, BPD stands out as one of the most commonly diagnosed personality disorders in clinical settings, affecting up to 22% of individuals in inpatient samples and 12% in outpatient samples (Eaton & Greene, 2018; Ellison et al., 2018). The impact of BPD extends beyond its symptomatology, encompassing a significant socioeconomic burden and reduced life expectancy (Gunderson et al., 2018; Leichsenring et al., 2024).

The Fourth Edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV, APA, 1994) and its subsequent revision (DSM-5, APA, 2013) operationalize BPD through nine criteria, requiring at least five for a diagnosis (see Table 1). These criteria span a wide array of personality problems, including self-pathology (e.g., Identity disturbances), interpersonal dysfunction (e.g., Unstable relationships), emotional dysregulation (e.g., Uncontrolled anger and Affective instability), and behavioral dysregulation (e.g., Impulsivity and Self-injurious behavior). In spite of its heterogeneity, research has largely supported the conception of BPD as a unidimensional construct (Aggen et al., 2009; Clifton & Pilkonis, 2007; Jane et al., 2007; Johansen et al., 2004; McMahon et al., 2019; Sharp et al., 2012).

Due to its comprehensiveness, unidimensional structure, and extensive overlap with other PDs, BPD is often viewed as a global index of PD severity rather than a separate diagnostic category (Gunderson et al., 2018; Paap et al., 2022; Sharp et al., 2015). Global severity typically encompasses elements related to self-functioning and interpersonal functioning (APA. 2013) but may also include behavior oriented indicators. For example, the DSM-5 Alternative Model for Personality Disorders defines general severity in terms of impairment of self-functioning and interpersonal functioning, whereas the ICD-11 model for Personality Disorders extends this to include self-injurious behavior and impulsivity (World Health Organization (WHO), 2019). Thus, there still seems to be some controversy regarding the definition of general severity. Under the assumption that BPD is a good proxy for general severity, in-depth analyses of the psychometric properties of the BPD criteria can illuminate which criteria are the most reliable indicators of general PD severity. Given that overall severity is crucial in PD diagnosis according to contemporary diagnostic manuals (APA, 2013; WHO, 2019), understanding which criteria are influenced by common demographic variables such as gender and age is important, as such biases may impact PD prevalence rates across demographic groups.

For several decades, it was widely believed that BPD was more prevalent in women than in men, partly supported by epidemiological research (APA, 2013; Trull et al., 2010). This assumption has spurred research into potential gender bias within the criteria for BPD as a potential cause of differences in prevalence rates (Bozzatello et al., 2024; Sharp et al., 2014). Gender bias refers to the potential for the criteria to be influenced by societal stereotypes or expectations about gender. For example, traits such as emotional dysregulation, self-harm, and intense interpersonal relationships, which are central to BPD diagnosis, may be more socially acceptable or expected in females, potentially leading to overrepresentation of women diagnosed with BPD. Conversely, certain behaviors that are more commonly associated with males, such as impulsivity and aggression, may be underemphasized or overlooked in female patients. This can result in disparities in access to treatment and support for individuals with BPD based on gender.

Though recent epidemiological studies have indicated that the prevalence of BPD may be similar in both genders (Bozzatello et al., 2024), gender bias remains a concern as certain criteria may be more prevalent in men while others may be more prevalent in women, which may go unnoticed when counting the total number of criteria. A straightforward way to evaluate gender bias would be to compare prevalence rates of individual criteria between female and male participants. However, this is not an appropriate approach because severity is not taken into account. As it is plausible that higher prevalence of one specific criterion is associated with higher prevalence of other criteria, it is necessary to control for the severity of the disorder. In Item Response Theory (IRT), severity is taken into account by evaluating the psychometric properties of the BPD criteria on a latent scale; the “latent BPD severity scale” in this situation. One commonly used approach for assessing the psychometric properties of a criteria set is through the application of the two-parameter IRT model (or in case of polytomous data, the Graded Response Model), which estimates the discrimination parameter and the location parameter for each item. For dichotomous data, the location parameter, also known as difficulty or threshold parameter, refers to the location at the latent severity scale at which individuals have a 50% probability of being assigned a particular criterion. It essentially represents how “easy” or “difficult” an item is– a smaller location parameter indicates an easier item (requires less psychopathology), while a larger parameter indicates a more difficult one (requires higher levels of psychopathology). The discrimination parameter indicates how well an item differentiates between individuals with high and low levels of BPD severity. Both parameters can display differential item functioning (DIF). When there is DIF for gender in the discrimination parameter, the criterion is not equally effective in discriminating between men and women with the same position on the latent BPD severity scale. DIF in the location parameter implies that women have a different probability of endorsing the criterion than men, given the same severity level.

Table 2 summarizes studies investigating DIF for gender of the BPD criteria as assessed by clinicians or trained laypersons. In line with the tradition in the PD field, we report the threshold parameters instead of location parameters. Of note, the study of Aggen et al. (2009) investigated DIF from a factor-analytical perspective. This study is included in the table due to similarity in the interpretation of the parameter estimates, i.e., intercepts are analogous to threshold parameters and factor loadings are analogous to discrimination parameters. If we look at the specific studies, we see that the first IRT study on gender bias of the DSM-IV BPD criteria was published by Jane et al. (2007), using a sample of 433 Air Force recruits and 166 students. Unexpectedly, the criteria showed no evidence of DIF in this study. However, subsequent DSM-IV based studies consistently identified DIF for several criteria. The threshold parameter for the Impulsivity criterion was larger in women in several studies (Aggen et al., 2009; Benson et al., 2017; Hoertel et al., 2014; Sharp et al., 2014), implying that female participants had a lower probability of endorsing the impulsivity criterion than men, given the same location on the latent BPD severity scale. The threshold parameter for the eighth BPD criterion, Uncontrolled anger, was also larger for women in some studies (Benson et al., 2017; Sharp et al., 2014). Larger thresholds for men were found in three studies, involving three criteria, i.e., Affective instability, Chronic emptiness, and Self-injurious behavior (Aggen et al., 2009; Benson et al., 2017; Hoertel et al., 2014).

With respect to the discrimination parameter, only two studies found DIF for gender (results not provided in Table 2). Using a large community sample, Hoertel et al. (2014) found larger discrimination parameters in women for both Affective instability and Chronic emptiness, indicating that in female participants, these criteria were better suited than the other criteria to discern between those who were situated at the higher end of the latent BPD scale versus those who were at the lower end. Aggen et al. (2009), on the other hand, found larger factor loadings in men for Impulsivity using data from the Norwegian twin register. In sum, it appears that most studies found considerable DIF for gender, most pronounced for the threshold parameter for Uncontrolled anger and Impulsivity. Therefore, based on these studies, it appears that the aspiration for gender-neutral criteria is not fully realized. However, it is worth mentioning that only three studies utilized clinical samples, with two employing structured diagnostic interviews. Further studies are required to gain deeper insights into whether there is any variation in item functioning for the BPD criteria in clinical populations.

Ideally, diagnostic criteria should also be free from age-related DIF. However, achieving age-neutrality of the BPD criteria may be challenging. As highlighted by Sharp et al. (2019), the BPD criteria were not constructed in developmentally sensitive ways, and therefore, it is reasonable to assume that the BPD criteria may behave differently across age groups. At the bottom of Table 2, two IRT studies and one factor-analytical study are presented that have investigated DIF across different age groups. Based on the classic taxonomy of developmental periods, Sharp et al. (2019) analyzed three age cohorts of psychiatric inpatients in the range of adolescents (12–17 years), young adults (18–25 years), and older adults (≥ 25 years). All patients were assessed by the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II), administered by trained master’s level research assistants. There was DIF for all nine criteria when the adolescent and adult groups were compared. However, no DIF emerged between the two adult groups (young adults versus older adults). A recent NESARC study showed that Self-injurious behavior manifested itself differentially across age in female participants, i.e., older women had more severe BPD pathology before this criterion was endorsed compared to younger women (McMahon et al., 2019). Moreover, Unstable relationships discriminated BPD severity better in younger adults as compared with older adults, in both genders (results not provided in the table). Of note, these effects were only found in the youngest group (20–33 years) compared to the oldest group (65 years and older).

Considering these findings collectively, a discernable trend emerges indicating that Impulsivity, Uncontrolled anger, and Self-injurious behavior were the criteria most frequently exhibiting DIF across gender and age in these studies, corroborating the findings of a recent narrative review on gender differences in BPD (Bozzatello et al., 2024). This may imply that criteria evaluating the behavioral aspects of BPD to a larger degree contribute to differences in prevalence rates across these demographics than criteria that focus on self-pathology and interpersonal dysfunction. Thus, the utility the behaviorally oriented criteria in diagnostic decision making might be constrained, as these criteria could contribute to variations in prevalence across age and gender. Translating this position to modern conceptualizations of PD, this could also suggest that behaviorally oriented criteria are less useful as indicators of overall severity. However, findings are rather heterogeneous with a few studies also reporting DIF for Affective instability and Chronic emptiness. Better empirical evidence is needed before the results of DIF studies can be taken into account in the development of future diagnostic manuals. Notably, the results of the DIF analyses should be considered in light of the findings from other psychometric analyses. Therefore, comprehensive psychometric studies of large clinical samples using structured clinical interviews may be especially useful in this endeavor.

Employing a variety of psychometric approaches, we addressed six specific research questions. First, we wanted to assess the dimensional structure of the BPD criteria, assuming to find support for a unidimensional scale. Second, we sought to identify where on the latent BPD scale reliability was most optimal by evaluating local reliability across the latent BPD scale. Third, we wanted to assess the overall psychometric properties of the BPD criteria from an IRT perspective by examining the discrimination parameter and location parameter for each of the nine BPD criteria. Fourth, we aimed at investigating whether diagnostic subthreshold scores according to the SCID-II provided additional information in the assessment of BPD severity. Fifth, we sought to assess whether the BPD criteria behaved differently across gender. Based on the findings of Benson et al. (2017) and Sharp et al. (2014), we assumed that male patients would be more frequently assigned Impulsivity and Uncontrolled anger, given the same standing on the latent BPD severity scale. Sixth, we aspired to explore whether there was DIF for age by comparing two age groups, i.e., patients aged 18 to 25 years and patients aged 26 years and older– the same age groups as in Sharp et al. (2019). In line with the study of McMahon et al. (2019), we expected that Unstable relationships would discriminate better in younger adults and Self-injurious behavior would be more easily assigned to younger adults given the same standing on the BPD severity scale as older adults.

Consistent with our hypotheses, the dimensionality analyses confirmed the unidimensionality of the BPD criteria within this extensive outpatient cohort evaluated through the SCID-II. The primary findings of this study indicated significant differential item functioning (DIF) across gender and age dimensions: Three BPD criteria showed DIF for gender and five BPD criteria showed DIF for age. In line with the conclusions of the review of Bozzatello et al. (2024), we found that women were more likely to endorse the criterion Self-injurious behavior, whereas men were more likely to endorse the criteria Uncontrolled anger and Impulsivity (though not at the alpha < .017 level), given the same level of BPD severity. In contrast to the conclusion of Bozzatello et al. (2024), we did not find any gender differences with respect to Affective instability and Chronic emptiness. However, the latter criterion had poor discriminative ability whereas Affective instability had substantial DIF for age, only surpassed by Self-injurious behavior.

Among the nine BPD criteria, only Identity problems and Chronic emptiness were free from DIF for gender or age. However, Identity problems displayed significantly better discriminative ability. The fact that the Identity criterion was free from DIF aligns with previous research (see Table 2), suggesting that subjective experiences like these can be assessed reliably—at least when structured clinical interviews are conducted by experienced clinicians. Considering that BPD may reflect overall PD severity, these results provide empirical justification for the inclusion of Identity in the Level of Personality Functioning Scale of the DSM-5 Alternative Model for Personality Disorders (APA, 2013) and the general PD severity description in the ICD-11 (WHO, 2019).

Self-injurious behavior stood out as the most problematic criterion from a DIF point of view. Both our study and the study of Hoertel et al. (2014) found that higher levels of BPD pathology were needed before Self-injurious behavior was allocated to male participants. In accordance with McMahon et al. (2019), this criterion also displayed DIF for age in our study, with a large effect size. We might then wonder whether it is advisable to include self-injurious behavior as a severity marker in future diagnostic manuals (WHO, 2019). Self-harm is seen in relation to many mental health disorders and might as such be a more general indicator of mental ill health (Reichl & Kaess, 2021). Thus, in considering general severity of PD, it might be preferable to focus more on aspects reflecting self and interpersonal functioning, such as identity problems, which have demonstrated both good discrimination and a high degree of pathology in our sample.

Unstable relationships, Impulsivity, and Dissociation showed no evidence of differential item functioning (DIF) for gender and exhibited moderate effect sizes in terms of DIF for age. For Impulsivity, this finding was surprising since several previous studies reported significant gender-related DIF for this criterion (Benson et al., 2017; Hoertel et al., 2014; Sharp et al., 2014). Selection bias could partly explain these differences. In our sample, female patients were relatively young compared with male patients, as well as when compared with participants in other studies. Aggen et al. (2009) noted that Differential Item Functioning (DIF) of the Impulsivity criterion in women was less noticeable in the younger age group, which could account for the lesser prominence of DIF for Impulsivity in our sample. It is important to note that we employed a conservative p-value threshold (p <.017) to mitigate the risk of false positives, albeit at the expense of potentially overlooking subtle gender-related differences in Impulsivity. This stringent statistical criterion increases the likelihood of Type II errors, meaning that certain nuanced distinctions in impulsivity between genders may not have been detected.

The finding that Uncontrolled anger required a lower level of BPD severity in men to be rated as present is in line with the findings from several previous studies (Benson et al., 2017; Sharp et al., 2014). This criterion is often viewed as representing the emotional dysregulation aspect of BPD (Weinberg et al., 2011). However, in DSM-IV and DSM-5, this criterion emphasizes behavioral expressions of anger, e.g., “recurrent physical fights,” which is reproduced in the SCID-II. The inclusion of this behavior may lead to undue focus on physical aggression, at the expense of focus on interpersonal aggression. Thus, DIF might be avoided by focusing more on gender-neutral behavior or by inquiring about anger experiences rather than on the behavior itself.

The criterion Fear of being abandoned demonstrated a significantly lower intercept for male patients, indicating that this criterion was less readily assigned to male patients as compared with female patients, given the same level of BPD severity. The description of this criterion in the DSM-5 emphasizes frantic efforts to avoid real or imagined abandonment. Male patients might not readily identify with such “frantic efforts” but still may experience a strong sense of desperation, which is acted out in other ways than by begging, pleading, or threatening. Consequently, clinical assessments may benefit from a greater focus on the internal experiences of loneliness and abandonment in male patients, rather than on overt behaviors.

Hoertel and colleagues (2014) posited that age-related variations in self-injurious behavior might result from maturation in emotional regulation throughout the lifespan. Our results suggest that individuals with less improvement in emotional regulation are prone to experience more severe BPD pathology. This observation may be partially attributable to the interrelationship between affective instability and self-injurious behavior, which both manifested more prominently among older individuals presenting with advanced BPD pathology. Longitudinal studies have shown that Affective instability to be one of the most stable BPD criteria (Skodol et al., 2005). Affective instability is described as one of the core aspects for BPD pathology (Miller & Pilkonis, 2006), and a probable driving force of additional problematic behaviors in BPD (Trull et al., 2008). Affective instability has also been reported as one of the maintaining factors of non-suicidal self-injury in BPD (Reichl & Kaess, 2021). It should also be noted that Affective instability was indicative of underlying BPD pathology at the lower severity levels within our sample. Even in the older cohort, Affective instability was not indicative of severe BPD pathology as compared with the other criteria.

Consistent with the findings of Sharp et al. (2014), our study revealed that the criteria Chronic emptiness and Dissociation had limited discriminative power. These criteria may also have contributed to the relatively poor reliability at both ends of the BPD severity scale since the Emptiness criterion discriminated best at the lower end of the BPD severity scale and Dissociation discriminated best the higher end. Chronic emptiness is a complex feeling state that is experienced in various ways by different individuals, and overlaps somewhat with other expressions of psychopathology, such as depression, narcissistic PD, and schizophrenia spectrum disorders (D’Agostino et al., 2020). A systematic review of Chronic emptiness calls the criterion “under-researched” and the target of several (albeit not unifying) theories, nevertheless finding emptiness to be experienced as a sense of disconnection from self and others (Miller et al., 2020). Troubles with understanding and feeling connected to self and others is also central to personality disorders in general, and to BPD in particular (Bender & Skodol, 2007). As proposed by Price et al. (2019), this suggests that the Emptiness criterion may not be unique to BPD, but rather a broader factor within PDs or psychopathology at large.

In agreement with Al-Shamali et al. (2022), Dissociation was identified as one of the most severe criteria, and also had low discriminating power in our sample. Thus, on the one hand, these results suggest that the Dissociation criterion is relatively rare, particularly in less severe cases; on the other hand, they indicate that this criterion is relatively common among patients without BPD. Dissociation has been described as “ubiquitous” and is part of many different psychiatric disorders (Lyssenko et al., 2018); hence, it is difficult to delineate dissociation as a separate psychiatric disorder. As its name implies, it is the core element of dissociative disorders, and also plays a role in acute stress disorder, PTSD, schizophrenia, eating disorders, panic disorders, affective disorders, and OCD (Lyssenko et al., 2018; Spitzer et al., 2006). According to the review of Al-Shamali et al. (2022), treatment effectiveness may be diminished for patients with BPD in the presence of dissociative symptoms. This conclusion may partly explain our finding that dissociative symptoms are associated with more severe BPD in younger patients, i.e., selection bias. It is conceivable that therapists encountering young patients with severe BPD and pronounced dissociative symptoms, may find these cases particularly challenging and, as a result, are more likely to refer these patients to specialized treatment.

Echoing findings from our prior research on antisocial PD (Paap et al., 2020), which shared a similar methodology, we observed that the diagnostic subthreshold scores of the SCID-II offered limited diagnostic information, raising questions about their utility in the diagnostic process. One possible explanation is that clinicians use these scores infrequently. Perhaps, they paid relatively little attention to these subthreshold scores, since these scores do not count when designating a BPD diagnosis. The latter may amount to an argument convincing enough to discard subthreshold scores altogether in future diagnostic interviews. On the other hand, it could also be argued that subthreshold scores should be given a larger role in the diagnostic process, provided that clinicians are better trained in how to use these scores.

It is important to note that the clinicians involved in this study were highly trained in assessing PDs and worked regularly with these patients. Thus, generalizability of our findings to general clinical settings might be somewhat limited. In less specialized environments, DIF for gender and age could be even more pronounced. However, we cannot entirely rule out assessment bias in our study. The clinicians in this network may have had a stronger interest in BPD than the average clinician, and potentially held a more favorable attitude toward these patients. Given that impulsivity does not inherently carry a negative connotation, one interpretation is that it might be considered a ‘preferred criterion’ for female patients, reflecting a leniency bias (Podsakoff et al., 2012). Another potential bias that might have affected our clinicians is ‘implicit theory bias’—the tendency to hold implicit beliefs about the relationships between BPD criteria, leading to illusory correlations (Podsakoff et al., 2012). This may have been amplified by the sequential assessment of the ten PD categories according to the structure of the SCID-II. As a result, clinicians might have been inclined to rate criteria more similarly within a given PD diagnosis, which could have inflated the coherence of BPD symptoms.

A significant limitation of this study is our inability to provide detailed information on the interrater reliability within our sample. However, a video-based interrater reliability study conducted by the Network for Personality Disorders involved the rescreening of 24 patients, including 11 with BPD, by an experienced clinician (Arnevik et al., 2009)^a. The study reported a kappa value of 0.66 for BPD, suggesting a level of agreement that could be deemed satisfactory. Another limitation worth mentioning is that we did not directly investigate other PD criteria that might be part of general PD severity (Paap et al., 2022; Sharp et al., 2015).

Therefore, caution is advised when extrapolating our findings to broader assessments of general PD severity. However, by focusing on the BPD criteria only, comparison with previous studies is facilitated and generalizability increased.

In sum, consistent with previous research, we observed a notable tendency for the behavior-oriented criteria (Impulsivity, Self-injurious behavior, and Uncontrolled anger) to display DIF, most pronounced for Self-injurious behavior. While some issues might be addressed by shifting the focus towards internal experiences rather than overt behavior, this approach is not feasible for Self-injurious behavior due to its inherent orientation towards observable actions. Consequently, we suggest that the utility of Self-injurious behavior as a diagnostic criterion for general severity is limited. It should also be considered to abandon Dissociation and Chronic emptiness since these criteria were relatively poor indicators of the BPD construct.

Among the other BPD criteria, Unstable relationships, Identity problems, and Affective instability had the best psychometric properties, as indicated by relatively large discrimination parameters and no DIF for gender. The negative consequences of DIF for Fears of abandonment might be avoided by focusing more on internal experiences related to abandonment fears rather than observable behaviors, i.e., “Frantic efforts to avoid real or imagined abandonment”. Particularly noteworthy is the absence of DIF for age in Identity problems, which is promising, since this is a criterion that is central to assessing general personality pathology.¹

Based on our findings and a synthesis of prior research, caution is advised when relying on behaviorally focused criteria, especially Self-injurious behavior, to diagnose BPD. Given that the concept of general severity of personality pathology, central to contemporary diagnostic approaches, overlaps significantly with the criteria for BPD, our cautionary note extends to the operationalization and assessment of general severity as well. We suggest that criteria assessing self-pathology and interpersonal dysfunction may be better indicators of overall PD severity than those emphasizing overt behavior.