The field of child psychopathology and assessment of behavior problems requires the collection of data from multiple informants which often presents discrepancies between them. These discrepancies have very important implications for diagnosis, assessment, and treatment(De Los Reyes & Kazdin, 2005) There is growing evidence that these differences are due to more than just the context in which the child is assessed (De Los Reyes, 2011). Many studies have attempted to answer the reason for these inconsistencies, however, there is insufficient information on why these differences exist and how they might impact diagnosis (Petric & Szamosközi, 2018). Numerous studies examine predictors that may influence these discrepancies(Achenbach et al., 1987; De Los Reyes & Kazdin, 2005; Petric & Szamosközi, 2018; Youngstrom et al., 2000) exposes three important factors: the characteristics of the informants assessing the child, the characteristics of the children being assessed, and whether the type of symptomatology is considered externalizing or internalizing. Regarding the first factor, some studies suggest that there is a relationship between parental stress, parental psychopathology (e.g., depression), family functioning, and reporting discrepancies (Chi & Hinshaw, 2002). Existing studies on multi-reporter inconsistencies have focused on child demographic characteristics, yielding inconsistent results for age, gender, and ethnicity (Harvey et al., 2013). A meta-analysis conducted byDuhig et al. (2000) specifically addresses mother-parent agreement and demonstrates that agreement is higher when evaluating adolescents, as compared to children in early and middle childhood, and that gender is a moderating variable between parental agreement. On the other hand, Achenbach et al. (1987) andDe Los Reyes et al. (2015) state that, in terms of age, the cross-informant (e.g., parent and teacher) agreement was higher for younger children relative to adolescents and found no differences in gender. Studies show more existent discrepancies when assessing African American children compared to European American children (Harvey et al., 2013). However, other studies report that there is no significant relationship between ethnicity and multi-reporter agreement (Duhig, Kimberly Renk et al., 2000). Another widely investigated factor is the type of problem reported. This refers to whether the type of problem is internalizing or externalizing. Results suggest that agreement is higher when symptoms are observable compared to ratings of unobservable symptoms and that teachers rate higher when behavior is disruptive while parents are more sensitive than teachers to depressive or anxious symptomatology in children (De Los Reyes, 2011; Duhig, Kimberly Renk et al., 2000; Youngstrom et al., 2000).
As previously commented, high parent-teacher disagreement is reported by several studies. However, few studies specifically examine parent-teacher disagreement in Oppositional Defiant Symptomatology (ODS), despite it being one of the main reasons for referral in the child mental health setting (Lavigne et al., 2009). According to the Diagnostic and Statistical Manual of Mental Disorders 5th Edition (DSM-5, American Psychiatric Association, 2013) this is a multidimensional disorder with subcategories of symptoms.
A meta-analysis made by (Petric & Szamosközi, 2018) examined parent- teacher agreement on ODS and child characteristics like age, sex and clinical status as moderators. The overall effect size for parent-teacher agreement was moderate and none of the moderators were significant. As stated before, externalizing problems show a greater correspondence than internalizing ones (Duhig, Kimberly Renk et al., 2000). Literature shows that there are some child characteristics associated with externalizing behaviors that include ODS. For instance, temperamental characteristics may enhance or attenuate the development of ODS and other behavioral disorders, and multiple studies show that extreme temperamental characteristics such as high negative emotionality, high extraversion, and poor effortful control are associated with early manifestations of behavioral disorders and are associated with externalizing disorders (Eisenberg et al., 2009; Hirshfeld-Becker et al., 2002; Lavigne et al., 2012). Research has also shown that depending on the irritability trajectory it can predict externalizing or internalizing symptomatology (Eisenberg et al., 2009; Ezpeleta et al., 2016). Finally, aggression and callousness, have shown to be a risk factor for subsequent delinquent acts and behavioral disorders (Gao & Zhang, 2016; Kolko & Pardini, 2010).
Given the lack of literature that focus specifically on how child emotional characteristics predict parent-teacher agreement of ODS, this study aims to (a) analyze the disagreement between parents and teachers on ODS, (b) identify what factors (from demographic and emotional characteristics of the child) related to parent-teacher agreement, and in what ways this might occur (e.g., linear, quadratic relationships). To our knowledge, no studies have investigated how emotional child characteristics may predict the disagreement in ODS symptomatology between parents and teachers in a Spanish sample. Furthermore, because parents and teachers tend to have less discrepancies when evaluating externalizing problem behaviors than internalizing ones, we hypothesized that variables such as aggression, temperament, callousness which are child characteristics that can prompt the development of externalizing behavior, compared to variables like irritability, depression and anxiety, will impact the informant’s disagreement on ODS diminishing the discrepancies between parents and teachers (Eisenberg et al., 2009; Lavigne et al., 2015; Smeekens et al., 2007).
Method
The sample was taken from a longitudinal study on vulnerability and risk factors for psychopathology in children aged 3 to 14 years old (Ezpeleta, de la Osa & Dómenech, 2014). Figure 1 outlines the sampling process. The initial sample consisted of 2283 families that were randomly selected from the census of 3-year-old preschoolers in the city of Barcelona (Spain) during the 2009–10 academic year. From this initial sample, 1341 families accepted the invitation to participate. The screening instrument used to select children with possible psychological problems included four items from the behavioral problems scale of the Strength and Difficulties Questionnaire (SDQ; Goodman, 1997) parent version plus four symptoms of Oppositional Defiant Disorder according to the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV, American Psychiatric Association, 1994) that were not present in the questionnaire (deliberately bothering people, blaming others, being touchy, angry and resentful). These eight items assess Oppositional Defiant Symptomatology (ODS) according to DSM-IV (range 0–16). The screening generated two groups: one included all the children with scores above the percentile 90 on the screening measure, or with a positive response for any of the eight DSM-IV ODD symptoms; and the other incorporated a random group of the children who did not reach the positive threshold. Therefore, the final sample for this study included 622 children, of whom 417 were part of the positive screening group (49.4% boys; mean ODS-parent = 12.3, SD = 4.8), and 205 children were part of the negative screening group (51.2% boys; mean ODS-parent = 7.3, SD = 3.5). Children with intellectual disabilities or pervasive deveopmental disorders, families with Spanish or Catalan language difficulties, without a primary caregiver who could report on the child, or who were moving to another city within the next year were excluded. Table 1 shows in detail the descriptive for the sample analyzed in the present study at three representative ages: 3 (N = 614), 8 (N = 390) and 14 (N = 287). There were no significant differences in sex (p =.220) or type of school (p =.461) between participants and drop-outs at the last follow-up (age 14). However, participating children were of higher SES (p <.001).
Fig. 1
Two-phase sampling design and study follow-ups
×
![]()
Table 1
Descriptive of the sample at ages 3, 8 and 14
Age 3 (N = 614) | Age 8 (N = 390) | Age 14 (N = 287) | |
|---|---|---|---|
% | % | % | |
Ethnicity | |||
White | 91.07 | 93.77 | 93.70 |
Other | 8.93 | 6.23 | 6.30 |
Sex | |||
Female | 49.62 | 48.96 | 51.13 |
Male | 50.38 | 51.04 | 48.87 |
SES | |||
High | 35.43 | 38.27 | 39.18 |
Medium + Medium-High | 46.24 | 46.53 | 44.13 |
Low + Low-Medium | 18.33 | 15.20 | 16.69 |
M (SD) | M (SD) | M (SD) | |
ODS Parents | 5.10 (2.71) | 3.26 (2.67) | 2.51 (3.04) |
ODS Teachers | 3.38 (3.21) | 2.75 (3.37) | 2.53 (3.29) |
CAS | 199.15 (24.58) | 198.83 (25.99) | 198.73 (24.76) |
ICU | 21.16 (10.01) | 20.15 (11.42) | 22.26 (10.49) |
CBCL Depressive | 1.98 (1.98) | 1.33 (1.93) | 1.50 (2.33) |
CBCL Anxious | 3.07 (2.60) | 2.22 (2.31) | 1.77 (1.99) |
Measures
Oppositional Defiant Symptomatology Scale
The ODS scale was first used as a screening instrument for selecting the final sample, and later completed by parents and teachers during each annual follow-up since children were 3 years onward. This measure identifies children who may potentially experience psychological distress, was a combination of four items retrieved from the behavioral problems scale of the Strength and Difficulties Questionnaire (SDQ, Goodman, 1997) parent version (“Frequently has temper tantrums or hot tempers”, “Usually obedient”, “Frequently argues with adults”, “Can be spiteful to others”) plus four DSM-IV ODS items (“Often deliberately annoys others”, “Often blames others for mistakes or misbehavior”, “Often touchy or easily annoyed by others”, “Often angry and resentful”) that are not included in the questionnaire. All the 8 items had three response options (0: not true; 1: somewhat true; 2: certainly true). The ordinal alpha for the 8-items of the ODS scale answered by parents ranges from 0.78 at age 3 to 0.92 at age 13 (mean 0.86), and from 0.91 at age 3 to 0.96 at age 8 (mean 0.93) when answered by teachers.
Children’s Aggression Scale (CAS) (Halperin & McKay, 2008)
This questionnaire was annually answered by teachers and assesses aggressive behavior according to the context, frequency, and severity of aggressive behavior in the child. It is comprised by 22 items with a 5-point Likert-type scale (0: never to 4: many days). The present study used the total aggression score. Cronbach’s alpha ranges from 0.82 at age 3 to 0.90 both at ages 8 and 14 (mean 0.87).
The CBCL is a questionnaire that measures child´s symptomatology from 1½ to 18 years of age. Information about the child´s behavioral and emotional problems was obtained from parents through 120 items since children were 3 up to 14 years. There are three response options (0: not true; 1: somewhat/sometimes true; 2: very true). The DSM-oriented depressive problems and anxious problems scales were used for this study. Cronbach’s alpha for depressive problems scale ranges from 0.81 at age 7 to 0.89 at age 14 (mean 0.85), and for anxious problems scale ranges from 0.79 at age 14 to 0.87 at ages 6 and 8 (mean 0.86).
Inventory of Callous-Unemotional Traits (ICU) (Frick, 2004)
This inventory examines affective traits such as insensitivity, indifference, and lack of emotion. The 24-item inventory was administered to teachers and information was obtained since children were 3 to 14 years old. Answers are coded on a 4-point Likert-type scale (0: not true at all to 3: definitively true). The total score will be used for this research. Cronbach’s alpha ranges from 0.88 at ages 7 and 14 to 0.93 at age 4 (mean 0.90).
Children Behavior Questionnaire Short Form (CBQ-SF) (Putnam & Rothbart, 2006)
The present study used the short version of the questionnaire with 94 items answered by parents in a 7 option Likert-type (1: extremely untrue to 7: extremely true). It has three second order dimensions to assess the children temperament: Surgency, Negative Affect (NA), and Effortful Control (EF). It was administered at the beginning of the study when the children were 3 years old and is followed up at 4, 5 and 7 years of age. Cronbach’s alpha for surgency scale ranges from 0.75 at age 7 to 0.84 at age 4 (mean 0.81), for negative affect scale ranges from 0.75 at age 7 to 0.85 at age 5 (mean 0.81), and for effortful control scale ranges from 0.71 at age 7 to 0.79 at age 3 (mean 0.75).
The Affective Reactivity Index (ARI) (Stringaris et al., 2012)
This scale allows to obtain a total score of the child´s irritability symptoms and respondents have to answer 7 items related to feelings, behaviors and impairment due to irritability. There are 3 response options (0: not true; 1: somewhat true; 2: certainly true). This scale was answered by teachers when children were 7 and 11 years old. Cronbach’s alpha was 0.86 and 0.88 at both ages respectively.
Diagnostic Interview of Children and Adolescents for Parents of Preschool and Young Children (DICA-PPYC)/ Diagnostic Interview of Children and Adolescents for Parents (DICA-P) (Ezpeleta, 1995; Ezpeleta et al., 2011).
The DICA is a computerized semi-structured interview for children aged 3 to 17 years old that generates diagnoses through algorithms following the DSM-IV/DSM-5 taxonomy (American Psychiatric Association, 1994, 2013). It was answered by parents annually during each follow up. For the present study, diagnoses for ODD were used to describe its prevalence as shown in Table 2.
Table 2
Comparison of the oppositional defiant symptomatology scores rated by parents and teachers
OD disorder | OD symptomathology | ||||||||
|---|---|---|---|---|---|---|---|---|---|
Parents | Teachers | Difference | |||||||
Age (N) | % | M | SD | M | SD | M | CI95% | p | g |
3 (614) | 10.5 | 5.10 | 2.71 | 3.38 | 3.21 | 1.72 | 1.42 to 2.02 | < 0.001 | 0.45 |
4 (598) | 8.9 | 3.54 | 2.60 | 3.32 | 3.20 | 0.22 | − 0.07 to 0.51 | 0.143 | 0.06 |
5 (526) | 8.5 | 3.30 | 2.67 | 2.88 | 3.04 | 0.42 | 0.13 to 0.70 | 0.004 | 0.13 |
6 (427) | 8.0 | 3.26 | 2.65 | 2.56 | 3.06 | 0.69 | 0.37 to 1.01 | < 0.001 | 0.21 |
7 (430) | 8.5 | 3.12 | 2.60 | 2.85 | 3.14 | 0.27 | − 0.05 to 0.60 | 0.101 | 0.08 |
8 (390) | 7.6 | 3.26 | 2.67 | 2.75 | 3.37 | 0.52 | 0.17 to 0.870 | 0.004 | 0.15 |
9 (396) | 9.2 | 3.04 | 2.74 | 2.90 | 3.18 | 0.13 | − 0.22 to 0.48 | 0.465 | 0.04 |
10 (371) | 10.3 | 2.94 | 2.73 | 2.83 | 3.34 | 0.11 | − 0.23 to 0.45 | 0.527 | 0.03 |
11 (357) | 10.1 | 2.71 | 2.50 | 2.48 | 3.17 | 0.23 | − 0.11 to 0.58 | 0.188 | 0.07 |
12 (281) | 10.4 | 2.47 | 2.67 | 2.26 | 3.07 | 0.22 | − 0.19 to 0.62 | 0.288 | 0.06 |
13 (311) | 11.9 | 2.23 | 2.89 | 2.71 | 3.51 | -0.48 | − 0.86 to − 0.10 | 0.013 | 0.14 |
14 (287) | 15.9 | 2.51 | 3.04 | 2.53 | 3.29 | -0.02 | − 0.43 to 0.38 | 0.919 | 0.01 |
Procedure
The study had the approval of the ethics committee of the Autonomous University of Barcelona. The recruitment of the families was done through the children´s schools starting in November 2009. All families gave written consent. In the screening phase of the sampling process parents of children of 3 years-old and their teachers completed the ODS scale and submitted it to the school.
Statistical Analysis
Statistical analyses were performed using Stata 17. The risk of Type I error was set at 5%. As children with behavioral problems were over-represented in the sample, in order to restore the population distribution of children with behavioral problems, all analyses were weighted with the inverse of the probability of being included in the sample, calculated for each screening group generated in the first phase of sampling.
Firstly, the usual descriptive analyses based on classical indices were performed. Comparison of means of ODS scores between parents and teachers was done through paired t test. Hedges’ g effect sizes for paired samples (Cohen, 1992) were calculated for each comparison. Absolute values were interpreted as: null effect for values < 0.20, small effect for values 0.20–0.50, medium effect for values 0.50–0.80 and large effect for values > 0.80.
Parent-teacher disagreement on ODS score was analyzed using the intraclass correlation coefficient (ICC) for absolute agreement, which was calculated at each follow-up. The Pearson correlation between both measures was also obtained. Magnitude of ICC was assessed according to Koo and Li (2016) who established poor agreement (ICC below 0.5), moderate agreement (ICC between 0.5 and 0.75), good agreement (ICC between 0.75 and 0.9) and excellent agreement (ICC above 0.9). Magnitude of Pearson correlation was assessed following Cohen’s guidelines (Cohen, 1988), who established small (0.10 < r <.30), medium (0.30 < r <.50) and large (r >.50) associations.
Following Laird and De Los Reyes (2013) to establish the relevant predictors of disagreement on ODS scores between parents (P) and teachers (T), separate polynomial regression models were estimated for each predictor (Z). A first polynomial model (Eq. 1) predicted T from P and Z including its second order interaction (PZ = P×Z), which test if the association between T and P ratings is moderated by Z. Therefore, a significant PZ term means that the association between T and P changes as Z change. In other words, a significant PZ term means that the agreement between teachers and parents lineally depends on the value of Z. To make easier the interpretation of the interaction coefficients the predictors P and Z were standardized (as a way of grand mean centering):
$$\:T\hspace{0.17em}={b}_{0}\hspace{0.17em}\hspace{0.17em}+\hspace{0.17em}{b}_{1}P\:+\:{b}_{2}{P}^{2\:}+\:{b}_{3}Z\:+\:{b}_{4}{Z}^{2}\:+\:{b}_{5}PZ\:+\:e$$
(1)
To detect non-linear disagreement between T and P informants at different ranges of values of the predictor Z, Saffer et al. (2021) suggested a second polynomial model including third order interaction (P2Z = P×P×Z and PZ2 = P×Z×Z) (Eq. 2). Therefore, a significant third order interaction means that: (1) the association between T and P2 changes as Z change, or (2) the association between T and P changes as Z2 change. In other words, a significant P2Z or PZ2 term means that the agreement between teachers and parents non-lineally depends on the value of Z:
$$\begin{aligned}T\hspace{0.17em}=&\hspace{0.17em}{b}_{0}\hspace{0.17em}+{\hspace{0.17em}b}_{1}P+{b}_{2}{P}^{2}+{b}_{3}{P}^{3}+{b}_{4}Z+{b}_{5}{Z}^{2}+\\&{b}_{6}{Z}^{3}+{b}_{7}PZ+{b}_{8}{P}^{2}Z+{b}_{9}{PZ}^{2}+e\end{aligned}$$
(2)
As each participant had a different number of follow-ups (from 1 to a maximum of 12) it was possible to control for individual variability over time by adding to each of the previous models a random intercept. The percentage of between-person variance with respect to the total variance was calculated using the intraclass coefficient correlation. Therefore, for each potential predictor of disagreement there were two multilevel random intercept models, one assessing linear disagreement and one assessing non-linear disagreement over time between ODS ratings of teachers and parents. The improvement achieved in Eq. 2 with respect Eq. 1 was tested through a likelihood ratio test.
Given the exploratory perspective of our work, all the PZ interaction estimations with p <.05 are commented. To also have an estimation of the results with a correction of the increased Type I error due to multiple comparisons, the false discovery rate (FDR) was applied and results that kept significant were highlighted (Benjamini & Hochberg, 1995; Glickman et al., 2014). Since attrition was related to SES, to make data conditionally missing at random each polynomial regression model included SES as an adjustment covariate.
For each predictor of teacher-parent disagreement with at least one statistically significant interaction, a graph of the predicted teacher ODS score against the parent ODS score and the predictor values was generated. The perfect agreement would correspond to a line over the main diagonal with the same value for parent and teacher ODS scores. The model used to obtain the predictions was the second or third order interactions depending on which level interactions were statistically significant. To have a common background, the 3 dimensions of the graphs were the standardized measures of T, P and Z, and the predictors were allowed to vary from − 3 to + 3.
Results
Comparison of ODS Score Ratings between Parents and Teachers
Table 2 summarizes the mean ODS scores obtained by parents and teachers at each age and its comparison. Parents reported statistically significant higher ODS than teachers for ages 3, 5, 6, 8, while teachers significantly reported more ODS only at age 13. Effect size was close to null for all ages except for age 3, where it was small.
Agreement and Association of ODS Score Ratings between Parents and Teachers
Table 3 shows ICC and Pearson correlation coefficients obtained to measure agreement and association between parents and teachers’ ratings at each age for the ODS scale. Both statistical indexes show an increasing pattern as age grew. Pearson correlations ranged from 0.17 at age 3 to 0.37 at age 14, which can be considered low to medium. The ICCs ranged from 0.27 at age 3 to 0.57 at age 14, which can be considered poor to moderate.
Table 3
Intraclass correlation coefficient (ICC) for ratings of the Oppositional Defiant Symptomatology by parents and teachers
Age | r | ICC | ICC CI95% |
|---|---|---|---|
3 | 0.17 | 0.27 | 0.12 to 0.39 |
4 | 0.23 | 0.37 | 0.26 to 0.46 |
5 | 0.32 | 0.50 | 0.40 to 0.57 |
6 | 0.31 | 0.46 | 0.35 to 0.55 |
7 | 0.32 | 0.45 | 0.33 to 0.54 |
8 | 0.35 | 0.49 | 0.58 to 0.38 |
9 | 0.29 | 0.47 | 0.35 to 0.56 |
10 | 0.39 | 0.56 | 0.46 to 0.64 |
11 | 0.30 | 0.48 | 0.36 to 0.58 |
12 | 0.29 | 0.45 | 0.31 to 0.57 |
13 | 0.35 | 0.61 | 0.51 to 0.69 |
14 | 0.37 | 0.57 | 0.45 to 0.66 |
Sociodemographic and Children Emotional Characteristics as Predictors of ODS parent-teacher Disagreement
Table 4 shows the results of the multilevel random intercept models with polynomial coefficients for the set of sociodemographic and children emotional characteristics analyzed. ICC derived from the variation captured by the random intercept ranged from 0.22 to 0.39, which are high enough to confirm the adequacy of the multilevel approach (Snijders & Bosker, 2019). Ethnicity, CBCL depression and CBCL anxiety did not significantly moderated the association between teachers’ and parents’ ODS scores.
Table 4
Second and third-order polynomial regression results
ICC | Number of follow-up measures | Parent ×Predictor B (95% CI) | LR p | Parent2 ×Predictor B (95% CI) | Parent ×Predictor2 B (95% CI) | |
|---|---|---|---|---|---|---|
Ethnicity Other vs. White | 0.39 | -0.27 (-0.66 to 0.11) | 0.908 | 0.05 (-0.20 to 0.30) | NA | |
Sex Male vs. Female | 0.38 | 0.25 (0.01 to 0.49)* | 0.966 | -0.02 (-0.17 to 0.14) | NA | |
Socioeconomical Status M + M-H vs. L + L-M H vs. L + L-M | 0.39 | 12 (ages 3 to 14) | -0.09 (-0.6 to 0.19) -0.01 (-0.35 to 0.33) | 0.094 | -0.08 (-0.29 to 0.12) -0.22 (-0.43 to-0.01)* | NA NA |
CAS – Total score | 0.22 | 12 (ages 3 to 14) | 0.12 (0.04 to 0.20)**a | < 0.001 | -0.03 (-0.09 to 0.03) | 0.01 (-0.04 to 0.05) |
CBCL – Depression | 0.40 | 12 (ages 3 to 14) | 0.03 (-0.07 to 0.13) | 0.631 | 0.03 (-0.04 to 0.10) | -0.01 (-0.04 to 0.03) |
CBCL – Anxiety | 0.40 | 12 (ages 3 to 14) | 0.06 (-0.04 to 0.15) | 0.173 | -0.04 (-0.11 to 0.03) | -0.01 (-0.05 to 0.04) |
ICU – Total score | 0.35 | 12 (ages 3 to 14) | 0.15 (0.06 to 0.24)**a | < 0.001 | 0.01 (-0.06 to 0.08) | -0.08 (-0.13 to -0.02)* |
CBQ – Surgency | 0.40 | 4 (ages 3, 4, 5, 7) | 0.13 (-0.01 to 0.27) | 0.755 | 0.03 (-0.07 to 0.14) | -0.03 (-0.13 to 0.06) |
CBQ – Negative Affect | 0.43 | 4 (ages 3, 4, 5, 7) | 0.05 (-0.10 to 0.19) | 0.660 | 0.02 (-0.10 to 0.13) | 0.05 (-0.04 to 0.14) |
CBQ – Effortful Control | 0.43 | 4 (ages 3, 4, 5, 7) | -0.21 (-0.35 to -0.08)**a | 0.920 | 0.01 (-0.11 to 0.14) | 0.01 (-0.09 to 0.10) |
ARI | 0.20 | 2 (ages 7, 11) | 0.12 (-0.03 to 0.28) | 0.474 | 0.03 (-0.11 to 0.17) | -0.09 (-0.18 to -0.01)* |
The predictor sex showed a second-order statistically significant interaction. The value 0.25 of the PZ interaction means that ODS teacher’s rating prediction increases 0.25 in males with respect females for each additional point in ODS parent’s rating. Consequently, Fig. 2 reflects those differences between sexes in disagreement arose as ODS scores increased, and that disagreement is higher in females because its predicted T score is farther from the main diagonal line. Socioeconomical status showed a third-order significant interaction in the coefficient comparing high vs. low + low/medium groups. Figure 3 indicates that the highest disagreement appeared in children with low + low/medium socio-economic status and with low ODS symptomatology, but also in participants from high social status with high ODS symptomatology. The reported level of aggressivity also affected disagreement as suggested by the second-order significant interaction found between the children aggression scale total score and the parents’ rating. Figure 4 shows that disagreement is maximum for very low or extremely low values of aggression, when the predicted ODS teachers’ score is hardly the same independently of the ODS parents’ score, and that disagreement decrease as aggression score increases. Additionally, among children with low to extremely low CAS scores (z-values from − 1 to -3) teachers reported much lower ODS symptoms than parents. Significant second and third-order interactions were found for the score of callousness, which means that both linear and non-linear patterns of disagreement were involved. The value 0.15 of the second-order interaction PZ suggest a higher predicted teacher’s ODS score as parent’s ODS score increases, but the negative sign of the value − 0.08 for the third-order interaction PZ2 indicates a reduction in the predicted change in function of the square’s parent’s score. The combination of both interactions results in Fig. 5, showing a similar but sharpened effect than for aggressiveness, with even a negative slope relating teacher and parent ODS score when callousness is extremely low or very low. As z-values of callousness goes from − 1 to 3 disagreement decreases, with the lowest discordance occurring for children who scored at the mean in ICU symptoms. Specially, among children with ICU scores above the mean (z-values from 1 to 3), teachers reported higher ODS than parents when ODS is low or medium. Of the three temperament variables analyzed, one statistically significant second-order interaction was observed for Effortful Control. Figure 6 revealed that parent-teacher disagreement was higher in children with low to extremely low ODS scores (z-values from − 3 to -2) or high to extremely high ODS scores (z-values from 2 to 3). In both conditions, the strongest disagreement occurred for children with high scores on CBQ- EF (z-values from 1 to 3), and disagreement diminished as this score decreased as well. The smallest parent-teacher incongruence was observed among children who scored near the mean for ODS. Finally, the variable irritability as measured by the ARI, showed a significant and statistically third-order interaction (irritability was measured only twice at 7 and 11 ages). As seen in Fig. 7, disagreement between parents and teachers is high for any irritability score and increased as irritability decreased. Interestingly, for children scoring below the mean values of irritability, teachers ODS scoring was clearly lower than parents.
Fig. 2
Predicted values of teachers ODS score as a function of parents ODS score and sex using the second order interaction model
×
![]()
Fig. 3
Predicted values of teachers ODS score as a function of parents ODS score and socioeconomical status using the third order interaction model
×
![]()
Fig. 4
Predicted values of teachers ODS score as a function of parents ODS score and Children aggression scale total score (CAS) using the second order interaction model
×
![]()
Fig. 5
Predicted values of teachers ODS score as a function of parents ODS score and Inventory of Callous-Unemotional Traits (ICU) score using the third order interaction model
×
![]()
Fig. 6
Predicted values of teachers ODS score as a function of parents ODS score and Children behaviour questionnaire – Effortful control (CBQ-EFF) using the second order interaction model
×
![]()
Fig. 7
Predicted values of teachers ODS score as a function of parents ODS score and Affective reactivity index (ARI) using the third order interaction model
×
![]()
From the previous seven statistically significant interactions only the second-order interactions of aggressiveness, callousness and effortful control with parents’ ODS score kept significant after false discovery rate correction was applied.
Discussion
This study aims to explore the disagreement between teachers and parents when rating ODS in a sample of children from 3 to 14 years old and investigate how emotional child characteristics (aggression, depression, anxiety and callousness) may predict it. These findings contribute to the existing literature on multi-informant discrepancies on child psychopathology assessment and provide insights on how child characteristics, such as emotional traits, may influence this disagreement.
Firstly, results showed that parents ratings on ODS were higher than teachers’ for almost all ages, except for ages 13 and 14 where parents scored slightly lower than teachers. Findings are consistent with Antrop et al. (2002) who stated that parents’ ratings were higher than teachers for ODS reports. Parents scoring higher than teachers can be attributed to several factors. For example, the child may exhibit different behavior across settings (e.g., group setting with same age peers vs. family setting alone which rarely includes various of same-age peers) and in school age children, teachers are less likely than parents to report more problematic behavior, except when children exhibit co-morbid peer or behavioral issues (Strickland et al., 2012). Moreover, parents and teachers have different expectations for appropriate conduct and symptomatology which may impact the degree of tolerance they have for specific behaviors. Also, informant characteristics such as parental stress or parental psychopathology may influence how they rate the children (Major et al., 2015; Mitsis et al., 2000; Van der Oord et al., 2006). Finally, ODD may occur in only home setting, which may cause that the disorder manifests exclusively with parents which lead to higher ratings (Newcorn et al., 2015).
Second, the informant disagreement on ODS was studied. Findings in the present study suggest that absolute agreement and correlation for ODS scale went from poor to moderate and low to medium respectively as the child grew older. These results are consistent with the findings of multiple studies in which the agreement between parent and teacher ratings of ODS is poor to moderate (Lavigne et al., 2015; Munkvold et al., 2009; Petric & Szamosközi, 2018). On the other hand, previous studies showed that cross-informant correspondence was greater for younger children 6–11 years old compared to adolescents 12 years onwards. Interestingly, there is an increasing pattern of parent-teacher agreement from age 3 to 14, which slightly differs from what De los Reyes & Kazdin, (2005) affirmed about agreement during adolescence. These results may be explained by the fact that developmentally changes in early childhood are more prominent causing problematic behaviors to be less stable, in contrast to behaviors in middle childhood which tend to become more consistent or diminish over time (Nagin & Tremblay, 1999). Also, given the developmental course of ODD, with very common transient oppositional behavior during the preschool years, parents may not be aware of what is normally expected compared to teachers. According to Newcorn et al. (2015) oppositional behaviors are typically noted by age 8 and the symptomatology increases with age making it easier for parents and teachers to observe the same symptoms. Additionally, because of the disruptive nature ODD has, teachers are more likely to discuss symptoms with parents which may lead to more consistent ratings across time (Achenbach et al., 1987; Narad et al., 2015).
Lastly, polynomial regression analyses identified two sociodemographic variables and four emotional variables related to parents and teachers’ disagreement on ratings of children´s ODS. The two sociodemographic variables related to parent-teacher disagreement were sex and SES. Sex revealed a statistically significant interaction, with females showing a higher disagreement level between parents and teachers specially for severe ODS.
Findings in previous literature examining gender influence on informant discrepancies showed inconsistent conclusions and suggest that gender does not account significantly for the variance shown by multiple informants ratings (De Los Reyes & Kazdin, 2005; Petric & Szamosközi, 2018; Rescorla et al., 2014). Our study showed that the disagreement related to sex increased as the ODS ratings were more severe, and that these differences were higher for girls than for boys. These may be explained by gender expectations of how a boy or girl has to behave in different settings. Boys are usually ascribed with higher behavioral problems (Collishaw et al., 2009; Rescorla et al., 2014) and the higher disagreement in girls as ODS ratings rose is atypical and may conflict with these expectations leading to more discrepancies.
Although SES have yielded inconclusive results, with some studies associating lower SES with higher multi-informants discrepancies and other studies not finding any association at all (Lavigne et al., 2015; Lawson et al., 2017; Stone et al., 2013), we observed significant interactions between SES and the ODS score for parent-teacher disagreement. In the present study, the highest levels of disagreement were observed in two groups: children with low ODS symptomatology from low or low/medium SES backgrounds but also for children belonging to high SES with severe ODS symptomatology. Perhaps these discrepancies can be explained by the parent´s level of education and the associated expectations, which impact the amount of information accessible about the child development leading to higher or lower problematic behavior recognition. Furthermore, parent´s tolerance in the low income group can also be diminished by the increased stress they experience, which may lead to poor family functioning and increased family conflict which can make them susceptible to report inaccurate ODS levels even when this symptomatology is low (Collishaw et al., 2009; Stone et al., 2013),
Finally, regarding to the emotional variables that could predict ODS discrepancies between parents and teachers, variables such as aggression, callousness and one dimension of temperament; EF, were found statistically significant. It was hypothesized that the emotional variables that elicit externalizing behavior would be significant predictors of the inter-rater discrepancies. The present findings are consistent with this hypothesis. Lower levels of disagreement for variables related to externalizing behaviors (e.g., high levels of aggression, callousness and low EF) while higher levels of disagreement for variables associated to internalizing problems (e.g.,irritability) were observed. These results build upon the previous literature review, ratifying that symptomatology which elicits externalizing behavior results in less disagreement due to observable behavior that can be objectively measured and clearly defined (Achenbach et al., 1987; De Los Reyes & Kazdin, 2005; Duhig, Kimberly Renk et al., 2000).
Higher discrepancies were observed for children with very low or extremely low ratings of aggression, and teachers were more likely than parents to provide lower ODS ratings. Aggression is categorized as an externalizing and observable problem, which results in greater levels of correspondence between informants (De Los Reyes & Kazdin, 2005). Parents reporting greater ODS scores may be attributed to the association between child aggression and higher levels of family disfunction, which in turn amplifies parent-reported ODS scores (Connolly & Vance, 2010).
Similar results are observed for callous-unemotional traits, with low levels of agreement between parent and teacher ratings of ODS when callousness scores are very low. These traits have been strongly associated to severe and persistent conduct problems in early childhood and adolescence (Longman et al., 2016) and higher levels of aggression and externalizing symptomatology. These results explain why the disagreement diminishes when there is a greater presence of more observable traits in children. Teachers reporting higher ODS scores than parents when ICU scores are above the mean can be attributed to situation specificity. Callous-unemotional traits have been associated to greater peer conflicts which are more observable in a classroom setting, enhancing the likelihood of higher ODS scores in teachers (Fanti et al., 2017).
Finally, parent-teacher disagreement for ODS was high for all irritability scores. Studies have shown that irritability can be also associated to internalizing psychopathology and has been conceptualized as a symptom dimension of ODD (Burke et al., 2010; Evans et al., 2016). As mentioned before, internalizing symptoms have shown less correspondence suggesting that problems that are less observable create more discrepancies (De Los Reyes & Kazdin, 2005; Duhig et al., 2000a). Furthermore, these findings are also consistent with results by Ezpeleta and Penelo (2015) who observed a higher disagreement between parents and teachers for irritability and mood related symptoms perhaps of greater difficulties when it comes to identify mood states than behavioral consequences.
Previous studies have examined the relationship between child temperament with parent-teacher discrepancies and found no relationship (Chen et al., 2017; Lavigne et al., 2015). Results of the present study provide evidence that high levels of effortful control in children were significantly associated with more multi-informant discrepancies. Higher scores on EF scales suggest that children have the ability to self-regulate and modulate their emotions and behavior which reduce the probability of externalizing problems (Eisenberg et al., 2009). Moreover, children who can manage better their own behavior and emotions may display different defiance at home or school which may contribute to the disagreement between parents and teachers.
Although previous literature on multi-informant agreement has found moderate to high levels of agreement when evaluating disruptive disorders, there are only a few studies that examine these discrepancies in ODD (Petric & Szamosközi, 2018). Regarding these studies investigating multi-informant discrepancies in ODD, only a few studies analyze how child characteristics can moderate the interrater disagreement with incongruent findings (Gomez, 2014; Korsch & Petermann, 2014; Lavigne et al., 2015; Munkvold et al., 2009; Strickland et al., 2012).
These results build upon the previous literature review ratifying that symptomatology which elicits externalizing behavior results in less disagreement due to observable behavior that can be objectively measured and clearly defined(Achenbach et al., 1987; De Los Reyes & Kazdin, 2005; Duhig et al., 2000a) The novel aspect of our study is how it expands the literature on source discrepancies specifically for ODD symptoms in a large international sample, instead of general children psychopathology, by analyzing the relationship between child emotional characteristics with parent and teacher ODS disagreement finding a significant relationship for aggression, CU traits, effortful control and irritability.
Furthermore, previous studies regarding discrepancies in ODS have not studied how emotional characteristics in children influence the degree of disagreement between parents and teachers. Also, analyzing a large, international community over a long period of time allows to investigate how discrepancies may change as children grow. Finally, to our knowledge, this is the first study using a polynomial regression approach to study interrater agreement on ODS. This allows to overcome mathematical limitations by using difference scores and having more precise examinations (e.g., linear or non-linear patterns of agreement) of how the disagreement between two informants is moderated by a variable of interest (Saffer et al., 2021).
All these findings have important significant implications when assessing ODS in children using parents and teachers as information sources. It stresses the importance of considering multiple informant´s perspectives but also using a comprehensive assessment approach when evaluating behavior problems that acknowledge the influence of children´s emotional characteristics. However, reports by parents and teachers should not be considered equivalent. Based on prior research the fact that there is a low level of agreement, is not only explained by the informant´s bias- including informant´s mental health such as depression, high levels of stress and knowledge about the child- or the context (e.g., school or home) (Chi & Hinshaw, 2002; De Los Reyes & Kazdin, 2005; Gross et al., 2004; Harvey et al., 2013). When evaluating young children, it is important to take in consideration which informant is best suited for assessing specific symptom dimensions-such as the emotional variables examined in this study, or for establishing a diagnosis- the age of the child (e.g., infants vs. school-aged children) due to the inaccurate descriptions that parents or teachers might give.
The current study has multiple strengths, such as a large community sample and the length of the study which allows the investigators to gather and compare multiple reports since children were at preschool throughout adolescence. Relatively few studies have attempted to address informant discrepancies longitudinally. Our study also has some limitations, like not taking in consideration informant´s characteristics (e.g. psychopathology, stress) that could influence their evaluation of the child´s behavior. Another limitation was that some findings are based on measures answered by parents or teachers, which introduces the possibility of biases and subjective interpretations. Literature shows that teacher´s ratings may vary if the child shows high pre-academic skills and there is evidence that parents reports usually are higher than teachers when rating ODS (Antrop et al., 2002; Harvey et al., 2013). Further research should focus on other than emotional child characteristics such as cognitive or social attributes.
In conclusion, present results show informant discrepancies on ODS were more prominent for emotional characteristics eliciting mood related symptoms as opposed to those linked to observable behaviors. Therefore: (a) it stresses the disagreement between parents and teachers on ODS ratings, (b) child emotional characteristics should always be taken in consideration, and (c) emotional traits that trigger externalizing and internalizing behaviors in ODS are predictive of the disagreement between parents and teachers.
Declarations
Ethical Approval
Approval of the ethics committee of the Autonomous University of Barcelona.
Informed Consent
Informed was obtained from legal guardians, mothers/fathers and teachers.
Consent for Publication
All authors have manifested their consent for the manuscript publication.
Competing Interests
The authors declare that they have no conflicts of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
