Part V: creating the EQ-i 2.0 and EQ 360 2.0

Standardization, Reliability, and Validity

EQ 360 2.0 Validity

EQ 360 2.0 validity analyses were performed to ensure that the validity of the observer-rated version of the EQ-i 2.0 is comparable to the self-report version. These analyses are summarized in the following section. Specifically,

Following these results, analyses examining potential bias of raters in relation to the race/ethnicity of ratees will be illustrated, (that is, the degree to which White and non-White observers rate Black, Hispanic/Latino, and White ratees similarly).
Based on these results, practical implications of the validity analyses include the use of EQ 360 2.0 assessments to inform decisions in occupational settings such as initial hiring or subsequent promotion, or acceptance into academic institutions, or the added value of using multiple sources (i.e., self-report, observer reports) in gathering this information about individuals, with the confidence that ratings are not affected by demographic variables.

CORRELATIONS AMONG
EQ 360 2.0 COMPOSITE SCALES & SUBSCALES

Correlations among the EQ 360 2.0 composite scales and subscales were examined in the normative sample to determine if the pattern of results found in the EQ-i 2.0 normative sample data would be replicated. Tables A.52 (Composite Scales) and A.53 (Subscales) display these correlations. These correlations were strong, and in most cases stronger than in the EQ-i 2.0 normative sample. Composite scale correlations ranged from r = .64 (Self-Expression/Interpersonal) to r = .86 (Decision Making/Stress Management). For the most part, subscale correlations were especially strong within the same composite, as expected (see shaded cells in Table A.53). Each of these values exceeded a medium effect size and most exceeded a large effect size, ranging from r = .37 (Emotional Expression/Independence) to r = .81 (Empathy/Interpersonal Relationships). These results suggest the composite scales and subscales share a relevant underlying factor (i.e., emotional intelligence), similar to that found in the EQ-i 2.0.

Relationship between the EQ-i 360 2.0 and the EQ-i 2.0

Associations between self- and other-ratings serve as another source of a scale’s validity. The EQ 360 2.0 can be validated by finding a strong level of agreement between EQ 360 2.0 and EQ-i 2.0 scores. In order to assess the association between self and observer ratings, a sample of 108 participants rated themselves on the EQ-i 2.0 and were also rated by a rater on the EQ 360 2.0. Most of the EQ 360 2.0 ratings were provided by family members or spouses (65.7%) or a friend (21.3%). Most (97.2%) of the raters knew the person they were rating for at least one year, with 81.5% of the raters stating that they knew the person they were rating “Very Well” (on a 4-point scale ranging from “Not Very Well” to “Very Well”) and 76.9% stating that they interacted with the person “Very Often” in the past month (on a 4-point scale ranging from “Occasionally” to “Very Often”). A breakdown of the sample (i.e., those who provided self-ratings and were also rated by others) is presented in Table A.54.

CORRELATIONS BETWEEN
EQ 360 2.0 aND EQ-i 2.0 (Self-Other Agreement)

The correlation between the EQ-i 2.0 and EQ 360 2.0 Total EI scores was  r = .60, p < .01 (Table A.55). Correlations for the composite scales and subscales were all significant at p < .01, and almost every correlation reached the criterion for a large effect size. Specifically, the correlations ranged from r = .44 (Stress Tolerance) to r = .72 (Happiness). These results suggest that self-other agreement for the EQ 360 2.0 (and EQ-i 2.0) is strong. Moreover, this pattern suggests that EI as measured by the EQ-i 2.0 and EQ 360 2.0 is a robust trait that is evaluated similarly via self-report and external observers. However, these correlations are not high enough to suggest redundancy; each measure is assessing unique information about the individual and both types of scores provide important information. Specifically, self-ratings will not always align with observer ratings.

COMPARING SCORES ON THE
EQ 360 2.0 aND EQ-i 2.0 (Self-Other Consistency)

To supplement the correlational results between the EQ 360 2.0 and EQ-i 2.0, standard scores were compared between the two measures. The correlations compare the rank order of individuals on the EQ 360 2.0 and EQ-i 2.0. That is, high correlations between the two measures suggest that individuals who are rated as high in EI by observers (EQ 360 2.0) also have high self-report (EQ-i 2.0) scores, and individuals with low EQ 360 2.0 ratings also have low EQ-i 2.0 scores. However, the EQ 360 2.0 and EQ-i 2.0 ratings themselves may be quite different on an absolute level. For example, scores on the EQ 360 2.0 may be dramatically and uniformly lower than EQ-i 2.0 ratings, but as long as the rank-order of the ratings remains similar across the two measures, the correlation between the two will be high. Examining the degree to which EQ 360 2.0 and EQ-i 2.0 standard scores differ will help determine the nature of the relationship between the EQ-i 2.0 and EQ 360 2.0.These analyses also summarize the consistency of scores between self- and other-ratings.

EQ 360 2.0 and EQ-i 2.0 standard scores were compared by calculating a difference score between the two measures, which consisted of subtracting each EQ 360 2.0 standard score from its corresponding EQ-i 2.0 standard score. Therefore, a positive difference represents higher EQ-i 2.0 scores relative to EQ 360 2.0 scores, and a negative difference represents higher EQ 360 2.0 scores relative to EQ-i 2.0 scores. Recall that the criterion for describing a meaningful difference between self- and other-ratings was determined to be 10 standard score points (see Planning the EQ 360 2.0 Assessment Process). Difference scores are displayed in Table A.56 to summarize the proportion of difference scores that fall above or below 10 standard score points. Just over half of EQ 360 2.0 and EQ-i 2.0 scores fell within 10 points of each other for the Total EI score and all composite scales and subscales. Overall, the results demonstrate a good degree of consistency between EQ 360 2.0 and EQ-i 2.0 scores; however, the fact that large differences are observed for close to half of the sample demonstrates the importance of collecting both self and observer ratings.

ASSOCIATIONS AMONG
EQ-i 2.0, EQ 360 2.0, aND SAS-SR

Emotional intelligence tends to show consistent associations with general adjustment. Social adjustment—as measured by the Social Adjustment Scale – Self-Report (SAS – SR, Weissman, 1999)—should therefore show strong associations with the EQ-i 2.0 and EQ 360 2.0. The SAS-SR is a 54-item self-report scale intended to measure “instrumental and expressive role performance” (p. 1) in six major areas of functioning: work (employed, homemaker, or student); social and leisure activities; relationships with extended family; role as a marital partner; parental role; and role within the family unit. Across these six role areas, SAS-SR questions cover four qualitative categories: performance at expected tasks; the amount of friction with people; finer aspects of interpersonal relations; and feelings and satisfactions. Items are rated on a five-point rating scale with higher scores reflecting higher levels of impairment.

The independent associations of the EQ-i 2.0 and EQ 360 2.0 to SAS-SR scores were examined through multiple regression analyses, to shed light on the unique contributions of observer EI ratings in predicting social adjustment over self-report ratings, and vice-versa. The demographic description of the participants in this sample is displayed in Table A.57, and Table A.58 displays the results of the analyses. Correlations with the SAS-SR for both the EQ-i 2.0 and EQ 360 2.0 were mostly strong and in the expected direction (correlations are negative because high SAS-SR scores reflect social maladjustment). Stepwise multiple regression analyses were then performed in two steps. In the first step, only the   EQ-i 2.0 scale was entered as a predictor, with SAS-SR scores as the outcome. In the second step, the EQ-i 2.0 scale and the EQ 360 2.0 scale were entered simultaneously. Therefore, it is possible to evaluate the independent associations of each scale with the SAS-SR. This analysis was conducted separately for each composite scale, subscale, and Total EI. For the Total EI score as well as most of the composite scales and subscales, both the EQ-i 2.0 and EQ 360 2.0 scales were independently related to the SAS-SR Total Score at the p < .05 significance level. In other words, self-report and observer ratings were each uniquely informative of SAS-SR scores.

A final set of statistics relevant to these analyses is the R2 change (Table A.58). This statistic communicates the amount of explanatory power the EQ 360 2.0 scale adds to the prediction of SAS-SR scores after accounting for its respective EQ-i 2.0 scale. In other words, the incremental validity of the EQ 360 2.0 scores can be quantified. The strongest effects were found for the Empathy and Reality Testing subscales and the Interpersonal composite. Overall, the pattern of results showed expected associations between EI and social adjustment for both the EQ-i 2.0 and the EQ 360 2.0. The EQ-i 2.0 and EQ 360 2.0 subscales and composite scales provided unique and incremental contributions towards social adjustment.

Examination of Potential Race/Ethnicity Effects in the Rater-Ratee Relationship in the EQ 360 2.0

Another important issue related to EQ 360 2.0 ratings is whether a race/ethnicity bias exists. That is, neither the race/ethnicity of the ratee nor the race/ethnicity of the rater should have an affect on EQ 360 2.0 scores. Analysis of covariance (ANCOVA) was used to examine these potential effects in the EQ 360 2.0 Total EI score, using rater race/ethnicity (White vs. non-White) and ratee race/ethnicity (Black vs. Hispanic/Latino vs. White) as independent variables, and ratee gender and age group as covariates. Two separate multivariate analyses of covariance (MANCOVAs) were used to examine the composite scales and subscales. Specifically, a significant or meaningful interaction between the two independent variables would provide evidence that raters’ race/ethnicity is influencing differences in ratings of White, Black and Hispanic/Latino ratees. Table A.59 demonstrates that this was not the case in the EQ 360 2.0 normative sample. The Wilk’s lambda values suggested that only a negligible amount of variance could be explained by the interaction between rater and rate race/ethnicity. The interaction terms were not significant at the p < .01 level, and none of the effect sizes met the minimum requirements for even a small effect size (i.e., η2 = .01). These results illustrate that raters did not show differences in their ratings based on the ethnicity of the ratees.

Validity Summary

Several validity analyses were conducted for the EQ 360 2.0. Support for the scale’s factor structure, as identified in the EQ-i 2.0, also emerged in the EQ 360 2.0. The validity of the EQ 360 2.0 was further supported through comparisons with the EQ-i 2.0 (self-other agreement and consistency) and a measure of social adjustment (unique and incremental validity relative to the EQ-i 2.0). There was no evidence of bias in relation to the race/ethnicity of the rater or the ratee. Overall, the analyses suggest the EQ 360 2.0 is a valid measure of EI.