Part V: creating the EQ-i 2.0 and EQ 360 2.0

Standardization, Reliability, and Validity

EQ-i 2.0 Validity

Reliability is necessary for, but does not ensure, validity. The validity of a test refers to whether the test measures what it claims to be measuring; in this case, does the EQ-i 2.0 measure emotional intelligence? The quality of inferences that can be made by the test’s scores, and the validity of an instrument like the EQ-i 2.0, rests upon the weight of accumulated evidence from a number of validity studies using various methodologies (Campbell & Fiske, 1959). Various types of validity were examined for the EQ-i 2.0. Specifically, how well does the EQ-i 2.0 measure the construct(s) it was designed to measure, how well are the claims regarding its use and applications supported by empirical evidence, and is the EQ-i 2.0 free of test-bias?

Evidence that the EQ-i 2.0 measures the constructs it was designed to measure include

In terms of the use and applications of the EQ-i 2.0, evidence is provided that the EQ-i 2.0 scores are related to external criteria, including expected differences between the following groups of individuals:

Following these analyses, results from an examination of potential bias across racial/ethnic groups will be presented. As a general psychological characteristic, EI is expected to be similar across racial/ethnic groups; group differences would indicate that the EQ-i 2.0 may be biased towards certain racial/ethnic groups.

Finally, the Validity scales were validated:

Content Validity

Content validity is achieved when an assessment shows adequate coverage of the content it is proposed to measure, based on the conceptual framework of the construct. Support for this type of validity is often provided through non-statistical methods (Jackson, 1971). For the EQ-i 2.0, content validity of the items was analyzed by mapping their relevance to the EI construct by content experts. The conceptual framework of the EQ-i 2.0 is highly similar to that of its predecessor, the EQ-i (see Bar-On, 2004). Content validity of the original EQ-i was established through the systematic method of item generation (see EQ-i 2.0 Stages of Development). Specifically, the essence of each of the factors relevant to EI was articulated through detailed definitions. Items were then developed to encompass these definitions. Content experts scrutinized these items for their relevance to EI and the factors with which they were associated. Any items deemed irrelevant to a particular factor were moved to a more relevant factor, or discarded if their relevance could not be established. Based on these procedures, the final form of the EQ-i 2.0 adequately satisfied the requirements of content and face validity (Anastasi, 1988).

Factor Structure

The conceptual framework of the EQ-i 2.0 can be considered hierarchical. As displayed in Figure 3.4, several correlated factors comprise EI. The 15 subscales are categorized into the five composite scales, which combine to form the overall EI factor (i.e., Total EI). Evidence for the existence and appropriateness of the proposed EQ-i 2.0 factor structure was examined in several ways:

For the EFA and CFA analyses, the normative sample was split equally into two demographically-matched subsamples (Table A.16) to provide independent replication of the factor structure. Correlations among the scales were computed on the entire normative sample.

EXPLORATORY FACTOR ANALYSES

The factor structure of the EQ-i 2.0 items was determined through a series of  exploratory factor analyses (EFAs). This analysis is exploratory, as the EQ-i 2.0 contains many new or revised items from the original EQ-i. Five EFAs were conducted on the exploratory subsample of the normative sample, analyzing the items within each composite scale separately. In each EFA, a three-factor solution was determined to be the most appropriate based on statistical (eigenvalues/scree plot) and non-statistical (interpretability) criteria. Principal axis factoring extraction was used because the goal of the analysis was to identify the underlying constructs expected to produce the EQ-i 2.0 scores. Direct oblimin (i.e., oblique) rotation was used because the factors were expected to correlate with each other, given that they all share a common underlying construct (i.e., the composite scale factor). Reverse-scoring was applied to relevant items prior to the analysis. Factor loadings were considered significant if they reached at least ± .300, and an item was defined as cross-loading if it was significant on more than one factor and had loadings within .100 of each other on these factors.

For the Self-Perception Composite EFA, the first factor contained eight items covering areas such as self-confidence, self-respect, and a generally positive self-image, matching the definition of the Self-Regard subscale. The second factor contained seven items covering awareness and understanding of one’s own emotions, matching the definition of the Emotional Self-Awareness subscale. The third factor contained nine items covering personal striving, ambition, and achievement, and matched the definition of the Self-Actualization subscale. Each item loaded significantly onto one factor and there were no cross-loadings.

For the Self-Expression Composite EFA, the first factor contained eight items covering areas such as autonomy and self-sufficiency, corresponding with the definition of the Independence subscale. The second factor contained eight items relating to one’s ability to describe, express, and share their emotions, matching the definition of the Emotional Expression subscale. The third subscale contained seven items referring to one’s tendencies towards being direct and “speaking one’s mind,” matching the definition of the Assertiveness subscale. Again, each item loaded significantly onto one factor and there were no cross-loadings.

In the Interpersonal Composite EFA, the first factor contained eight items covering areas such as sociability and friendliness, corresponding to the definition of the Interpersonal Relationships subscale. The second factor contained nine items referring to one’s awareness, receptiveness, and respectfulness towards the emotions of others, corresponding with the definition of the Empathy subscale. The third factor covered consciousness of social/global issues and one’s contributions towards addressing these issues, matching the definition of the Social Responsibility subscale. Each item loaded onto one factor with no cross-loadings.

The first factor emerging from the Decision Making Composite EFA included eight items referring to one’s emotional process when faced with problems, matching the definition of the Problem Solving subscale. The second factor contained eight items describing one’s general awareness and tendency to be objective and impartial, corresponding to the definition of the Reality Testing subscale. The third factor contained eight items covering one’s ability to combat impulses and temptations, matching the definition of the Impulse Control subscale. Each item loaded significantly onto one factor except for one item (I interrupt when others are speaking). This item was retained on the Impulse Control factor due to its theoretical relevance and the fact that it loaded more highly on the Impulse Control factor (.248) than on the other two factors (.034 and .004). No items cross-loaded across multiple factors.

Finally, the first factor generated from the Stress Management Composite EFA included eight items describing one’s positive outlook towards other people and the future in general, matching the definition of the Optimism subscale. The second factor contained eight items describing one’s ability to manage change and unpredictability, corresponding to the definition of the Flexibility subscale. The third factor contained eight items referring to one’s ability to endure and cope with high-pressure situations and matched the definition of the Stress Tolerance subscale. Each item loaded onto a single factor with no cross-loadings.

To summarize, the EFAs generated an easily interpretable set of fifteen factors from the EQ-i 2.0 items. In addition, the items empirically grouped into the factors outlined by the theoretical framework of the instrument.

CONFIRMATORY FACTOR ANALYSIS

Confirmatory factor analyses (CFAs) were conducted on the confirmatory subsample of the EQ-i 2.0 normative data. Six models were tested. The first, called the Overall Model, consisted of the five composite scales loading onto Total EI. The other five CFAs were conducted at the composite scale level, each with the three relevant subscales loading onto their respective composite scale. Results from these analyses provide further support for the theoretical factor structure of the         EQ-i 2.0, as well as the empirical results generated by the EFAs. Goodness of fit indices are displayed in Table A.17. Specifically, the Goodness of Fit Index (GFI; Jöreskog & Sörbom, 1986), Adjusted Goodness of Fit Index (AGFI; Jöreskog & Sörbom, 1986), Normed Fit Index (NFI; Bentler & Bonett, 1980), Non-Normed Fit Index (NNFI; Bentler & Bonett, 1980), Comparative Fit Index (CFI; Bentler, 1990), and Root Mean Square Error of Approximation (RMSEA; Steiger & Lind, 1980) were examined to evaluate the fit of the models. General guidelines for adequate model fit are values below .10 for the RMSEA and above .90 for the remaining fit indices. Values suggested adequate fit for the models, providing further support for the factor structure of the EQ-i 2.0 as outlined by theory and EFA results.

CORRELATIONS AMONG EQ-i 2.0 COMPOSITE SCALES AND SUBSCALES

After establishing the existence of the proposed subscales through EFA and obtaining further verification through CFA, correlations among the EQ-i 2.0 composite scales and subscales were examined to determine the degree of cohesiveness among them. It is expected that these correlations will be generally high, given that they are all measuring the same underlying construct—emotional intelligence—but they should not be so high as to indicate redundancy between the subscales. Tables A.18 (Composite Scales) and A.19 (Subscales) display these correlations observed in the EQ-i 2.0 normative sample. These correlations matched closely to hypotheses. Each composite scale correlation reached at least a large effect size, ranging from r = .50 (Interpersonal/Decision Making) to  r = .78 (Self-Perception/Stress Management). Subscale correlations were also of the expected magnitude. As highlighted in Table A.19, virtually all subscale correlations within a composite reached at least a medium effect size and over half reached at least a large effect size, ranging from  r = .27 (Reality Testing/Impulse Control) to r = .70 (Self-Regard/Self-Actualization). These results support the notion that a single, underlying dimension is being represented in the EQ-i 2.0, yet there is clear evidence for the multidimensional nature of the assessment.

Relationship of the EQ-i 2.0 to Other Measures

The validity of the EQ-i 2.0 was further evaluated by examining its overlap with other psychological measures. These analyses inform whether the EQ-i 2.0 assesses the construct it is intended to assess—namely, emotional intelligence. Specifically, correlations between the EQ-i 2.0 and these other measures are examined. The expected pattern of correlations (magnitude, direction) depends on the relevance and degree of overlap among the psychological constructs these measures are proposed to assess. Validity is supported by the extent to which the actual correlations correspond with these theoretical associations. For example, is the EQ-i 2.0 related to other measures of emotional intelligence but unrelated to measures of different content, like critical thinking? For the EQ-i 2.0, these external psychological measures included

Demographic characteristics of the samples used in these analyses are displayed in Table A.20.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND THE ORIGINAL EQ-i

The original EQ-i (Bar-On, 2004) is a 133-item self-report measure designed to assess emotional intelligence (EI). Bar-On defines EI as “an array of non-cognitive capabilities, competencies, and skills that influence one’s ability to succeed in coping with environmental demands and pressures” (p. 14). Other key features of the EQ-i’s conceptual framework are that it is multifactorial and relates to potential for performance rather than performance itself (i.e., the potential to succeed rather than success itself). It is process-oriented rather than outcome-oriented, unlike ability-based conceptualizations of EI such as that measured by the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT; Mayer et al., 2002). The 15 EI constructs assessed by the EQ-i are measured by the EQ-i subscales, organized as outlined in Figure 3.1. This figure also illustrates the slight changes made to the organization of the subscales in the EQ-i 2.0 revision. Because the models are very similar, it is expected that correlations between the EQ-i and EQ-i 2.0 will be large.

Correlations between the EQ-i 2.0 and the original EQ-i are displayed in Table A.21. Correlations between overlapping subscales are presented in this table (i.e., the correlation between the two Interpersonal Relationships subscales, the correlation between the two Flexibility subscales, and so on). Despite the updates made to the EQ-i 2.0 from the original EQ-i, correlations between the subscales on the two measures were high. Correlations between the Total EI score of each measure was  r = .90, suggesting a high degree of overlap between the two versions at the overall EI level. The majority of the subscale correlations between the EQ-i and EQ-i 2.0 were high. This trend was particularly evident for subscales that underwent very minor changes between the two versions of the scale, with correlations ranging from r = .65 to r = .88 (see shaded cells in Table A.21). Conversely, for subscales that underwent more dramatic changes between versions (see The EQ-i 2.0 Framework; unshaded cells in Table A.21), correlations were still high but lower, as expected, than those found for the unchanged subscales. These correlations ranged from r = .49 to r = .57. One exception was the correlation between Emotional Expression and the original EQ-i Emotional Self-Awareness subscale (r = .84). Many of the Emotional Self-Awareness items from the original EQ-i were incorporated into the new Emotional Expression subscale, which explains this high correlation. Overall, correlations between the EQ-i 2.0 and the original EQ-i reflect not only the stability of the construct measured by the two assessments, but also the changes in item content made in the recent update to the EQ-i 2.0.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND SSI

The Social Skills Inventory (SSI; Riggio & Carney, 2003) is a 90-item self-report measure designed to assess “basic social communication skills” (Riggio & Carney, p. 5). The scale captures the expression, sensitivity, and control (i.e., regulation) aspects of communication in two domains: emotional (nonverbal) and social (verbal). This conceptualization results in six subscales: Emotional Expression, Emotional Sensitivity, Emotional Control, Social Expression, Social Sensitivity, and Social Control. Along with a Total SSI Score, these subscales are collapsed into Total Emotional and Social Scales as well as Total Expression, Control, and Sensitivity Scales. It is expected that the EQ-i 2.0 will correlate more strongly with the Emotional Scales than the Social Scales. For instance, although the SSI authors admit that the tool does not fully capture all aspects of EI, they specifically state that the SSI Emotional subscales “can be used as indicators of emotional intelligence” and “could be used as an alternative to existing self-report measures of emotional intelligence” (Riggio & Carney, p. 6). These statements summarize the relevance of the SSI to the EQ-i 2.0.

Emotional intelligence is proposed to be relevant to social skills as measured by the SSI, especially the SSI Emotional Subscales. Therefore, most correlations between the EQ-i 2.0 and the SSI should be strong and positive. As illustrated in Table A.22, the EQ-i 2.0 Total EI score correlated positively with the SSI Total Score (r = .54; p < .01). With the exception of Impulse Control (r = -.13; p = .19), each of the EQ-i 2.0 composite scales and subscales correlated significantly with the SSI Total Score. The EQ-i 2.0 Total EI score also showed significant positive correlations with most of the SSI Subscales. Exceptions were a non-significant correlation with the Total Sensitivity Scale (r = .08; p = .43) and a significant negative correlation with the Social Sensitivity Scale (r = -.35; p < .01). Riggio and Carney describe the Social Sensitivity Scale as measuring “an individual’s sensitivity to and understanding of the norms governing appropriate social behavior” (p. 5), and also suggest that extremely high scores may indicate self-consciousness and general insecurity, which could explain the negative correlation with the EQ-i 2.0. The nonsignificant correlation between the EQ-i 2.0 and the Total Sensitivity Scale is likely due to the former’s positive correlation with the Emotional Sensitivity Scale and negative correlation with the Social Sensitivity Scale cancelling each other out. These results provide support for the idea that higher EI is related to stronger social skills.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND NEO-FFI

The NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, 1992) is a shortened, 60-item version of the NEO Personality Inventory-Revised (NEO-PI-R; Costa & McCrae, 1992). This scale measures what are considered to be the five fundamental personality traits according to the Five-Factor Model of personality: Neuroticism, Conscientiousness, Openness to Experience, Agreeableness, and Extraversion. Conceptually, the Big Five and emotional intelligence share certain features, such as positive correlations with occupational performance (e.g., Mount & Barrick, 1998). In a recent meta-analysis, Van Rooy and Viswesvaran (2004) found significant positive correlations between EI and each of the Big Five factors, ranging from r = .23 (Agreeableness, Openness to Experience) to r = .34 (Extraversion). Therefore, it is expected that the EQ-i 2.0 will correlate positively with the NEO-FFI subscales (except for Neuroticism, where negative correlations are expected).

The EQ-i 2.0 Total EI score correlated significantly with the NEO-FFI Neuroticism (note that the negative correlations are in the expected direction), Extraversion, Agreeableness, and Conscientiousness subscales, but not with Openness to Experience (Table A.23). The pattern of correlations suggests that EI is distinct from personality. The correlations also support the hypotheses that high levels of Neuroticism may inhibit EI development, whereas high levels of Extraversion and Conscientiousness may help facilitate EI skills.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND MSCEIT

The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT; Mayer et al., 2002) is a 141-item ability-based measure of EI. The MSCEIT is ability-based in that it considers EI as a skill and measures it through items that require the respondent to demonstrate their level of EI by performing various relevant tasks and solving emotional problems. The scale is a “test” in the true sense of the word in that items are considered to have correct and incorrect responses, based on either general consensus or expert consensus. This feature defines the MSCEIT as outcome-oriented as opposed to process-oriented as in the EQ-i 2.0 (see Bar-On, 2004). The distinction between ability-based measures like the MSCEIT and trait-based measures like the EQ-i 2.0 has long been established by researchers (Austin, 2010; Brackett & Mayer, 2003; Mayer et al., 2002; O’Boyle et al., 2011; Van Rooy & Viswesvaran, 2004). In line with this research it is expected that the EQ-i 2.0 and the MSCEIT would not be strongly correlated.

In the MSCEIT, the Total Emotional Intelligence Quotient (EIQ) score comprises eight subscales called Tasks: Faces, Pictures, Sensations, Facilitation, Changes, Blends, Emotion Management, and Emotional Relations. These Tasks are categorized into four Branch scores: Perceiving Emotions (Faces, Pictures), Facilitating Thought (Sensation, Facilitation), Understanding Emotions (Changes, Blends), and Managing Emotions (Emotion Management, Emotional Relations). The Branch scores are further categorized into two Area scores: Experiential (Perceiving Emotions, Facilitating Thought) and Strategic (Understanding Emotions, Managing Emotions). Task, Branch, and Area scores provide different levels of scope of the individual’s EI abilities. Further description of these scales is provided by Mayer et al. (2002).

The EQ-i 2.0 conceptualizes EI as a trait-based measure, whereas the MSCEIT assesses EI as an ability-based measure. For these reasons, it is expected that the relationship between MSCEIT and EQ-i 2.0 scores will be moderate, at best. In our sample, these correlations are displayed in Tables A.24 and A.25. Indeed, the correlation between the EQ-i 2.0 Total EI score and the MSCEIT Total EI Score was r = .12 (p = .22). The vast majority of MSCEIT Task Scores, Branch Scores, and Area Scores were not significantly correlated with EQ-i 2.0 composite or subscale scores. This pattern of results demonstrates that the EQ-i 2.0 measures trait-based EI that does not overlap with EI as measured by the MSCEIT. On a larger, conceptual level, these results support the idea that trait-based EI and ability-based EI are independent constructs.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND WATSON-GLASER II

The Watson-Glaser II Critical Thinking Appraisal (Watson & Glaser, 2009) is “(d)esigned to measure important abilities and skills involved in critical thinking” (p. 1). Along with a Total Score, the three subscales of the Watson-Glaser II are Recognize Assumptions, Evaluate Arguments, and Draw Conclusions. Individuals must evaluate a series of exercises that cover these areas, such as rating the degree of truth or falsity of various inferences. Validity of the scale is demonstrated through correlations with similar ability measures such as the Wechsler Adult Intelligence Scales-IV (WAIS-IV; Wechsler, 2008) and occupational and academic success. However, emotional intelligence is considered to be independent of more traditional cognitive abilities such as critical thinking. Therefore, it is presumed that the EQ-i 2.0 will be largely uncorrelated with the Watson-Glaser II.

The correlations between the Watson-Glaser II and the EQ-i 2.0 are displayed in Table A.26. The correlation between the Total Scores of the EQ-i 2.0 and the Watson-Glaser II was not statistically significant (r = -.05; p = .62). Regarding the subscales of the Watson-Glaser II, the EQ-i 2.0 Total EI score was also uncorrelated with the Recognize Assumptions (r = .03; p = .76) and Draw Conclusions (r = .02, p = .84) subscales, but was significantly negatively correlated with the Evaluate Arguments subscale (r = -.25, p < .01). Watson and Glaser (2009) state that lower scores on the Evaluate Arguments subscale may be found in individuals who allow high levels of emotion to “cloud objectivity and the ability to accurately evaluate arguments” (p. 3) . This trend was also found for the EQ-i 2.0 composite scales and subscales. That is, the majority of EQ-i 2.0 composite scales and subscales were uncorrelated with the Watson-Glaser II Total Score and the Recognize Assumptions and Draw Conclusions subscales, but were negatively correlated with the Evaluate Arguments subscale. These results provide support for the independence of EI and cognitive intelligence; however, they also demonstrate the impact of emotional skills on the ability to effectively evaluate arguments.
In summary, strong evidence has been provided that the EQ-i 2.0 measures the constructs it was designed to measure. It shows strong correlations with measures of similar constructs, and little or no correlation with measures of divergent constructs.

Group Differences in EQ-i 2.0 Scores

The validity of the EQ-i 2.0 was further evaluated by examining scores among groups that are expected to show differences in EI. Specifically, validity was assessed by examining (a) corporate job success: corporate leaders vs. the general population; (b) academic achievement: individuals with higher (i.e., post-graduate) compared to lower (high school or less) levels of education; and (c) clinical group differences: individuals with a diagnosed psychological illness vs. a demographically matched control group.

RALATIONSHIP BETWEEN
EQ-i 2.0 aND CORPORATE JOB SUCCESS

Occupational success is one highly relevant, consistent, and important outcome of high emotional intelligence. Therefore, the EQ-i 2.0 would be validated by showing higher scores among individuals who have excelled in their profession. To test this hypothesis, EQ-i 2.0 scores were compared between 221 corporate leaders (i.e., CEOs and other C-level leaders, senior executives, directors, and managers; see Table A.27 for demographics) and the normative sample. Results are displayed in Table A.28. Relative to the normative mean score of 100, leaders scored consistently higher on the EQ-i 2.0 Total EI score and all composite scales and subscales. Leaders produced a mean score of 112.2 (SD = 11.7) on the Total EI score, which represents a large difference when compared to the normative average (d = 0.82). Mean scores on the composite scales and subscales ranged from 104.2 (SD = 14.0; Impulse Control) to 113.1 (SD = 10.4; Self-Actualization), with most differences representing medium or large effects. These results indicate that occupational success, measured by one’s advancement into a senior-level corporate position, is related to greater emotional intelligence.

RALATIONSHIP BETWEEN
EQ-i 2.0 SCORES AND ACADEMIC ACHIEVEMENT

Academic achievement is another key outcome related to emotional intelligence. Therefore, EQ-i 2.0 scores are expected to be higher among individuals who have achieved higher levels of accomplishment in educational pursuits. EQ-i 2.0 scores were compared between individuals in the normative sample who achieved a post-graduate degree (e.g., M.A., Ph.D., MBA; N = 402) and those who progressed no farther than a high school degree (N = 1,451). Comparisons were conducted using analysis of covariance (ANCOVA) for the EQ-i 2.0 Total EI score and two multivariate analyses of covariance (MANCOVA) for the subscales and composite scales, with age group, gender, and race/ethnicity (White vs. non-White) included as covariates. As illustrated in Table A.29, higher  Total EI scores were found for post-secondary graduates (M = 103.2, SD = 14.8) relative to high school graduates (M = 98.1, SD = 15.5), showing a small-to-moderate effect size (d = 0.33). Post-graduates also scored higher on most of the composite scales and subscales. Scores that showed at least a small difference ranged from Problem Solving (d = 0.22) to Self-Actualization (d = 0.54), with most differences being found in the Decision Making and Stress Management areas. Overall, the results demonstrate that greater academic achievement tends to be associated with higher EI.

CLINICAL GROUP DIFFERENCES IN EQ-i 2.0 SCORES

Because emotional intelligence is associated with daily functioning, it is presumed to be lower in individuals with various psychiatric or psychological conditions. Based on this assumption, it follows that differences in EQ-i 2.0 scores should be found between clinical and non-clinical (i.e., general population) individuals. Analysis of covariance (ANCOVA) was used to compare mean EQ-i 2.0 Total EI scores across three groups: general population (or the control group taken from the normative sample), individuals diagnosed as depressed or dysthymic, and individuals with another clinical diagnosis (see Table A.30 for demographic characteristics of the samples). Age, gender, and race/ethnicity were included in the analysis as covariates. This procedure was repeated using two separate multivariate analyses of covariance (MANCOVAs) to examine the EQ-i 2.0 composite scales and subscales. Results (Table A.31) demonstrated a significant effect of clinical status for the EQ-i 2.0 Total EI score (F [2, 221] = 7.89, p < .01). Specifically, the mean score for the general population group was higher than for each of the depressed/dysthymic and other clinical groups, and each difference approached or exceeded a medium effect size (Cohen’s d = 0.57 and 0.45, respectively). This trend was replicated for all of the composite scales except Interpersonal. The Interpersonal composite was not significant because there were no differences for Empathy or Social Responsibility. The Interpersonal Relationships subscale, however, did show significant differences between groups. At the subscale level, the effect of the general population group scoring higher than the clinical group was found for more than half of the subscales. The subscales that showed the largest differences were those that would be expected on a conceptual level. For example, the largest differences between the general population and depressed/dysthymic groups were found for the Self-Regard and Happiness subscales. These results provide further evidence for the validity of the EQ-i 2.0.

Comparisons among Racial/Ethnic Groups

The examination of potential racial or ethnic bias is always of critical importance in the development of an assessment. Specifically, it is vital to ensure that assessment scores do not show large differences among racial/ethnic groups when they are not expected to. For the EQ-i 2.0, test bias was examined by comparing mean scores across various racial/ethnic groups (White, Black, Hispanic/Latino) in the normative sample. Analysis of covariance (ANCOVA) was used to compare these three groups on the EQ-i 2.0 Total EI score, using gender, age group, and education level as covariates (in order to control for the effects of these demographic variables). Two separate multivariate analyses of covariance (MANCOVAs) were used to examine the composite scales and subscales. Results demonstrated that the effect of race/ethnicity on EQ-i 2.0 scores was statistically significant; however, the effect sizes were in the small-to-medium range. In scales that did show differences, Black and Hispanic/Latino respondents generally showed slightly higher scores than White respondents, though these differences were typically only a few standard score points in magnitude (see Table A.32). These results demonstrate that the EQ-i 2.0 does not show strong differences among racial/ethnic groups and there was no evidence of test bias toward minority groups.

Validity Scale Validation

Dishonest or exaggerated responses are always a concern with self-report instruments. Insincere responses undermine the veracity of an individual’s scores on a self-report assessment, which can have significant consequences. The original EQ-i included three scales—Positive Impression, Negative Impression, and Inconsistency Index—to detect illegitimate response styles. These scales were also developed for the  EQ-i 2.0. Validity studies were conducted in order to determine if the Validity scales do, in fact, capture positive, negative, and/or inconsistent response styles.

POSITIVE IMPRESSION & NEGATIVE IMPRESSION SCALES

Positive and negative impression styles might be used intentionally or unintentionally when responding to a self-report questionnaire. Positive impression occurs when an individual responds to questions in such a way as to make themselves appear in an unrealistically positive light. The reasons behind positive impression include self-deception, lack of insight, an unwillingness to face one’s limitations, or various needs such as social conformity, approval, self-protection, or avoidance of criticism (Crowne & Marlowe, 1964; Edwards, 1966; Frederiksen, 1965; Jackson, 1974). An attempt to make a positive impression is more apt to occur when, for example, one is applying for a job, seeking admission to an educational institution, or simply trying to impress someone. Conversely, a negative impression style consists of making oneself appear in an unrealistically negative light. Elevated negative impression scores can be caused by low self-esteem, or various needs such as attention, sympathy, or help in resolving personal problems (Crowne & Marlowe, 1964; Frederiksen, 1965; Jackson, 1974). To detect these response styles in the EQ-i 2.0, Positive Impression (PI) and Negative Impression (NI) scales were developed (see EQ-i 2.0 Stages of Development). PI and NI scales are traditionally validated by examining the scores of individuals who are motivated to present themselves favorably or unfavorably, respectively, to individuals who respond to the assessment under standard instructions without such motivation. The PI and NI scales were validated using a standard between-subjects simulation study conducted during the norming phase of development. Participants were given instructions designed to elicit either a positive or negative response style while completing the EQ-i 2.0. Instructions designed to elicit a positive response style asked the respondent to imagine they are completing the EQ-i 2.0 as part of an application for a highly desirable job, and must therefore try to give themselves the highest scores possible. Instructions for the negative response style condition asked respondents to imagine they are completing the EQ-i 2.0 as part of a mandatory application for a mentoring program that he or she does not want to participate in, and must therefore try to give themselves the lowest scores possible in order to be selected out of the program. Two demographically-matched control groups who completed the EQ-i 2.0 under standard instructions were selected for comparison with the two simulation groups. Presumably, PI and NI scores would be higher in individuals who were instructed to simulate positive or negative response styles, respectively, than those who responded under standard conditions.

Results from the simulation studies are displayed in Table A.33. As expected, PI scores from the positive response style group were significantly higher than those in the control group. The difference between the two groups is quantified as a medium-to-large effect size. Similarly, NI scores from the negative response style group were significantly higher than those in the control group. This difference exceeded the standard guideline for a large effect size. These results provide support for the validity of the PI and NI scales.

INCONSISTENCY INDEX

Inconsistent responding occurs when a respondent rates similar items in dissimilar or opposite ways. For example, a respondent who endorses (i.e., responds “Always/Almost Always”) both of the items “I like parties” and “I don’t like parties” would be responding inconsistently. Like positive impression and negative impression styles, inconsistent responding might occur intentionally or unintentionally. Various reasons for inconsistent responding include deliberate sabotage or noncompliance, fatigue, incomprehension of the items or instructions, inattention, disinterest, and a lack of motivation.

To detect inconsistent responding in the EQ-i 2.0, an Inconsistency Index (IncX) was developed. This scale is comprised of 10 pairs of highly related items, which should elicit similar responses within each pair of items. If the respondent provides very different ratings to several pairs of items that should be rated similarly, then inconsistent responding may be suspected (see The EQ-i 2.0 Framework). Traditionally, inconsistency scales are validated by comparing scores generated from individuals who respond to assessment items randomly, to individuals who respond under standard conditions. These random protocols can be generated by human respondents or computer programs. A computer program (IBM SPSS Statistics 19.0.0, 2010) was used to generate a data set of 4,000 random EQ-i 2.0 response sets to compare to the normative data. Evidence of the validity of the IncX would be demonstrated if the cutoff identified a large proportion of the random response sets, and if IncX scores were higher, on average, than those in a control sample. Furthermore, these results would provide independent validation of the choice of cutoff to be used to identify scores as potentially invalid that was developed from the normative sample. Table A.34 illustrates the proportion of response sets at each IncX raw score. Results demonstrated that a score of 3, which identified only 3.5% of the normative sample as potentially inconsistent, identified 93.3% of the random response sets as potentially inconsistent. Furthermore, mean IncX scores were dramatically higher in the random sample than in the normative sample (d = 3.36; Table A.34), a difference that easily exceeded the criteria for a large effect size. These results demonstrate a high degree of predictive validity for the EQ-i 2.0 IncX.

Validity Summary

Several analyses were conducted to examine the validity of the EQ-i 2.0. Content validity analyses suggest that all relevant facets of the Bar-On conceptualization of EI are being captured by the EQ-i 2.0. Exploratory factor analyses suggested that this overarching single factor (EI) may be represented by 15 correlated subscales, which in turn may be combined into five correlated composite scales (i.e., a 1-5-15 Factor Model of Emotional Intelligence). This factor structure was corroborated through confirmatory factor analyses. Correlations among the composite scales and subscales provide support for the unidimensionality of the EQ-i 2.0. Validity was supported by expected correlations with the original EQ-i and measures of social skills and general personality, as well as a lack of correlation with measures of ability-based EI and cognitive intelligence. Further validity evidence was provided by expected group differences with regard to occupational success, academic achievement, and psychological adjustment. Comparisons among racial/ethnic groups in the normative sample provided no evidence for racial/ethnic bias against minority groups in the EQ-i 2.0. The validity scales (Positive Impression, Negative Impression, and Inconsistency Index) were validated through expected differences in scores between known invalid responses and those of control groups. Overall, the analyses suggest that the EQ-i 2.0 is a valid measure of EI.