Part V: creating the EQ-i 2.0 and EQ 360 2.0

Australian Norms

EQ 360 2.0 Australian Standardization

Standardization

NORMATIVE SAMPLE

Collection of data for the Australian EQ 360 2.0 took place from October to December of 2011. Respondents (“the raters”) were required to rate an individual (“the ratee”) on the EQ 360 2.0 and provide demographic information about both themselves and the ratees. In order to create a representative normative sample, specific demographics regarding the ratees, guided by recent Census information, were considered during the data collection process. Information was collected on each ratee’s gender, age group, employment status, and geographic region. This information about the ratee (i.e., the person being rated) was provided by the rater (i.e., the person completing the assessment). Information about the type (i.e., manager, work peer, direct report, or family/friend) and strength of the rater-ratee relationship (i.e., how long they have known each other, how well they know each other, and how often they interact with each other) was also collected.

Rater Description: The sample of 1,720 raters (i.e., the participants providing the ratings) was 44.2% male with a mean age of 43.6 years (SD = 13.9 years). Approximately half of the raters (49.1%) had a Bachelor’s degree or equivalent/higher qualifications, and the remaining raters indicated having either an Advanced Diploma (10.1%), a Certificate (I/II/III/IV;16.6%), or Year 12 or below (24.2%). The majority of raters knew the ratee for longer than one year (86.1%; see Table D.12), and more than half stated that they knew the ratee well or very well (55.2%; see Table D.13). More than half (55.5%) of the raters indicated that they interacted with the ratee very often (daily or almost daily; see Table D.14).

Ratee Sample: TThe normative sample was stratified to match the Census as closely as possible based on the ratees’ demographic characteristics. Table D.15 shows the ratee age group by gender distribution. The ratee sample included an equal ratio of males to females, stratified across four rater types: direct report, manager, work peer, and friend/family member (see Table D.16). Representation by geographic region is shown in Table D.17, with percentages meeting Census targets within 2%.

NORMING PROCEDURES

Similar to the EQ-i 2.0, the first step in the EQ 360 2.0 norming procedure was to determine if any demographic trends existed in the Australian normative data. Large differences in scores between rater types (i.e., managers, work peers, direct reports, friends/family members) would suggest a need to create an option for separate rater type norm groups, while a lack of such differences would suggest a need to create a single norm option with the rater types combined. Similarly, large differences in scores between male and female ratees, or across various ratee age groups, would suggest a need to create an option for separate gender- or age-based norm groups. Conversely, a lack of such differences may dictate the use of a single norm group with genders and age groups combined.

A series of analyses of variance (ANOVA; for the Total EI score) and multivariate analyses of variance (MANOVA; for the composites and subscales) were used to examine the relationships between EQ 360 2.0 scores and rater type, ratee gender, and ratee age. To better control for Type I errors that might occur with multiple analyses, a more conservative criterion of p < .01 was used for all F-tests. Results at the multivariate level revealed significant effects of gender, age, and rater type for the composites and subscales (see Table D.18). Interaction effects at the multivariate level were not significant. Given these results, the univariate main effects of gender, age, and rater type are described in detail next.

Gender and Age Effects. Overall (and similar to what was seen with the North American EQ 360 2.0 data), gender and age effects were less pronounced in the EQ 360 2.0 normative sample than they were in the EQ-i 2.0 normative sample (see Table D.19 for effect sizes and Table D.20 and Table D.21 for descriptive statistics and significance test results).

At the Total EI and composite level, only the Interpersonal composite reached even a small effect size for gender. At the subscale level, Emotional Self-Awareness, Emotional Expression, and Empathy showed small effects, with females scoring higher than males.

With respect to age, several scales reached a small effect size (the Self-Expression and Decision Making composites, and the Assertiveness, Independence, Social Responsibility, Problem Solving, Impulse Control, and Flexibility subscales). For most scales, the effect was attributable to lower scores for the 18–29 year-old group compared to the other age groups, but for Flexibility it was the oldest group (60+) who were rated lower than the younger groups.

Rater Type Effects. No meaningful effect size was observed across rater types for the Total EI score. At the composite and subscale level, a number of scales showed differences between rater types; however, all effect sizes were small. Direct Reports provided the highest ratings and Managers gave the lowest ratings compared to other rater groups on the Self-Expression composite and the Self-Regard, Assertiveness, Independence, and Problem Solving subscales. Both Managers and Direct Reports provided lower ratings on Interpersonal Relationships compared to Family/Friends. For Flexibility, Direct Reports and Work Peers gave higher ratings compared to Managers and Family/Friends. See Table D.19 for effect sizes and Table D.22 for descriptive statistics and significance test results.

Norm Groups and Norm Construction. Overall, the effect sizes (i.e., d and partial η2 values) found in the normative data suggest negligible or very small effects of ratee gender, ratee age, and rater type. The scarcity of meaningful effects suggested that it was not necessary to create specific gender-, age-, or rater type-based norms for the Australian EQ 360 2.0, and therefore only Overall norms were developed. These norms were created using the same procedure as the EQ-i 2.0 norms. Standard scores for all scales were computed with a mean of 100 and standard deviation of 15. Results revealed that skewness and kurtosis values were close to 0 (skewness values ranged from 0.68 to 0.01; kurtosis values ranged from -0.44 to 0.38), and an examination of the scale histograms did not reveal any significant departures from a bell-shaped (Gaussian) curve. A histogram for the EQ 360 2.0 Total EI score is provided in Figure D.2. The other composite scales and subscales show comparable distributions. Therefore, artificial transformation of scores to fit normal distributions was deemed unnecessary.

Comparison of Australian Norms to North American Norms. The Australian sample was compared against the North American normative sample by computing standard scores for the EQ 360 2.0 scales with the North American EQ 360 2.0 norms, and comparing these scores against a mean of 100 (see Table D.23). Although the effects for all scales did reach statistical significance due to the large sample size, most differences were within 3 standard score points, and only one scale (Emotional Expression) reached a meaningful effect size; for all other scales, Cohen’s d values were below 0.20. The only scale on which the Australian sample scored higher than the North American sample was Assertiveness, but the effect size was negligible (d = 0.06). These results suggest that the norms for Australia and North America are comparable, but the use of specific norms for Australia may offer more precision for use of the instrument in that country.

Internal Consistency

Internal consistency conveys the degree to which a set of items are associated with one another. High levels of internal consistency suggest that the items are measuring a single, cohesive construct. Internal consistency is typically measured using Cronbach’s alpha (Cronbach, 1951), which ranges from 0.0 to 1.0 with higher values reflecting higher internal consistency.

Cronbach’s alpha values for the Australian EQ 360 2.0 normative sample are displayed in Table D.24. Similar to results found with the North American EQ 360 2.0 data, most values ranged from good to excellent (.83 to .98) across the Total EI, composite, and subscale scores, with only Assertiveness reaching a lower, but still acceptable, level (.77).

Factorial Validity

EXPLORATORY FACTOR ANALYSIS

Exploratory factor analysis (EFA) was used to determine whether the subscales established with the North American EQ 360 2.0 normative data empirically emerge from the Australian normative dataset. Five EFAs were conducted, analyzing the items within each composite scale separately. In each EFA, a three-factor solution was forced to examine whether the items corresponding to each subscale within the composite also loaded together in the Australian normative data. Principal axis factoring extraction was used, with direct oblimin (i.e., oblique) rotation, as the factors within each composite are expected to correlate with each other. Reverse scoring was applied to relevant items prior to the analysis. Factor loadings were considered significant if they reached at least ± .30, and an item was defined as cross-loading if it was significant on more than one factor and had loadings within .10 of each other on these factors.

For the Self-Perception Composite EFA, items for the three subscales (Self-Regard, Self-Actualization, and Emotional Self-Awareness) loaded significantly onto their respective factors as expected by the established factor structure, with no cross-loading items.

For the Self-Expression Composite EFA, all Emotional Expression, Assertiveness, and Independence items loaded onto their respective factors, with the exception of one Assertiveness item that fell just below the factor loading cutoff of ± .30, and one Assertiveness item that cross-loaded with Emotional Expression.

For the Interpersonal Composite EFA, all Empathy items loaded together with no cross-loadings. For Social Responsibility, all of the items loaded significantly except for one item that was cross-loaded with Interpersonal Relationships and one item that loaded onto Empathy. For Interpersonal Relationships, all of the items loaded significantly with the exception of one item that loaded onto Empathy, and three items cross-loaded with Empathy.

For the Decision Making Composite EFA, all Problem Solving, Reality Testing, and Impulse Control items loaded onto their respective factors, with only one Problem Solving item cross-loading with Impulse Control.

For the Stress Management Composite EFA, the Flexibility and Optimism items loaded onto their respective factors with only one cross-loading item. For Stress Tolerance, all of the items loaded significantly although three items cross-loaded with Optimism and two items cross-loaded with Flexibility.

To summarize, for the most part the EFAs generated solutions that strongly correspond to the established EQ 360 2.0 factor structure, with the items for each subscale empirically grouping together onto the expected factors.

CORRELATIONS AMONG EQ 360 2.0 COMPOSITE SCALES AND SUBSCALES

Correlations among the EQ 360 2.0 composite scales and subscales were examined in the Australian normative sample to determine if the pattern of results found in the Australian EQ-i 2.0 normative data could be replicated. Table D.25 (composite scales) and Table D.26 (subscales) display these correlations. Overall, in the Australian data, the correlations were stronger in the EQ 360 2.0 data than in the EQ-i 2.0 data. Composite scale correlations ranged from r = .61 (Self-Expression/Interpersonal) to r = .86 (Decision Making/Stress Management), with an average correlation of r = .76. For the subscales, correlations ranged from r = .19 (Assertiveness/Impulse Control) to r = .86 (Optimism/Happiness), with an average correlation of r = .57.

These results support the notion that a single, underlying dimension is being represented in the EQ 360 2.0, yet the values are not overly high and there is enough variation in the correlations to provide clear evidence of the multidimensional nature of the assessment, and support for the existence of composite scales and subscales.