Australian Norms
The release of the Australian Norms provides users with the ability to score their clients against data collected from Australia. This chapter is designed to provide normative and psychometric information particular to populations in Australia. The EQ-i 2.0 and EQ 360 2.0 assessments remain unchanged, but additional normative samples are now available (the original North American normative samples are described in detail in Standardization, Reliability, and Validity).
| This chapter describes the development of the EQ-i 2.0 and EQ 360 2.0 Australian Norm samples. For information on the EQ-i 2.0, including administration, interpretation, and development of the North American Norms, please refer to Parts I–V of the EQ-i 2.0 User’s Handbook. |
The first section of this chapter is devoted to the development, standardization, reliability, and validity of the EQ-i 2.0 Australian Norms, followed by a section describing these same properties for the EQ 360 2.0 Australian Norms.
EQ-i 2.0 data were collected from 1,250 Australians, evenly proportioned by gender within 5 age intervals from across the country. Similar to what was found in samples from other countries (i.e., Canada, United States, United Kingdom, and Ireland) small to moderate effects were found for gender (women scored higher than men on Empathy, Emotional Self-Awareness, and Emotional Expression; men scored higher than women on Independence, Stress Tolerance, and Independence) and small to moderate effects were also found for age (scores increased with age), leading to the creation of both overall norms, as well as age by gender norms. No meaningful differences were found between how Australians and North Americans score on the EQ-i 2.0. Finally, EQ-i 2.0 scores were found to be highly reliable in the Australian sample, and the factor structure that was developed in North America was replicated with the Australian sample.
EQ 360 2.0 data were collected from 1,720 Australian raters. Similar to what was found in samples from other countries (i.e., Canada, United States, United Kingdom, and Ireland) negligible effects were found for rater-type (i.e., Manager, Direct Report, Work Peer, and Friend/Family Member), age, and gender of the ratee. As a result, one overall norm group was created that collapses across all of these variables. Similar to what was found with the EQ-i 2.0, no meaningful differences were found between how Australian raters and North American raters rate individuals using the EQ 360 2.0. Finally, EQ 360 2.0 scores were found to be reliable in the Australian rater sample, and the Australian sample data was consistent with the factor structure that was developed in North America.
This section describes the psychometric properties of the Australian norms for the EQ-i 2.0, including standardization, reliability, and validity.
| All tables and figures representing detailed depictions of the analyses described in this chapter are available in Appendix D. |
EQ-i 2.0 Australian Standardization
Standardization
NORMATIVE SAMPLE
Normative data for the Australian General Population Norm sample (N = 1,250) were collected from October to December of 2011.
The demographic composition of the normative sample is shown in Tables D.1-D.3. The General Population Norm sample was collected across five age ranges, evenly proportioned by gender within each age interval. The normative sample included respondents from a variety of geographic regions and education levels. In terms of employment status, 82.0% of the sample were employed or self-employed, 3.8% were unemployed, 12.1% were retired, and 2.2% indicated “other.”
NORMING PROCEDURES
The first step in the preparation of the Australian norms was to determine if any age or gender trends existed in the data. Large differences in scores between men and women, or across various age groups, would suggest a need to create an option for separate gender- or age-based norm groups. Conversely, a lack of such differences may dictate the use of a single norm group with genders and age groups combined.
A series of analyses of covariance (ANCOVA; for the Total EI score) and multivariate analyses of covariance (MANCOVA; for the composites and subscales) were used to examine the relationships between gender and age with EQ-i 2.0 scores. Education level was used as a covariate in order to control for the effect of this demographic variable. To better control for Type I errors that might occur with multiple analyses, a more conservative criterion of p < .01 was used for all F-tests.
The Wilks’ lambda statistic generated from these analyses ranges from 0.00 to 1.00 and conveys the proportion of variance that is not explained by the effect (gender, age, or the interaction between gender and age) in the multivariate analyses. These values were all close to 1.00, suggesting that only a small amount of variance could be explained by the effects of these variables. However, F-tests revealed significant effects of gender, age, and the interaction between gender and age (see Table D.4). Given these results, the univariate effects are described in detail below.
Gender Effects. Results of the gender analyses showed that males and females did not differ significantly on the Total EI score, indicating that overall emotional intelligence as measured by the EQ-i 2.0 is about the same for males and females. However, small to medium effects were seen on a number of scales (see Table D.5 for effect sizes and Table D.6 for descriptive statistics and significance test results). The largest gender difference seen in the Australian sample was on Empathy, with women scoring higher than men with a moderate effect size (d = -0.60). Smaller differences were found with women scoring higher than men on the Interpersonal Composite (d = -0.36), Emotional Self-Awareness (d = -0.35), and Emotional Expression (d = -0.38). Men scored higher than women with small effect sizes on Independence (d = 0.27), Problem Solving (d = 0.31) and Stress Tolerance (d = 0.34).
Age Effects. Significant effects were found across age groups for the Total EI score, as well as all composite scales and subscales, mostly with small to medium effect sizes (Independence being the only scale to reach a large effect size). See Table D.5 for effect sizes and Table D.7 for descriptive statistics and significance test results. Though patterns across the age groups differed across scales, scores were generally lowest for the 18–29 year-old age group and highest for the 60+ age group, showing a clear pattern of increases in EI with age. Differences between the youngest and oldest age groups ranged from 5.7 (Emotional Expression) to 16.2 (Independence) standard score points.
Gender × Age Interaction. None of the scales showed a significant interaction effect between age and gender, and only one scale (the Decision Making Composite) reached the minimum partial η2 criterion for a small effect size. Overall, age effects were largely consistent within males and females, and gender effects were largely consistent across age groups.
Norm Groups and Norm Construction. Overall, the age and gender analyses revealed significant effects that were small to medium in size. Therefore, specific Age and Gender General Population Norms, as well as Overall General Population Norms (i.e., collapsed across ages and genders), were both developed. Results revealed that skewness and kurtosis values were close to 0 (skewness values ranged from -0.76 to -0.07; kurtosis values ranged from -0.32 to 0.80), and an examination of the scale histograms did not reveal any significant departures from a bell-shaped (Gaussian) curve. A histogram for the EQ-i 2.0 Australian Total EI score is provided in Figure D.1. This figure displays the shape of the distribution for the Total EI score. The shapes of the distributions for the other scales are comparable to this one. Since the data closely match a bell-shaped curve, artificial transformation of scores was deemed unnecessary. Actual construction of the norms was conducted in the same manner as the North American Norms, including the use of statistical smoothing (see Standardization, Reliability, and Validity for more information on the construction of the North American General Population Norms).
Comparison of Australian General Population Norms to North American General Population Norms. The Australian sample was compared against the North American normative sample by computing standard scores for the EQ-i 2.0 scales with the North American norms, and comparing these scores against a mean of 100. No difference between the Australian sample and the North American norms was observed on the Total EI score. Although for a number of scales the effects did reach statistical significance because of the large sample size, all differences were within 3 standard score points, and none reached a meaningful effect size (all Cohen’s d values were below 0.20). Results are presented in Table D.8.
Internal Consistency
Internal consistency, a measure of reliability, conveys the degree to which a set of items are associated with one another. High levels of internal consistency suggest that the set of items are measuring a single, cohesive construct. Internal consistency is typically measured using Cronbach’s alpha (Cronbach, 1951). Cronbach’s alpha ranges from 0.0 to 1.0 and is a function of both the interrelatedness of the items in a test or scale and the length of the test (John & Benet-Martinez, 2000). Higher values reflect higher internal consistency.
Cronbach’s alpha values for the EQ-i 2.0 scales for the Australian normative sample are presented in Table D.9. Although there is no universal criterion for a good alpha level, informal cutoffs for evaluating alpha are typically .90 is “excellent,” .80 is “good,” .70 is “acceptable,” and lower than .70 is “questionable.” Most of the values found in Table D.9 demonstrate excellent or good reliability, and these values are particularly favorable given the small number of items included in most subscales. For the overall sample, the alpha value of the Total EI scale was .97, values for the composite scales ranged from .87 to .92, and values for the subscales ranged from .77 to .91. Similar patterns were seen across the age and gender normative groups, including a Total EI alpha value of .96 or higher for each normative group. The high level of internal consistency found in the Total EI score supports the idea that the EQ-i 2.0 items are measuring a single cohesive construct, namely emotional intelligence. The same can be said of the individual components of emotional intelligence that make up the EQ-i 2.0 (i.e., the composite scales and subscales).
Factor Validity
EXPLORATORY FACTOR ANALYSIS
Exploratory factor analysis (EFA) was used to determine whether the subscales established with the North American EQ-i 2.0 normative data empirically emerge from the Australian normative dataset. Five EFAs were conducted, analyzing the items within each composite scale separately. In each EFA, a three-factor solution was forced to examine whether the items corresponding to each subscale within the composite loaded together in the Australian normative data. As with the North American normative data, principal axis factoring extraction was used, with direct oblimin (i.e., oblique) rotation, as the factors within each composite are expected to correlate with each other. Reverse scoring was applied to relevant items prior to the analysis. Factor loadings were considered significant if they reached at least ± .30, and an item was defined as cross-loading if it was significant on more than one factor and had loadings within .10 of each other on these factors.
For the Self-Perception Composite EFA, items for the Self-Regard, Emotional Self-Awareness, and Self-Actualization subscales loaded together as expected by the established factor structure (i.e., all items loaded significantly onto their respective factors, with no cross-loadings).
For the Self-Expression Composite EFA, all but one item loaded onto their respective factors for the Independence, Emotional Expression, and Assertiveness subscales. The factor loading for one Assertiveness item was .25, just below the cutoff.
For the Interpersonal Composite EFA, items for the Interpersonal Relationships, Empathy, and Social Responsibility subscales all loaded onto their respective factors, with only one Interpersonal Relationships item cross-loading with Empathy.
For the Decision Making Composite EFA, Problem Solving, Impulse Control, and Reality Testing items loaded onto their respective factors with no cross-loadings, with the exception of one Impulse Control item that loaded onto Problem Solving.
For the Stress Management Composite EFA, Optimism, Flexibility, and Stress Tolerance items loaded onto their respective factors with no cross-loadings, except for a single Optimism item that cross-loaded with Flexibility.
To summarize, the EFAs generated solutions that strongly correspond to the established EQ-i 2.0 factor structure, with the items for each subscale empirically grouping together onto the expected factors. Of the 118 items entered in the EFAs, only two items showed cross-loadings, one item fell slightly below the .30 criteria, and one item loaded onto a factor other than the expected one.
CORRELATIONS AMONG EQ-i 2.0 COMPOSITE SCALES AND SUBSCALES
Correlations among the EQ-i 2.0 composite scales and subscales were examined to determine the degree of cohesiveness among them. It is expected that these correlations will generally be high, given that they are all measuring the same underlying construct of emotional intelligence, but they should not be so high as to indicate redundancy between the scales. Correlations observed in the Australian normative sample are presented in Tables D.10 (composite scales) and D.11 (subscales). These results are similar to what is seen with the North American normative sample.
The composite scale correlations ranged from r = .46 (Interpersonal/Decision Making) to r = .76 (Self-Perception/Stress Management), with an average correlation of r = .66. For the subscales, correlations ranged from r = .10 (Assertiveness/Impulse Control) to r = .80 (Happiness with both Self-Regard and Optimism), with an average correlation of r = .45. These results support the notion that a single, underlying dimension is being represented in the EQ-i 2.0, yet the values are not overly high and there is enough variation in the correlations to provide clear evidence for the multidimensional nature of the assessment, and support the existence of composite scales and subscales.
The release of the Australian Norms for the EQ 360 2.0 provides users with the ability to score their clients against data collected from this country. Research has found that self and others’ ratings are influenced by cultural characteristics (Atwater et al., 2009). Further, the discrepancy in self-observer ratings can be predicted by cultural values (Eckert et al., 2010). Use of the Australian General Population norms for the EQ 360 allows consultants to obtain 360 feedback scores that are more relevant to Australian cultural characteristics. (The North American normative sample for the EQ 360 is described in detail in Standardization, Reliability, and Validity.)
The following sections describe the psychometric properties of the Australian norms for the EQ 360 2.0, including standardization, reliability, and validity.
| All tables and figures representing detailed depictions of the analyses described in this chapter are available in Appendix D. |
Standardization
NORMATIVE SAMPLE
Collection of data for the Australian EQ 360 2.0 took place from October to December of 2011. Respondents (“the raters”) were required to rate an individual (“the ratee”) on the EQ 360 2.0 and provide demographic information about both themselves and the ratees. In order to create a representative normative sample, specific demographics regarding the ratees, guided by recent Census information, were considered during the data collection process. Information was collected on each ratee’s gender, age group, employment status, and geographic region. This information about the ratee (i.e., the person being rated) was provided by the rater (i.e., the person completing the assessment). Information about the type (i.e., manager, work peer, direct report, or family/friend) and strength of the rater-ratee relationship (i.e., how long they have known each other, how well they know each other, and how often they interact with each other) was also collected.
Rater Description: The sample of 1,720 raters (i.e., the participants providing the ratings) was 44.2% male with a mean age of 43.6 years (SD = 13.9 years). Approximately half of the raters (49.1%) had a Bachelor’s degree or equivalent/higher qualifications, and the remaining raters indicated having either an Advanced Diploma (10.1%), a Certificate (I/II/III/IV;16.6%), or Year 12 or below (24.2%). The majority of raters knew the ratee for longer than one year (86.1%; see Table D.12), and more than half stated that they knew the ratee well or very well (55.2%; see Table D.13). More than half (55.5%) of the raters indicated that they interacted with the ratee very often (daily or almost daily; see Table D.14).
Ratee Sample: TThe normative sample was stratified to match the Census as closely as possible based on the ratees’ demographic characteristics. Table D.15 shows the ratee age group by gender distribution. The ratee sample included an equal ratio of males to females, stratified across four rater types: direct report, manager, work peer, and friend/family member (see Table D.16). Representation by geographic region is shown in Table D.17, with percentages meeting Census targets within 2%.
NORMING PROCEDURES
Similar to the EQ-i 2.0, the first step in the EQ 360 2.0 norming procedure was to determine if any demographic trends existed in the Australian normative data. Large differences in scores between rater types (i.e., managers, work peers, direct reports, friends/family members) would suggest a need to create an option for separate rater type norm groups, while a lack of such differences would suggest a need to create a single norm option with the rater types combined. Similarly, large differences in scores between male and female ratees, or across various ratee age groups, would suggest a need to create an option for separate gender- or age-based norm groups. Conversely, a lack of such differences may dictate the use of a single norm group with genders and age groups combined.
A series of analyses of variance (ANOVA; for the Total EI score) and multivariate analyses of variance (MANOVA; for the composites and subscales) were used to examine the relationships between EQ 360 2.0 scores and rater type, ratee gender, and ratee age. To better control for Type I errors that might occur with multiple analyses, a more conservative criterion of p < .01 was used for all F-tests. Results at the multivariate level revealed significant effects of gender, age, and rater type for the composites and subscales (see Table D.18). Interaction effects at the multivariate level were not significant. Given these results, the univariate main effects of gender, age, and rater type are described in detail next.
Gender and Age Effects. Overall (and similar to what was seen with the North American EQ 360 2.0 data), gender and age effects were less pronounced in the EQ 360 2.0 normative sample than they were in the EQ-i 2.0 normative sample (see Table D.19 for effect sizes and Table D.20 and Table D.21 for descriptive statistics and significance test results).
At the Total EI and composite level, only the Interpersonal composite reached even a small effect size for gender. At the subscale level, Emotional Self-Awareness, Emotional Expression, and Empathy showed small effects, with females scoring higher than males.
With respect to age, several scales reached a small effect size (the Self-Expression and Decision Making composites, and the Assertiveness, Independence, Social Responsibility, Problem Solving, Impulse Control, and Flexibility subscales). For most scales, the effect was attributable to lower scores for the 18–29 year-old group compared to the other age groups, but for Flexibility it was the oldest group (60+) who were rated lower than the younger groups.
Rater Type Effects. No meaningful effect size was observed across rater types for the Total EI score. At the composite and subscale level, a number of scales showed differences between rater types; however, all effect sizes were small. Direct Reports provided the highest ratings and Managers gave the lowest ratings compared to other rater groups on the Self-Expression composite and the Self-Regard, Assertiveness, Independence, and Problem Solving subscales. Both Managers and Direct Reports provided lower ratings on Interpersonal Relationships compared to Family/Friends. For Flexibility, Direct Reports and Work Peers gave higher ratings compared to Managers and Family/Friends. See Table D.19 for effect sizes and Table D.22 for descriptive statistics and significance test results.
Norm Groups and Norm Construction. Overall, the effect sizes (i.e., d and partial η2 values) found in the normative data suggest negligible or very small effects of ratee gender, ratee age, and rater type. The scarcity of meaningful effects suggested that it was not necessary to create specific gender-, age-, or rater type-based norms for the Australian EQ 360 2.0, and therefore only Overall norms were developed. These norms were created using the same procedure as the EQ-i 2.0 norms. Standard scores for all scales were computed with a mean of 100 and standard deviation of 15. Results revealed that skewness and kurtosis values were close to 0 (skewness values ranged from 0.68 to 0.01; kurtosis values ranged from -0.44 to 0.38), and an examination of the scale histograms did not reveal any significant departures from a bell-shaped (Gaussian) curve. A histogram for the EQ 360 2.0 Total EI score is provided in Figure D.2. The other composite scales and subscales show comparable distributions. Therefore, artificial transformation of scores to fit normal distributions was deemed unnecessary.
Comparison of Australian Norms to North American Norms. The Australian sample was compared against the North American normative sample by computing standard scores for the EQ 360 2.0 scales with the North American EQ 360 2.0 norms, and comparing these scores against a mean of 100 (see Table D.23). Although the effects for all scales did reach statistical significance due to the large sample size, most differences were within 3 standard score points, and only one scale (Emotional Expression) reached a meaningful effect size; for all other scales, Cohen’s d values were below 0.20. The only scale on which the Australian sample scored higher than the North American sample was Assertiveness, but the effect size was negligible (d = 0.06). These results suggest that the norms for Australia and North America are comparable, but the use of specific norms for Australia may offer more precision for use of the instrument in that country.
Internal Consistency
Internal consistency conveys the degree to which a set of items are associated with one another. High levels of internal consistency suggest that the items are measuring a single, cohesive construct. Internal consistency is typically measured using Cronbach’s alpha (Cronbach, 1951), which ranges from 0.0 to 1.0 with higher values reflecting higher internal consistency.
Cronbach’s alpha values for the Australian EQ 360 2.0 normative sample are displayed in Table D.24. Similar to results found with the North American EQ 360 2.0 data, most values ranged from good to excellent (.83 to .98) across the Total EI, composite, and subscale scores, with only Assertiveness reaching a lower, but still acceptable, level (.77).
Factorial Validity
EXPLORATORY FACTOR ANALYSIS
Exploratory factor analysis (EFA) was used to determine whether the subscales established with the North American EQ 360 2.0 normative data empirically emerge from the Australian normative dataset. Five EFAs were conducted, analyzing the items within each composite scale separately. In each EFA, a three-factor solution was forced to examine whether the items corresponding to each subscale within the composite also loaded together in the Australian normative data. Principal axis factoring extraction was used, with direct oblimin (i.e., oblique) rotation, as the factors within each composite are expected to correlate with each other. Reverse scoring was applied to relevant items prior to the analysis. Factor loadings were considered significant if they reached at least ± .30, and an item was defined as cross-loading if it was significant on more than one factor and had loadings within .10 of each other on these factors.
For the Self-Perception Composite EFA, items for the three subscales (Self-Regard, Self-Actualization, and Emotional Self-Awareness) loaded significantly onto their respective factors as expected by the established factor structure, with no cross-loading items.
For the Self-Expression Composite EFA, all Emotional Expression, Assertiveness, and Independence items loaded onto their respective factors, with the exception of one Assertiveness item that fell just below the factor loading cutoff of ± .30, and one Assertiveness item that cross-loaded with Emotional Expression.
For the Interpersonal Composite EFA, all Empathy items loaded together with no cross-loadings. For Social Responsibility, all of the items loaded significantly except for one item that was cross-loaded with Interpersonal Relationships and one item that loaded onto Empathy. For Interpersonal Relationships, all of the items loaded significantly with the exception of one item that loaded onto Empathy, and three items cross-loaded with Empathy.
For the Decision Making Composite EFA, all Problem Solving, Reality Testing, and Impulse Control items loaded onto their respective factors, with only one Problem Solving item cross-loading with Impulse Control.
For the Stress Management Composite EFA, the Flexibility and Optimism items loaded onto their respective factors with only one cross-loading item. For Stress Tolerance, all of the items loaded significantly although three items cross-loaded with Optimism and two items cross-loaded with Flexibility.
To summarize, for the most part the EFAs generated solutions that strongly correspond to the established EQ 360 2.0 factor structure, with the items for each subscale empirically grouping together onto the expected factors.
CORRELATIONS AMONG EQ 360 2.0 COMPOSITE SCALES AND SUBSCALES
Correlations among the EQ 360 2.0 composite scales and subscales were examined in the Australian normative sample to determine if the pattern of results found in the Australian EQ-i 2.0 normative data could be replicated. Table D.25 (composite scales) and Table D.26 (subscales) display these correlations. Overall, in the Australian data, the correlations were stronger in the EQ 360 2.0 data than in the EQ-i 2.0 data. Composite scale correlations ranged from r = .61 (Self-Expression/Interpersonal) to r = .86 (Decision Making/Stress Management), with an average correlation of r = .76. For the subscales, correlations ranged from r = .19 (Assertiveness/Impulse Control) to r = .86 (Optimism/Happiness), with an average correlation of r = .57.
These results support the notion that a single, underlying dimension is being represented in the EQ 360 2.0, yet the values are not overly high and there is enough variation in the correlations to provide clear evidence of the multidimensional nature of the assessment, and support for the existence of composite scales and subscales.