Part V: creating the EQ-i 2.0 and EQ 360 2.0

South African Norms

EQ 360 2.0 South African Norms – Standardization

NORMATIVE SAMPLE

Normative data for the South African EQ 360 2.0 were collected from November, 2011 to June, 2013. Respondents (“the raters”) were required to rate an individual (“the ratee”) on the EQ 360 2.0 and provide demographic information about both themselves and the ratees. In order to create a representative normative sample, specific demographics regarding the ratees were considered during the data collection process. Information was collected on each ratee’s gender, age group, geographic region, ethnicity, education level, employment status, and occupation area. Information about the ratee (i.e., the person being rated) was provided by the rater (i.e., the person completing the assessment). Information was also collected about the type (i.e., manager, work peer, direct report, or family/friend) and strength of the rater-ratee relationship (i.e., how long they have known each other, how well they know each other, and how often they interact with each other).

Rater Description: The sample of 1,000 raters (i.e., the participants providing the ratings) was 49.0% male and 51.0% female. The mean age of the rater sample was 38.3 years (SD = 10.9 years). Most raters indicated their ethnicity as White (46.4%) or Black (34.6%), with smaller proportions of Coloured (10.9%) and Asian/Indian (7.9%) raters; 0.2% of raters indicated ‘Other.’ More than half of the raters (57.5%) had a Bachelor’s degree or equivalent/higher qualifications, and the remaining raters had either a National Diploma (17.2%) or Grade 12 education level or below (25.3%). The majority of raters knew the ratee for at least one year (86.6%; see Table F.18), and approximately two-thirds stated that they knew the ratee well or very well (68.3%; see Table F.19). More than half (59.6%) of the raters indicated that they interacted with the ratee very often (daily or almost daily; see Table F.20).

Ratee Sample: The normative sample was formed based on the ratees’ demographic characteristics. Table F.21 shows the ratee age group by gender distribution. The ratee sample included an equal ratio of men to women, stratified across four rater types: direct report, manager, work peer, and friend/family member (see Table F.22).

Representation by geographic region is shown in Table F.23. Regarding ethnicity (see Table F.24), most ratees were Black (42.6%) or White (38.8%), with smaller proportions of Coloured (9.9%) and Asian/Indian (8.7%) ratees. The majority of ratees in the normative sample (72.1%) had an education level higher than Grade 12 (see Table F.25), and most were employed full-time (86.0%); there were no unemployed ratees in the normative sample (see Table F.26). The sample breakdown by occupation area is shown in Table F.27, with the largest proportions of ratees working in Business/Commerce/Management (26.7%), Education/Training/Development (12.5%), Manufacturing/Engineering/Technology (12.2%), and Human/Social Science (12.1%).

NORMING PROCEDURES

Similar to the South African EQ-i 2.0, the first step in the South African EQ 360 2.0 norming procedure was to determine if any demographic trends existed in the South African normative data. Large score differences between rater types (i.e., managers, work peers, direct reports, friends/family members) would suggest a need to create an option for separate rater type norm groups, while a lack of such differences would suggest a need to create a single norm option with the rater types combined. Similarly, large score differences between male and female ratees, or across various ratee age groups, would suggest a need to create an option for separate gender- or age-based norm groups. Conversely, a lack of such differences may dictate the use of a single norm group with genders and age groups combined.

A series of analyses of variance (ANOVA) was used to examine the relationships between EQ 360 2.0 scores and ratee gender, age, and rater type. To better control for Type I errors that might occur with multiple analyses, a more conservative criterion of p < .01 was used for all F-tests.

Gender Effects. Overall (and similar to results from the North American EQ 360 2.0 data), gender effects were less pronounced in the EQ 360 2.0 South African normative sample than they were in the EQ-i 2.0 South African normative sample. Results showed that for most scales, there was no meaningful effect of gender. Significant differences were seen only on the following scales, each reaching only a small effect size: women were rated higher than men on the Interpersonal composite and the Emotional Self-Awareness, Emotional Expression, and Empathy subscales, and men were rated higher than women on Self-Regard. See Table F.28 for effect sizes and Table F.29 for descriptive statistics and significance test results.

Age Effects. Results showed that for most scales, there was no meaningful effect of age. Significant effects across age groups were found only for the Interpersonal Relationships, Flexibility, and Happiness subscales. For each of these effects, which were all very small in magnitude, the older age groups had the lowest scores. See Table F.28 for effect sizes and Table F.30 for descriptive statistics and significance test results.

Rater Type Effects. The Total EI score, as well as a number of composite scales and subscales, showed significant differences between rater types; however, all effect sizes were small. The typical pattern showed that the highest ratings were provided by direct reports, and the lowest ratings were provided by managers. See Table F.28 for effect sizes and Table F.31 for descriptive statistics and significance test results. Although small in magnitude, the rater type differences observed in the South African EQ 360 2.0 normative data were somewhat larger and more pervasive than what has been observed in the normative data for other countries.

Norm Groups and Norm Construction. Overall, the effect sizes (i.e., d and partial η² values) found in the normative data suggest negligible or small effects of ratee gender and age. The scarcity of meaningful effects suggested that it was not necessary to create specific gender- or age-based norms for the South African EQ 360 2.0. Although numerous meaningful rater-type effects were observed, the differences were not so large to require specific rater-type norms, and on the whole it was preferable to be consistent with the EQ 360 2.0 norm options used in other countries. Therefore, only overall norms were developed. Accordingly, some sensitivity may be required in interpreting results obtained from different types of raters. For instance, managers’ ratings might be expected to be lower than those obtained from other rater types.

These norms were created using the same procedure as the South African EQ-i 2.0 norms. Standard scores for all scales were computed with a mean of 100 and standard deviation of 15. Results revealed that skewness and kurtosis values were relatively small (skewness values ranged from -1.19 to -0.38; kurtosis values ranged from -0.51 to 1.39). Examination of the scale histograms indicated a slight negative skew for some scales, but did not reveal any significant departures from a bell-shaped (Gaussian) curve. Therefore, artificial transformation of scores to fit normal distributions was deemed unnecessary. A histogram for the EQ 360 2.0 Total EI score is provided in Figure F.7.

Comparison of South African Norms to North American Norms. The South African sample was compared against the North American normative sample by computing standard scores for the EQ 360 2.0 scales with the North American EQ 360 2.0 norms, and comparing these scores against a mean of 100. Significant differences were observed on all but one scale, and most reached at least a small effect size (i.e., Cohen’s d ≥ 0.20). Mean differences ranged between 0.41 and 8.52 standard score points; on most scales, the South African sample obtained higher scores and had small to medium-sized effects. Where lower scores were observed for the South African sample, none of these differences showed meaningful effects. The largest differences were observed on the Self-Perception composite (d = 0.61), and the following subscales: Self-Actualisation (d = 0.61), Emotional Self-Awareness (d = 0.57), Reality Testing (d = 0.49), Social Responsibility (d = 0.46), and Assertiveness (d = 0.41). There were a number of other meaningful differences that were smaller in magnitude. Results are presented in Table F.32. The observed differences between EQ 360 2.0 ratings in North America and in South Africa suggest that using specific norms for the South African population may offer more precise results when using the instrument in that country.

Ethnicity Effects. A series of analyses of variance (ANOVA) was used to examine the relationships between ratee ethnicity group and EQ 360 2.0 scores. Ratee age group and gender were included as factors in order to control for the effects of these demographic variables. To better control for Type I errors that might occur with multiple analyses, a more conservative criterion of p < .01 was used for all F-tests.

Results showed that for most scales, there was no meaningful effect of ethnicity. The only statistically significant scale was Interpersonal Relationships, with a small effect size (partial η² = .013). See Table F.33 for descriptive statistics, significance test results, and overall effect sizes (partial η²).

To more closely examine the relationship between EQ 360 2.0 scores and ethnicity, effect sizes (Cohen’s d) were computed for each pairwise comparison between ethnicity groups for the Interpersonal Relationships subscale. Results showed that the Black group scored higher, with small effect sizes, than both the Asian/Indian group (d = -0.29) and the White group (d = -0.26), but no meaningful effects were observed for the other ethnicity group comparisons.

These results demonstrate that overall EQ 360 2.0 results are comparable for different ethnic groups in South Africa.

Internal Consistency

Cronbach’s alpha values for the EQ 360 2.0 scales for the South African normative sample are presented in Table F.34. These values demonstrate good to excellent reliability, and are particularly favourable given the small number of items included in most subscales. The alpha value of the Total EI scale was .98, values for the composite scales ranged from .86 to .95, and values for the subscales ranged from .77 to .94. The high level of internal consistency found in the Total EI score supports the idea that the EQ 360 2.0 items are measuring a single, cohesive construct, namely emotional intelligence. The same can be said of the individual components of emotional intelligence that make up the EQ 360 2.0 (i.e., the composite scales and subscales).

Factor Validity

EXPLORATORY FACTOR ANALYSIS

Exploratory factor analysis (EFA) was used to determine whether the subscales established with the North American EQ 360 2.0 normative data empirically emerge from the South African normative dataset. Five EFAs were conducted, where the items within each composite scale were analysed separately. In each EFA, a three-factor solution was forced to examine whether the items that corresponded to each subscale within the composite also loaded together in the South African normative data. Principal axis factoring extraction was used, with direct oblimin (i.e., oblique) rotation, as the factors within each composite are expected to correlate with each other. Reverse scoring was applied to relevant items prior to the analysis. Factor loadings were considered significant if they reached at least ± .30, and an item was defined as cross-loading if it was significant on more than one factor and had loadings within .10 of each other on these factors.

For the Self-Perception Composite EFA, items on the three subscales (Self-Regard, Self-Actualisation, and Emotional Self-Awareness) loaded significantly onto their respective factors as expected by the established factor structure, with no cross-loading items.

For the Self-Expression Composite EFA, all items on the Emotional Expression, Assertiveness, and Independence subscales loaded onto their respective factors, with the exception of one Assertiveness item that loaded onto Emotional Expression.

For the Interpersonal Composite EFA, most items on the Interpersonal Relationships, Empathy, and Social Responsibility subscales loaded onto their respective factors. For Interpersonal Relationships, one item loaded onto Empathy and two items cross-loaded with Empathy.

For the Decision Making Composite EFA, all Problem Solving, Reality Testing, and Impulse Control items loaded onto their respective factors, with the exception of one Impulse Control item with a factor loading below the cut-off. One Problem Solving item cross-loaded with Impulse Control, and one Reality Testing item cross-loaded with Problem Solving.

For the Stress Management Composite EFA, items on the three subscales (Flexibility, Stress Tolerance, and Optimism) loaded significantly onto their respective factors with no cross-loadings, although one Optimism item fell just a bit below the cut-off with a factor loading of .29.

To summarize, the EFAs generated solutions that strongly correspond to the established EQ 360 2.0 factor structure, with 115 out of 118 items empirically grouping together onto the expected factors.

CORRELATIONS AMONG EQ 360 2.0 COMPOSITE SCALES AND SUBSCALES

Correlations among the EQ 360 2.0 composite scales and subscales were examined. These correlations were expected to be generally high, given that they all measure the same underlying construct of emotional intelligence; however, they should not be so high as to indicate redundancy between the scales. Correlations observed in the South African normative sample are presented in Tables F.35 (composite scales) and F.36 (subscales).

The composite scale correlations ranged from r = .53 (Self-Expression/Interpersonal) to r = .81 (Stress Management/Decision Making), with an average correlation of r = .70. For the subscales, correlations ranged from r = .17 (Empathy/Independence and Impulse Control/Assertiveness) to r = .91 (Optimism/Happiness), with an average correlation of r = .52. As highlighted in Table F.36, subscale correlations within composite scales ranged from r = .21 (Emotional Expression/Independence) to r = .74 (Interpersonal Relationships/Empathy). These results support the notion that a single, underlying dimension is being represented in the EQ 360 2.0, yet the values are not overly high, and there is enough variation in the correlations to provide clear evidence of the multidimensional nature of the assessment, and support the existence of composite scales and subscales.