Planning the EQ 360 2.0 Assessment Process

Overview

Planning a successful EQ 360® 2.0 administration involves a few additional considerations beyond the ones covered in Planning the EQ-i 2.0 Assessment Process. This page provides 360-specific best practices and recommendations aimed to reduce the administrative complexity of the 360 process. Although many online features have been added to streamline the setup and administration of the EQ 360 2.0, you should take the time to properly plan the 360 process before you even reach the setup stage to greatly reduce the chance of any logistical issues popping up later in the process.

Due to the complexity of a multirater assessment, prepare to be faced with different schedules, several correspondences, and many questions. Some questions need to be addressed early in the assessment process, long before the participants fill out the actual inventory. Common questions asked by participants early on in the administration process include (but are not limited to):

Why have we been chosen to participate?
How will the results be used?
How will confidentiality be ensured?
How will the raters be chosen?
How many raters need to be chosen?
When does the inventory need to be completed by?
Will there be any follow-up once the inventory has been completed?

Addressing these questions early in the 360 initiative will help the participants, the administrator, and the organization get the most out of the process and the results by fostering a climate of trust and shared understanding. Since not all people invited to take part in the EQ 360 2.0 will actually follow through with the assessment due to work demands or other reasons, 100% participation is sometimes not possible. Low response rates, however, invariably denote problems with the survey process, poor timing, or respondent attitudes towards the survey.

The best practices provided on this page can help you implement an effective EQ 360 2.0 administration with maximum participation. These principles are summarized in Table 6.1.

Table 6.1. Multirater Assessment Planning Dos and Don’ts

Do	Don’t
Identify what the organization or individual hopes to gain from the process by connecting it to specific business needs. Clearly state the goal (e.g., EQ 360 2.0 assessment will be used to improve leadership skills) and how the company will get there (e.g., through the assessment, coaching, and development of skills).	Forget to communicate the purpose of the assessment. Participants should understand what behaviors will be measured and how it is relevant to their jobs and the organization. It is much easier to gain people’s commitment if participants believe the behaviors that will be measured by the assessment tool will be directly related to improving individual and organizational performance.
Prepare a business case. Make the value proposition of the EQ 360 2.0 very clear by linking it to the organization’s strategic objectives. Senior management will want to know what are the costs and benefits of the initiative. Well-constructed summaries of research that demonstrate the benefits of EI can have an enormous influence on the decision-making of senior executives. The best EI business cases show a return on investment (ROI) such as increased sales, better customer satisfaction ratings, or higher productivity. It may also be beneficial to list some companies that have used EI assessments. Most organizations like to know what other companies and competitors are using to get an edge.	Ignore senior management. Start by speaking with senior management on how critical it is to track organizational and individual growth and development. Just as a company would collect and analyze its financial information, information about change initiatives should also be analyzed to understand what impact the human resources side of the equation has on overall productivity. Senior management may ask to participate in a pilot study to assess the application of the EQ 360 2.0.
Consider the timing of the assessment. Be sensitive to what is going on in the organization that might get in the way of a successful 360 implementation. For example, midway through a downsizing may not be the best time to conduct a 360 assessment.	Assume everyone’s schedule is open. Consider the availability of managers, raters, and stakeholders (e.g., are there many participants on vacation or away from the office for an extended period?) and whether there are other survey-intensive activities occurring at the same time. If so, reschedule the assessment to a more convenient time.
Involve any necessary stakeholders in decisions around who can select raters and who will have access to the results.	Leave the participant out of the rater selection process. They have the best sense of who interacts with them most and will feel more accountable to the results if they have input in selecting raters.

Getting Organizational Buy-In

print this section

When possible, share the details of the EQ 360 2.0 assessment with the senior management team. The administrator can communicate such issues as what resources will be required, time commitment required, sample timelines, and information about the instrument that will be used. Keep in mind that some individuals within this team may not have heard about emotional intelligence or may not understand what emotional intelligence is. Therefore, a brief definition of EI and the scales of the EQ 360 2.0 will serve to get people thinking in the same direction. Because the scales of the EQ 360 2.0 have good face validity, most prospects quickly see for themselves how these skills would facilitate individual and organizational success.

It may also be advantageous to mention that the EQ-i 2.0® and EQ 360 2.0 were created from over twenty years of research demonstrating the importance of emotional intelligence in the workplace, and that EI skills can be learned. EQ 360 2.0 assessment efforts should not be undertaken unless there is strong and visible support from senior executives. In order to achieve maximum payoff from the assessment, executives should provide the administrator adequate resources for completion, a commitment to ongoing follow-up, as well as a visible presence throughout the assessment and development phases. For more information on gaining support for your EI initiative, refer to Planning the EQ-i 2.0 Assessment Process.

Selecting EQ 360 2.0 Raters

print this section

As part of an EQ 360 2.0 assessment, raters must be recruited for each individual who is to be assessed (also referred to as the “participant” in this guide). Recruiting can be handled by the administrator, the participant, or both. Whoever is involved in the recruiting process must understand that the interpretability and the usefulness of EQ 360 2.0 results rest on the careful assignment of raters to the correct rater groups. There are five rater groups: (1) managers, (2) peers, (3) direct reports, (4) family/friends, and (5) others. These groups will be described in detail later in this section.

In accordance with the multirater approach, each rater group should include as many raters as possible. For each participant, it is recommended that a minimum of three raters be included in each rater group involved in the assessment, although it is common for the manager group to only contain one manager. Inclusion of a sufficient number of raters in each group not only helps to increase the quality of feedback obtained from the particular groups that participate in the assessment, but also helps ensure the confidentiality of the responses provided by of each rater within the group. If there are too few raters (i.e., less than three) in the peers, direct reports or friends/family groups to meet the confidentiality requirement, all raters in that group should be assigned to the “other” rater group. Bear in mind that the quality of feedback increases when the number of raters increases, and the robustness of the overall assessment also increases with the number of rater groups involved.

Selecting the right raters is generally not a straightforward process. Figuring out who will choose the raters, who to choose as raters, and what criteria to use for both can be difficult. Choosing who selects the raters is generally performed either by the organization (i.e., Human Resources), the participant’s manager, or the participant. It is, however, important that organizations allow participant input into the rater selection process. Perhaps the participant selects several peers, clients, direct reports and knowledgeable co-workers. Then either the participant’s manager or Human Resources selects several more to round out the rater groups.

Some participants might choose certain raters in order to minimize negative feedback, either in the hopes of improving the chance of being seen in a positive light, or out of a belief that the positive results may increase chances of a promotion or monetary gain. A representative of the organization should speak to participants before completion of the inventories in order to make clear that the assessment is for developmental purposes only. This way people can seek honest feedback from the right people, receive information that might truly be helpful, and not worry about hurting their chances of promotion or financial gain.

Be sure to consider the purpose of the EQ 360 2.0 assessment and the type of information required from it when selecting raters. Determine the number of raters that will provide the most robust information for the participant. The more diligently the raters are selected, the more reliable the assessment tool will be.

Issues to Consider When Selecting Raters

Choose raters who...

are credible and trustworthy. Raters with these characteristics will give fair and accurate information. These individuals generally will not inflate results that are not deserved and will not be malicious or opportunistic.
work closely with the person being assessed. These people are in a better position to rate the individual’s performance in different settings.
have worked with the person being assessed for some time (more than 1 year). They will be more familiar with how the individual performs in different areas.
have worked with the individual being assessed for a shorter period of time (less than 1 year); they will be more likely to respond given the current context and be less swayed by history.
represent as many different groups as needed. Each rater group offers a perspective of performance that raters from the other groups may not observe, and certain rater groups are better than others at rating certain aspects of behavior (e.g., direct reports are the targets and therefore the best observers of leadership).

Rater Definitions

In most organizations, assessments will be based on the manager, peer, and direct report groups. Although the administrator should strive to maximize the number of raters per group, many participants will have only one manager, and some will not have individuals who report directly to them.

Discuss with your client whether the rater definition you use will be narrow or broad so that results from similar raters can be pooled and interpreted in a meaningful way. For smaller organizations with fewer potential raters, a broad definition may be most appropriate. For larger organizations, however, potential raters will be more plentiful, allowing a narrower approach to defining rater groups. Broad and narrow rater definitions are suggested in the following examples.

Manager

Narrow: A manager, supervisor or director who the participant reports to within the organizational chain of command.
Broad: Any individual within the organization who holds a position or rank above that of the participant.

Peer

Narrow: Any coworker or colleague in the participant’s work environment who holds a similar rank within the organizational hierarchy.
Broad: Any individual who works with the participant on an equal one-to-one basis.

Direct Report

Narrow: Any individual who is responsible to, or reports directly to, the participant.
Broad: Any individual holding a lower rank or position relative to the participant within the organizational context.

Family/Friend

Narrow: A member of the participant’s immediate family or a close friend outside of the workplace.
Broad: A member of the participant’s extended family, a guardian or godparent, or an acquaintance outside of the participant’s workplace.

Other

Narrow: Any person not fitting the above definitions (e.g., an external or internal customer or client), or someone who belongs to more than one rater group (e.g., a worker peer who is also a close friend or family member).
Broad: The Other rater group designation can be utilized by the administrator to combine raters from more than one category. This option may be useful, for example, for combining rater groups that consist of one rater each with raters from other levels. Since showing the participant any results in which a rater could be identified represents a clear violation of rater confidentiality, the system will not allow you to generate results for rater groups with fewer than three raters. Using the Other rater group will ensure rater confidentiality while capitalizing on feedback from raters who would otherwise be unheard.

Preparing Participants and Raters

print this section

Speaking with the participant and raters before the administration is essential to completing a successful EQ 360 2.0 assessment. All individuals participating in the 360 process need to have an understanding of the purpose, process and expectations of the assessment. The more individuals understand about why a 360-degree assessment is being introduced into the organization and how the information will be used, the more likely participants are to support the effort.

Participants should be notified of a meeting to discuss the 360 process well in advance of the administration of the instrument. The meeting notification should be distributed to all participants and senior management. The notification should include information about the importance of attending the meeting, as well as a meeting agenda.

Issues to Consider when Preparing Participants and Raters

Depending on the knowledge and experience level of the participants, the administrator may want to discuss the construct of emotional intelligence, how the construct is important to job performance, what a 360-degree assessment is, the EQ 360 2.0 and definitions of its scales, how long it will take to complete, and the look of the report.
Discuss with the participant and raters the expected outcomes of the assessment – what do they want to get out of it?
Discuss with the participant and raters the reason why they were selected to participate.
Discuss with the participants their right to confidentiality, informed consent, where their data will be kept, and who will have access to their data.
Discuss the importance of providing answers that are honest and reflect current functioning.
Discuss with the participants and raters how they will receive ongoing support and follow-up. Participants need clarification about what is expected of them after they receive their feedback. Raters need to hear what outcomes will result from their feedback; will they be involved in the action planning process or subsequent progress check points? In any case, participants and raters need to know that the 360-degree assessment is not a one-time event, nor one that is a replacement for having healthy and timely conversations around performance.

In an organizational setting, it may be most efficient to introduce the EQ 360 2.0 at a meeting attended by all intended participants or through a well-detailed email. This will help to ensure that all the participants and potential raters are equally and fully informed about what is entailed in the assessment process, and will provide the administrator with the opportunity to address important questions or concerns prior to beginning the process.

The general approach to introducing the EQ 360 2.0 is to stress the positive, constructive, and beneficial aspects of the assessment. Encourage questions and attempt to answer them in a manner that will not bias the ratings.

Introducing EI and the EQ 360 2.0

Briefly describe the inventory, the number of items, the response format, and how long it takes to complete the questions (approximately 20 minutes). Stress that the EQ 360 2.0 is not a test.Explain that it is designed to provide the participant with a meaningful learning experience and a chance to develop one’s self. You can also say that the participant will benefit personally from the feedback that arises from the EQ 360 2.0. The results will help identify the individual’s stronger abilities and skills as well as those areas that need to be strengthened in order to increase his or her ability to cope more efficiently with daily demands in various areas of life.

Convey to raters and participants that their participation will help the participant to identify personal strengths, as well as facilitate personal growth where weaker areas are observed.

You may wish to hand out an instructional letter following a group discussion, have the participant distribute a letter among his or her raters, or create an email communication that a senior staff member can send to participants. In cases where preparatory meetings and discussion are not feasible, it is recommended that you create an introductory educational document and allow time for participants’ concerns to be addressed before sending administration instructions. When used in lieu of a preparatory meeting, a primary introductory correspondence should include:

A brief description of the EQ 360 2.0
Reasons why they are being asked to complete the EQ 360 2.0 and the benefits of their participation
An explanation of how aggregate results help to maintain anonymity
That their participation is voluntary and they are free to withdraw at any time
What will be done with the results
Who to contact with questions or concerns

Regardless of your preparatory methods, it is essential that detailed written instructions be provided to the raters about proper conditions for instrument completion. The administration instructions should reiterate the purpose of the assessment, why emotional intelligence is being measured, the importance of honest responses, why the participant was selected, and that they are free to withdraw at any time.