The on-line article at the SHRM site loads somewhat slowly.  For your convenience, we've provided you the complete article below. However, if you'd still like to transfer to the SHRM site, click on the gold arrow to the right.   Otherwise, just scroll down the page.

Today’s 360º feedback software developers offer an overwhelming assortment of excellent, highly sophisticated programs that gather, process, and report employee feedback. These full-featured programs offer optimal flexibility, resources, and power. They include customizable surveys, a library of questionnaire items, multiple reporting options with graphic displays, action planners, and implementation procedures. They may also offer applications in addition to performance feedback, such as employee or customer satisfaction surveys.

These programs display inviting, user-friendly screens that dazzle you with creative graphics, tool bars, drop down menus, and on-line help. And the price is right. These systems are often so affordable that many companies extend use beyond management and top executives to all employees. With all these pluses, then, where are the minuses?

A common mistake shoppers make is falling into a point-and-click mentality. It is easy to get seduced by software features and lose sight of your real purpose. For that reason, you must step back. Software is only a tool. It doesn’t replace process, communication, and performance coaching and counseling training. You’ll still have to do that at varying levels, depending on your organizational readiness.

Before considering a software program, ask a series of critical questions about what information you really need and how you want to use it. For example, do you want this system for purposes beyond employee development, such as organizational climate and customer satisfaction surveys? Do you want to gather cross-functional information to compare performance across your organization? What is your philosophy regarding total respondent confidentiality? If you value accountability, what impact will that have on your concerns for confidentiality? For example, you may want to track respondents’ action on survey completion, examine potential collusion among raters, or help respondents improve the quality of their feedback. Once you have identified what you want the system to measure, then you need to ask yourself, how do I want to initiate the process?

Next, you must determine the level of your organization’s readiness for multi-source feedback. How will a multi-source feedback system align with your organization’s vision and values? What will you need to orient and prepare your organization to accept this process? This involves systems, staffing, and communication issues. Do you have the internal resources to administer the system, and how do you want to establish your process? How frequently do you want to collect data?

Wayne Young, employment specialist, at First Technology Credit Union in Beaverton, Oregon told us his review team asked many of these questions. "I think the biggest thing we did right is that we took our time. We spent considerable effort clarifying our purpose and our desired outcomes before we even started looking at software. We wanted a program that offered us tremendous flexibility that could be administered twice a year, but wouldn’t require a lot of back-end administrative time. With that in mind, 20/20® Insight GOLD became the clear choice."

We identified three primary criteria to select programs for review. First, the program could be purchased independent of consulting services, second it could run on a PC, and third, the developer would provide the complete program and all related documentation for testing. Four vendors met these criteria. CompStar® Appraiser Plus 360º was about three-fourths through its beta testing, and 20/20® Insight GOLD was near completion of beta testing when reviewed. These systems will be available by early fourth quarter 1998 and may contain further enhancements after this review is published. PulseTools™, an upgrade and companion module to CorporatePulse™, was introduced earlier this year. Intelligent Consensus® 3.1 has been on the market since 1994. Intelligent Consensus® will introduce its 3.2 version some time during fourth quarter 1998, but the beta version was unavailable for review. To help you assess these programs, we’ll walk through some scenarios and identify how these programs stack up.

Suppose your company has 300 employees and a budget of $10,000. You have been using paper forms for performance management and recently tested the 360º process. Your culture has accepted it and it works fine, but the paperwork, data collection, and consolidation are cumbersome. You need to automate the process and track respondents to administer the system more efficiently. You want to keep things simple and avoid overwhelming employees with extensive reports.

CompStar® Appraiser Plus 360 can handle these basic needs within your budget. It is a solid tool that packs in some powerful features for a modest price. It’s primarily designed to work on a network, with options to export surveys for off-site evaluations. It contains a modest library of core job behavior questionnaire items and includes goal planning and development modules. Questionnaire items are fully customizable, and you can choose from a variety of rating scales and customize weighting descriptors. Respondents can insert comments to support job behavior ratings and also include summary comments. The system also permits weighting of individual job behaviors, goals, and development plans. Weighting can also be applied to the type of respondent as well as within a respondent category.

Currently, the program only generates individual reports containing each respondent’s ratings and comments. It does not provide a consolidated report which groups ratings or comments to allow an at-a-glance review on an item-by-item basis of job behaviors. There are also no options to have reports displayed graphically, such as shaded or color coded horizontal bars, to compare respondent ratings. Reports do contain a final summary page by respondent category (e.g., self, peer, supervisor), which lists a consolidation of average feedback for overall job behaviors, goals, and development.

While the current software’s reports and data displays are limited, the database is managed in Microsoft Access. Thus, the user has the capability of creating custom reports by exporting the data and using Access’s report writer function to generate other reports.

CompStar® does a good job of tracking respondents. However, the system is not designed to integrate e-mail notices to respondents of pending evaluations. Additionally, respondent categories are limited. The system only permits self, supervisor, or peer as options and requires other respondents, such as customers and direct reports, to be placed in the existing category of peer.

"Limiting respondent categories was not a problem for our project at Sharp Electronics," said Susan C. White, an external implementation consultant from HR Performance Solutions in Wayne, New Jersey. "Sharp’s application is for high performance technical teams, so we are primarily concerned with peer feedback. Initial set up of the system was more difficult than expected, but once complete, it really hummed--and employees found it very easy to use. We like it because it offers so much flexibility for the price. However, we’d prefer to see the reporting function improved and discovered that the rating level descriptions did not show up on the data entry screens. This required us to provide employees with an explanation of the ratings before they could enter their evaluations."

20/20® Insight GOLD and Intelligent Consensus® could perform all desired functions and give greater flexibility in reporting and respondent options. They also have many more media distribution options, such as paper forms, diskette and e-mail attachments through Internet and Intranets. Additionally, these systems integrate a respondent notification feature to alert respondents of pending evaluations. However, these additional features are probably overkill in this instance and carry a higher price tag. CorporatePulse™ - PulseTools™ could meet the price demands and deliver equally well on all criteria except one. It does not track individual respondents. The program maintains ultimate confidentiality by design and, therefore, does not permit this option

Assume you’re in the initial stages of 360º feedback. You need help orienting your employees to the process and assistance in selecting and writing valid questions. You want an efficient, highly-flexible, confidential system with many reporting options and graphic displays. You want action planning, development options, and suggested resources built in.

Three systems will meet most of these needs at varying levels. All offer considerable flexibility, customization, and resources for questionnaire development and reporting functions. CorporatePulse™ provides three days of extensive training. While participants spend part of their training on system administration, they primarily focus on writing good questionnaire items and piloting procedures. The last half day of training, participants work on a "live" project and receive individualized assistance with questionnaire design. The software also contains an expansive library of pretested questions. Both training and library items are included in the base price.

"We found the training CorporatePulse™ provided was excellent. It helped us customize our questions and tie them directly our leadership academy," said Sherri Sayre, a member of the leader enhancement and associate development team for NationsBank in Jacksonville, Florida.

"We also needed media options. Not everyone in the company has computer access or is comfortable using a computer. Generating scannable forms directly from the system was a real plus for us."

20/20® Insight GOLD also provides a broad library of well-researched questions included in its software price. However, one and one-half days of systems-focused administrator’s training are charged separately. Intelligent Consensus® offers access to their survey library at additional cost but includes two days’ systems administrators training in the base price.

Specific planning and implementation issues are extensively addressed by 20/20® Insight GOLD’s bonus module, The 360 Smart Kit. This includes information on the 360º process, frequently asked questions and answers, suggested PowerPoint presentation slides for explaining the project to employees and management, and lesson plans and objectives for training employees on how to provide appropriate performance feedback.

The Project Planner, a module in CorporatePulse™ assists in project administration. It’s a time-line tracking function of all project steps in general survey development, administration, and analysis. The on-line tutorial gives additional, but limited, project planning and communication strategies related to 360º feedback.

Intelligent Consensus® provides comprehensive consulting services at additional costs for design team training and general process consulting. They also offer other related training options, such as motivating and sustaining change, action planning, and leadership coaching.

"We were confident that we could design the survey on our own based on the flexibility of Intelligent Consensus®. Where we really needed help was with process, so we engaged their consultants. In two days we had all the issues resolved, which resulted in a smooth implementation." explained Lili Cordeiro, organizational development consultant at Westpark Hospital in Toronto. "The software increased our efficiency. Our end-users love it and tell us they find it very easy. However, our system administrators found the set up complex and we needed some additional hand-holding beyond the initial training. Fortunately, their technical staff was extremely patient and accommodating--even when the error was ours!"

All three software programs contain a variety of reporting options with graphics, gap analysis, and summaries. However, CorporatePulse™ is unable to incorporate comments into the overall report because the system is designed as a flat file, rather than a relational data base. Thus, a separate comments report is required. Intelligent Consensus® and 20/20® Insight GOLD are highly flexible and offer a range of reporting options to display data and integrate comments within the report.

CorporatePulse™ does not have on-line action planning or development resource modules. They offer a paper-based booklet called The Action Planner used with a proprietary 360º content set for interpersonal communication, which can be adapted to a general 360º environment.

20/20® Insight GOLD uses two features to develop action plans and provide development resources. Employees load an Individual Development Planner diskette on their system and to track personal progress. This feature contains an algorithm with 16 questions that allows the employees to analyze why they may have received low ratings on specific items. Then, it offers general suggestions about how to improve these areas. It uses the same mechanism to address strengths.

Its other feature is a development recommendations report. This comprehensive report identifies low-rated items and gives detailed explanations about what actions to take and which resources to use for improvement. Users can customize this section by adding development suggestions and updated resources.

This powerful development recommendation feature is also contained in the Action Planning Module of Intelligent Consensus®. This module creates a Criteria Ranking Report which ranks all ratings from top to bottom. Next, it automatically selects those areas which have been rated low and customizes a highly specific action plan, including detailed activities and resources to use for development. This module is an extra cost item.

Perhaps you want to install 360º feedback, but are afraid some participants may try to "game" the system. You want to be able to identify if collusion exists. Additionally, you’re looking for ways to assess your respondents’ rating capabilities to reduce evaluator bias. Finally, you need a broad view of your organizational improvement over time and want to compare performance across organizational sectors or between departments.

Intelligent Consensus® handles all these functions with IntelliReporting, a special module offered at additional cost. This module provides specific reports to analyze rater collusion and rater bias. It also can seamlessly gather and summarize all performance data across the enterprise for comparison reports.

Rater collusion and rater bias reports are not available through the other systems. However, 20/20® can easily provide organizational cuts of performance data through its aggregate reporting feature. While CorporatePulse™ can provide organization-wide data, the feature is not built into the system. The administrator must export and manipulate the data to another program develop reports. CompStar® has a limited built-in reporting capability to address this need. However, the data can also be exported and manipulated in Access to achieve the desired results.

As your evaluation team proceeds through the vendor selection process, make sure you select a vendor with a solid reputation and staying power. Ask about technical support, maintenance contracts, and upgrades. Finally, call their customers. Find out why they selected the software system, what they like most about it, and the quality of service they received. Investing in 360º software is more than merely licensing a program. It’s a significant tool designed to help you sustain the process of continuous employee development. Choose wisely.

N. Elizabeth Fried, Ph.D., CCP is president of N.E. Fried and Associates, Inc., a Dublin, Ohio consulting firm specializing in compensation survey research and organizational effectiveness. E-mail: elizabeth@nefried.com

	N. E. FRIED AND ASSOCIATES, INC.
	Ohio Office:  5590 Dumfries Court West   -    Dublin, OH 43017-9429   -  Phone (614) 766-9800· Send E-Mail
	California Office: 7564 Romeria  -  Carlsbad CA 92009 - Phone (760) 602-0163
Copyright © 2002 by N. E. FRIED AND ASSOCIATES, INC.