In the December 1998 issue of HR Magazine, we assessed four major 360 degree software tools based on a variety of technical criteria and organizational needs. Prior to this article, we had no vendor relationship. However, as a result of extensive testing, we first chose 20/20® Insight GOLD as the best option for our clients. At the time, we felt it was the most flexible, comprehensive, user friendly 360 degree software system available on the market.  

During the past several years, we continued to monitor the market for improvements and technological advancements and assess new products.  With the explosion of the Internet, we found other equally appealing products to serve our clients.  As the year  2000 approached, a new wave of 360 degree feedback products hit the market known as Application Service Providers (ASPs). The ASP created new opportunities for customers to go directly to the Internet to deploy their 360 projects and get them up and running with as little as four hours training.  Several  software based products jumped on the bandwagon and added an ASP option to their suite of offerings, or they added the Internet as  media option to their software.  We were particularly impressed with Panoramic Feedback's™ ease of use, colorful reports, quick loading and responsiveness and added them to our toolbox.

Additionally, there is a third form of 360 feedback service that has always been there, but also serves a need for clients:  The 360 feedback Service Bureau.  In the past service bureaus having been highly labor intensive and relatively costly.  However,  when combined with new Internet-based products, this service has become a less costly and quite a viable option for clients. Our firm offers this service to clients who have very small one-time projects, larger projects with no staff to manage them, or who want to pilot the 360 feedback process before investing in a an organizational wide rollout.  We can offer this service through through a variety of products that we represent.  It is always the client's choice as to which product they want to use.

Brief Overview of the Difference Between Software, ASPs, and Service Bureaus

As you investigate which type of product or service is right for you, we encourage you to do two key things.  First, get a handle on the differences among all three forms of 360-degree feedback offerings and then determine which will best fit your needs.  Next compare products within the type of configuration you choose.  To help you, let's define the three configurations very simply, examine how they work, and consider some pros and cons.  Before we begin, let's make the following assumptions about all three configurations: 

1.  They will allow you to customize your questions.

2.  They will allow you to add comments.

3.  They typically provide a library of questions from which to draw or modify. 

4.  They will automatically generate reports and do not require any data entry of feedback items unless you need to use some paper-based feedbacks for certain individuals.

5.  They will allow you to customize your rating scales and define them as you wish.

6.  They will allow you have your respondents provide feedback directly on the Internet

360-Degree Feedback Software Systems

Software systems typically offer you administrative software, in which you pay a one-time license fee to the developer to run the 360 applications.  You maintain the system on your PC, local area network (LAN), or database server that runs your InTRAnet.  You create all your surveys at your desktop and then easily upload your survey project to the Internet at a hosted site or connect to your InTRAnet database server and set up from there, depending on the product and your configuration.  Good systems will be designed to load quickly, save response data quickly, and allow respondents to leave the system and return later to complete their survey without losing data if they are interrupted. The systems provide opportunities for email notifications and reminders.  They may generate individual and aggregate reports directly from the web in HTML format or require the data to be quickly downloaded for report generation in PDF format, which can then be automatically sent to recipients as an email attachment. Many allow on-line rater selection, relieving the administrator of many of the initial setup chores. 

As mentioned above, you pay a base price for the administration software.  This is a fixed cost.  You then have some potentially variable costs for usages and maintenance. Usages are typically priced based on the subject being evaluated and can be single usage or unlimited (perpetual) usages.  Sometimes you are also charged for respondents.  Each vendor prices things a little differently, and some offer a one-time enterprise-wide usage license that covers all employees with no on-going additional charges.  You'll have to compare to determine which is best for you.  (See my vendor comparison chart to get the details on this.) Some vendors charge you a 15% to 20% annual maintenance fee to cover their costs of product improvements, technical support, and upgrades.  Be sure to examine this carefully and run your costs out over three years. Consider your average turnover for replacement usage licenses to get a true picture of your real costs.

Pros:  These systems can be cost effective, particularly if you want to survey your employees on an ongoing basis or multiple times a year, have relatively low turnover, and desire complete control of your data.  Some programs will allow on-line rater selection, relieving the administrator of many of the initial setup chores

Cons:  These systems, because they are often highly flexible and feature-rich, require a couple of days training and someone to insure that any updates are downloaded and installed.  (This download is usually uncomplicated and quick.) While the training is not particularly difficult, it does require standard computer literacy with MS Windows and involves a longer initial set up time to decide which features you want to include in your project. 

360-Degree Feedback Application Service Providers (ASPs)

ASPs are a wonderful way to get your 360 feedback projects off to a fast start and keep your administrative time to a minimum. These systems typically will allow you to import your employee data to the web and provide you with all the tools to set up your project online with easy, step-by-step instructions.  Good systems will be designed to load quickly, save response data quickly, and allow respondents to leave the system and return later to complete their survey without losing data if they are interrupted. They provide opportunities for direct email notifications and automatic email reminders and will generate individual and aggregate reports directly from the web and send them to recipients or the administrator in  PDF format.   A review of our updated vendor grid that shows that ASPs have dramatically increased their menu of features, enabling greater reporting capabilities and flexibility.

Fees for an ASP generally begin with an initial setup charge.  Once that has been established, clients purchase usages as they need them.  Typically there are volume discounts.  This is essentially a pay as you go program.   Some ASPs are now beginning to offer perpetual, transferable usages for large volume purchases.  When perpetual licenses are offered, be sure to factor in annual hosting and maintenance charges to get a true picture of first year and on-going the costs.   As mentioned earlier, it’s a good idea to run your costs out over three years when making comparisons.

Pros:  You can begin a 360-feedback project for relatively low time and investment costs.  There is no software to maintain and upgrade.  The ASP continually upgrades its product, which is transparent to the administrator or respondent.  The ASP will notify the administrator of new available features, but typically there is some type of on-line wizard to guide you through new features.  Some ASPs will allow on-line rater selection, relieving the administrator of many of the initial setup chores.  Data is stored on a secure server which can enhance the perception of confidentiality to employees, because a third party is managing the data, rather than having it sit on a local server.  This data is typically exportable, should the company prefer to store it on its own server, or create specialized reports, offering companies storage options.

Cons:  Your employee data is housed on another's site, which may make some companies uncomfortable.  However, good ASPs have secure servers that basically eliminate this concern.  From a pricing perspective, ASP’s pay-as-you-go fee structure over the long run may generate significantly higher costs than a software-based product that provides for a one-time enterprise usage license or unlimited usage assigned to individual subjects.  You will have to conduct a cost comparison over a three-year period to get a true picture.  Conversely, these projected costs may be offset by the lower initial investment of an ASP, allowing you to plan for this through general operating budgets rather than capital budgets.     

360-Degree Feedback Service Bureaus

Service Bureaus essentially do all the work for you.  You just need to provide them basic employee information in electronic format (easy to do as an export from your HRIS system), and they do all the setup and administration for you.  Some will charge for purchasing their data bank of questions; others will not.  Some service bureaus only operate on the Internet.  Others will accept other forms of media, such as paper, scan forms, or CDs.   They will often work with you in the design of your questionnaire and any process consulting you need (e.g., communication, policy issues, etc.), or they will simply act as a processor for you.  Many firms use this mode of 360 feedback for their senior managers when they want to do a one-time project. Our firm has frequently supplied this service to organizations that want to run a pilot prior to purchasing a product we represent or for organizations that simply don’t have the time or resources to do it themselves.

Pros:  Zero training and your administrative involvement is at a bare minimum.  The service bureau handles all the administrative details.  They'll set up the survey questions, invite the participants, collect the data, and generate and distribute the reports for you.  You are often not restricted to standard reports.  They often will customize reports and give you data exactly as you want it.  The fact that the data is being collected and crunched by a third party may be appealing if you have confidentiality concerns about who sees the final reports.  If you are a highly technical company where there are concerns that individual respondent data may be compromised, third-party service bureaus remove the confidentiality concerns. (In reality, however, ASPs or software programs are typically very secure and encrypt individual respondent data.) 

Cons:  Costs are typically higher because someone else is doing the labor to set up on the project, and you are using their system for data collection and web access.  If you want special customization, be prepared to pay more.  Depending on the service bureau, they may have a menu of charges and will charge you for every change.  Others will have a package price to allow for these changes.  Check your agreements carefully. 

  Conclusion

We are not focused on selling you one specific product.  We listen to your needs and narrow down your choices to the one or two products that fit your budget and meet your requirements. We can also act as a service bureau, since we are also licensed on several of the products we represent. Furthermore, we offer process consulting, communication, training, and coaching as part of our suite of services. Our objective is to provide you with a solution to your 360-degree feedback needs so that you can sort through this process with ease.  To view our free vendor comparison grid click on the HRMagazine cover below.  Once there, you can request a PDF format of the detailed comparison which will be sent to shortly.

Process Consulting

If you you need help assessing organizational readiness or guidance with implementation, we can help.  We provide the following services:

• Assessing organizational readiness
• Aligning performance criteria with your vision and strategy
• Identifying policy and procedure considerations
• Developing behaviorally based questionnaire items
• Communicating the 360 degree program components to employees
• Planning employee development
• Coaching

• Linking pay to performance

Copyright © 2003 by N. E. FRIED AND ASSOCIATES, INC.