J
360 Degree Feedback Vendor Comparison
360 feedback, 360 feedback software, upward feedback, performance feedback, 360 feedback vendors, performance management, ASP, 360 feedback service bureau
Panoramic Feedback, 2020 Insight Gold,
Evaluating Your Options
By
N. Elizabeth Fried, Ph.D., CCP
Updated 1/3/11
Introduction
In our article for HRMagazine, we
discussed how to evaluate 360 feedback program applications based on your
needs. We described several different scenarios with varying requirements
and indicated which applications fit best. We also developed a
comprehensive grid, providing detailed comparisons of the programs,
feature by feature. This grid is much like one seen in PC Magazine
or Consumer Reports. However, the editors at
HRMagazine had space constraints and significantly reduced this
section. We believe that when you are seriously looking at 360-feedback
software, the comparison grid is exactly what you want to help you make
the right choice. We wanted to give you that option, so we continue to
update it and have reproduced the grid for you from our site to a PDF
Version. The following grid is our eleventh update and compares sxi
products: 20/20® Insight Gold, 2020® for Small Business, Panoramic
Feedback™, Results on Demand - Performance™, eCustom™, and Compass™.
During the past several years, we continued to monitor the market for new products, improvements, and technological advancements. With the explosion of the Internet, we found other equally appealing products to serve our clients. As the year 2000 approached, a new wave of 360 degree feedback products hit the market known as Application Service Providers (ASPs). The ASP created new opportunities for customers to go directly to the Internet to run their 360 projects, learn the system, and deploy their projects with as little as four hours training. Several software-based products jumped on the bandwagon and added an ASP option to their suite of offerings, or they added the Internet as a media option to their software. In this update we’ve discovered the next generation ASP, which is known as Software as a Service (SaaS).
Note: We typically update our grid about every 9-12 months. Some of the products covered in the original article will not appear on the grid because the vendor either no longer supports them, or the vendor has gone out of business. Thus, vendor reputation, staying power, and quality of service should be key considerations in your analysis.
Brief Overview of the Difference Between 360 Degree Feedback Software, ASPs, SaaS, and Service Bureaus
As you investigate which type of product or service is right for you, we encourage you to do two key things. First, get a handle on the differences among all three forms of 360-degree feedback offerings and then determine which will best fit your needs. Next compare products within the type of configuration you choose. To help you, let's define the three configurations very simply, examine how they work, and consider some pros and cons. Before we begin, let's make the following assumptions about all three configurations:
1. They will allow you to customize your questions.
2. They will allow you to add comments.
3. They typically provide a library of questions from which to draw or modify.
4. They will automatically generate reports and do not require any data entry of feedback items unless you need to use some paper-based feedbacks for certain individuals.
5. They will allow you to customize your rating scales and define them as you wish.
6. They will allow you have your respondents provide feedback directly on the Internet
360-Degree Feedback Software Systems
Software systems typically offer you administrative software, in which you pay a one-time license fee to the developer to run the 360 applications. You maintain the system on your PC, local area network (LAN), or database server that runs your InTRAnet. You create all your surveys at your desktop and then easily upload your survey project to the Internet at a hosted site or connect to your InTRAnet database server and set up from there, depending on the product and your configuration. Good systems will be designed to load quickly, save response data quickly, and allow respondents to leave the system and return later to complete their survey without losing data if they are interrupted. The systems provide opportunities for email notifications and reminders. They may generate individual and aggregate reports directly from the web in HTML format or require the data to be quickly downloaded for report generation in PDF format, which can then be automatically sent to recipients as an email attachment. Many allow on-line rater selection, relieving the administrator of many of the initial setup chores.
As mentioned above, you pay a base price for the administration software. This is a fixed cost. You then have some potentially variable costs for usages and maintenance. Usages are typically priced based on the subject being evaluated and can be single usage or unlimited (perpetual) usages. Sometimes you are also charged for respondents. Each vendor prices things a little differently, and some offer a one-time enterprise-wide usage license that covers all employees with no on-going additional charges. You'll have to compare to determine which is best for you. (See my vendor comparison chart to get the details on this.) Some vendors charge you a 15% to 20% annual maintenance fee to cover their costs of product improvements, technical support, and upgrades. Be sure to examine this carefully and run your costs out over three years. Consider your average turnover for replacement usage licenses to get a true picture of your real costs.
Pros: These systems can be cost effective, particularly if you want to survey your employees on an ongoing basis or multiple times a year, have relatively low turnover, and desire complete control of your data. Some programs will allow on-line rater selection, relieving the administrator of many of the initial setup chores.
Cons: These systems, because they are often highly flexible and feature-rich, require a couple of days training and someone to insure that any updates are downloaded and installed. (This download is usually uncomplicated and quick.) While the training is not particularly difficult, it does require standard computer literacy with MS Windows and involves a longer initial set up time to decide which features you want to include in your project.
360-Degree Feedback Application Service Providers (ASPs)
ASPs are a wonderful way to get your 360 feedback projects off to a fast start and keep your administrative time to a minimum. These systems typically will allow you to import your employee data to the web and provide you with all the tools to set up your project online with easy, step-by-step instructions. Good systems will be designed to load quickly, save response data quickly, and allow respondents to leave the system and return later to complete their survey without losing data if they are interrupted. They provide opportunities for direct email notifications and automatic email reminders and will generate individual and aggregate reports directly from the web and send them to recipients or the administrator in PDF format. A review of the updated grid that follows shows that ASPs have dramatically increased their menu of features, enabling greater reporting capabilities and flexibility.
Fees for an ASP generally begin with an initial setup charge. Once that has been established, clients purchase usages as they need them. Typically there are volume discounts. This is essentially a pay as you go program. Some ASPs are now beginning to offer perpetual, transferable usages for large volume purchases. When perpetual licenses are offered, be sure to factor in annual hosting and maintenance charges to get a true picture of first year and on-going the costs. As mentioned earlier, it’s a good idea to run your costs out over three years when making comparisons.
Pros: You can begin a 360-feedback project for relatively low time and investment costs. There is no software to maintain and upgrade. The ASP continually upgrades its product, which is transparent to the administrator or respondent. The ASP will notify the administrator of new available features, but typically there is some type of on-line wizard to guide you through new features. Some ASPs will allow on-line rater selection, relieving the administrator of many of the initial setup chores. Data is stored on a secure server which can enhance the perception of confidentiality to employees, because a third party is managing the data, rather than having it sit on a local server. This data is typically exportable, should the company prefer to store it on its own server, or create specialized reports, offering companies storage options.
Cons: Your employee data is housed on another's site, which may make some companies uncomfortable. However, good ASPs have secure servers that basically eliminate this concern. From a pricing perspective, ASP’s pay-as-you-go fee structure over the long run may generate significantly higher costs than a software-based product that provides for a one-time enterprise usage license or unlimited usage assigned to individual subjects. You will have to conduct a cost comparison over a three-year period to get a true picture. Conversely, these projected costs may be offset by the lower initial investment of an ASP, allowing you to plan for this through general operating budgets rather than capital budgets.
360 Degree Feedback Software as a Service (SaaS)
This approach began with the intent of obtaining rights to use software as needed versus licensing all devices with all application. The two best sources that I found to describe this in detail can be found at http://en.wikipedia.org/wiki/Software_as_a_service and http://msdn.microsoft.com/en-us/library/aa905332.aspx
These documents cover the aims, history, philosophy, key characteristics, pros and cons. To attempt to distill them into a few paragraphs would be insufficient. To quote Wiki in its introductory definition: “Software as a Service (SaaS, typically pronounced 'sass') is a model of software deployment whereby a provider licenses an application to customers for use as a service on demand. SaaS software vendors may host the application on their own web servers or download the application to the consumer device, disabling it after use or after the on-demand contract expires. The on-demand function may be handled internally to share licenses within a firm or by a third-party application service provider (ASP) sharing licenses between firms.”
360-Degree Feedback Service Bureaus
Service Bureaus essentially do all the work for you. You just need to provide them basic employee information in electronic format (easy to do as an export from your HRIS system), and they do all the setup and administration for you. Some will charge for purchasing their data bank of questions; others will not. Some service bureaus only operate on the Internet. Others will accept other forms of media, such as paper, scan forms, or CDs. They will often work with you in the design of your questionnaire and any process consulting you need (e.g., communication, policy issues, etc.), or they will simply act as a processor for you. Many firms use this mode of 360 feedback for their senior managers when they want to do a one-time project. Our firm has frequently supplied this service to organizations that want to run a pilot prior to purchasing a product we represent or for organizations that simply don’t have the time or resources to do it themselves.
Pros: Zero training and your administrative involvement is at a bare minimum. The service bureau handles all the administrative details. They'll set up the survey questions, invite the participants, collect the data, and generate and distribute the reports for you. You are often not restricted to standard reports. They often will customize reports and give you data exactly as you want it. The fact that the data is being collected and crunched by a third party may be appealing if you have confidentiality concerns about who sees the final reports. If you are a highly technical company where there are concerns that individual respondent data may be compromised, third-party service bureaus remove the confidentiality concerns. (In reality, however, ASPs or software programs are typically very secure and encrypt individual respondent data.)
Cons: Costs are typically higher because someone else is doing the labor to set up on the project, and you are using their system for data collection and web access. If you want special customization, be prepared to pay more. Depending on the service bureau, they may have a menu of charges and will charge you for every change. Others will have a package price to allow for these changes. Check your agreements carefully.
Conclusion
Both the original article and grid serve distinct purposes, and each will help you reach a conclusion. The HR Magazine article has been expanded and updated as part of another article we wrote for IPMA News (International Personnel Management Association). Subsequently, we have been invited by several vendors to represent their products. With our broad understanding of the capabilities of each, we can help you find the right solution for your needs. If you want more information or would like a demo on any of these products, we are pleased to offer our help on all of these products except ECustomä or Compassä because Right Associates is their exclusive representative.
Please note, we are able to help you and provide demos on all of these products except ECustom™ and Compass™. Our objective is to provide you with a solution to your 360 degree feedback needs so that you can sort through this with ease. Pricing will be the same whether you go directly to the software developer or take advantage of our service. With our broad understanding of the capabilities of each of those listed, we can help you find the right solution for your needs. You benefit because we are not focused on selling you one specific product. We listen to your needs and narrow down your choices to the one or two products that fit your budget and meet your requirements.
360
Degree Feedback Vendor
Feature Comparison Grid
By
N. Elizabeth Fried, Ph.D., CCP
760.633.4444
(Last Updated: 1/3/11)
Reminder: Be sure to use your frame arrows to enable you to view the entire grid horizontally--it is very very large. Please complete the request form at the bottom of this page, if you want to receive a PDF file of the grid that will print on regular size paper in landscape format. It is far more manageable. We will email it to you shortly.
