Multimedia & Internet Training Newsletter
Needs Analysis Tool Review: ADVISOR 2.0
Front-End Decisions That
Reduce Tail-End Discomfort
By Cathi L. Billings, MS
Instructional Systems Specialist
Training Technology Flight,
Sheppard Air Force Base, TX
Managing multimedia development and capturing associated costs have never been easy or "easy to swallow" for that matter. In fact, research from the
reflects that "31 percent of all software projects are canceled each year. Over 52 percent of projects will cost 189 percent of their original estimates. Only 16 percent of projects are completed on time and within budget." These dismal statistics remind us of the turbulent nature of technology and the need for flexibility in project management. Traditional ROI justifications have been replaced with emphasis on organizational image or innovative approaches to training. Industry leaders at a recent Intranet-Based Learning Conference openly admitted their inability to accurately capture cost and project management data. Thus, a growing number of companies are chasing the latest technology without a clear knowledge of the cost, or even a simple media comparison. The Department of Defense, with its shrinking budget, does not have that luxury. An extensive front-end analysis addressing feasibility, media selection, and cost comparisons is required before beginning a multimedia development project.
The Training Technology Flight located at Sheppard Air Force Base, Wichita Falls, Texas, is a support function of the United States Air Force, Air Education and Training Command. The mission of the flight is to provide cost-effective training, analyze courses for conversion, and maximize the use of training technologies to support over 5,000 trainees per month. Instructional systems development (ISD) is practiced in its entirety, although only the multimedia specific front-end analysis portion will be address in this article.
Experienced instructional systems specialists manage the increasing complexity of decisions required to analyze multiple/mixed media training options. Selecting the appropriate mix of media is often difficult with constraints such as hardware, bandwidth, and the many operational locations and skill levels of the learners. Selecting the appropriate media while reducing costs per trainee is a daily challenge.
The Training Technology Flight, along with counterparts across the Air Force and Department of Defense, is using an off-the-shelf decision support tool called Advisor 2.0. After extensive data collection and research, a costing template was created that reflects all processes and decisions related to use of any media in an instructional setting. With these templates, static information does not have to be entered with each analysis. Only course specific information has to be entered by training managers or course experts. Other entries are computed from data already entered, or from training development and implementation benchmarks in the rule-based system.
Course experts can work within the software program, or can fill in the various worksheets associated with it. The results are summarized and can be printed in graphic and text formats. The software stores each course analysis as a separate file. These files can be used over time to reconcile planned with actual data, as well as to develop benchmarks for statistics like average costs, average development, time, or total corporate savings from technology-based training solutions. Performing initial front-end analyses in this manner simplifies number crunching because of pre-filled templates, thus standardizing costing predictions across the Department of Defense.
Utilizing a tool such as Advisor has been beneficial in many ways. It has become the business case/presentation tool of choice for many; it is invaluable as a costing prediction tool when submitting proposals; and it is an excellent educational experience for new users not familiar with the intricacies of developing courses in differing formats. With all media options available, users can immediately see which options are not viable, based on course constraints. Help buttons provide clarification for each field, and the notes can be customized for more specific guidance.
The use of a standardized data collection and reporting tool was a deliberate attempt to increase return-on-investment. The front-end analysis process can typically take anywhere from 30 to 90 days to complete. A team of course experts and instructional specialists can perform an accelerated analysis, generate the necessary reports, and make informed decisions in one day. With all parties involved in front-end analysis decision making, tail-end discomfort is kept to a minimum. The number of re-starts also decreases through this education process (for every 100 projects, there are 94 that restart). Although this step is just one small part of overall project management, it's a powerful tool when used to compare media requirements, determine feasibility, and estimate ROI. The team is then on its way to compete for the 16 percent of project completed on time and with budget.
Based on user input, this screen displays each of the delivery methods/media in rank order of how well suited they are to the training challenge. The horizontal bars have a percentage rating that allows for quantitative comparisons. When the score for any delivery method is below 100 percent, the question mark icon beside each option can be clicked to have the screen on the right indicate why the effectiveness of each media option is less than perfect. As the scoring is relative, even the top-rated delivery medium may still involve considerable challenges or have specific shortcomings.
Training Cost Analysis
The Costs screen provides a cost comparison between various delivery media broken down into the various key components of cost. It provides a total cost and a cost per trainee for each method of instruction. Cost can also be depicted graphically.