ADVISOR Enterprise

    Strategic Training Planning System

TRAINING Media Review

2003, Jeff Merrell 

ADVISOR Enterprise 5.4

Ultimately, it’s all about the training budget. The wow factor of a slick new program might set a roomful of training managers abuzz, but someone’s budget must take a hit if the program is to have life. And the budget decision-making process is usually where the buzz goes silent.

The problem is that deciding between program delivery Option A, Option B, Option C, or a combination is still a process based on assumptions. Without a method to make assumptions explicit, organizations risk letting the buzz back into the decision process.

ADVISOR Enterprise claims to resolve exactly this dilemma. ADVISOR does not provide a rearview mirror to analyze the impact of training programs. It instead supports an organization’s efforts to make decisions between competing options before spending money. The idea behind ADVISOR is that smart, consistent, and objective analysis of training alternatives before a program is developed or purchased will lead to better budget and resource allocation decisions.

But what becomes clear when you begin to look deeper into the tool is that it also leads to continuously improving the decision-making process. Every time training professionals put a bit of data into the system, they are making their implicit understanding explicit—and recording their experience for the next decision-making effort. In this sense, ADVISOR is really an expert system.

How it Works

The key to understanding the value of ADVISOR is looking at how it imposes discipline on the decision-making process. Historical data plays an important role in this process, but the tool is designed to make explicit those implicit lessons learned from history and then turn them into a useful format to support decision-making on future programs.

ADVISOR product documentation positions the product as being built around four phases of training analysis. Each leverages the actual data and assumptions built into the tool by the client organization.

• Discover: a rigorous baseline data-gathering exercise
• Strategize: using data-driven analysis to understand differences between alternatives based on the organization’s own data and experience.
• Prioritize: reviewing which options offer the greatest impact
• Optimize: the continuous process of reflecting on analyses as new information enters the system

From this high-level starting point, ADVISOR then delivers a detailed decision-making workflow that forces the user to make explicit all of the data and assumptions required to create an objective, comprehensive analysis.

Much of the logic in the workflow is intuitive:

• List instructional goals
• Group goals into instructional modules (optional)
• Evaluate effectiveness of plausible delivery options
• Estimate development time
• Compute direct and indirect costs
• Rate plausible delivery options
• Assess risk and compute hidden costs
• Determine the right blend of delivery options

But Jay Bahlis—the brains behind ADVISOR—uses a combination of trusted academic research and actual client feedback to build in an impressive level of analytical detail. As you go through the steps in building the baseline data, you walk away with a level of confidence that you are covering all of the essential details. Existing ADVISOR customers interviewed for this review support the point and recount stories in which specific details used in the ADVISOR data-gathering steps led to important insights in choosing between alternatives.

Delivery Option Decisions

Embedded in ADVISOR is a line of thinking that all delivery options supported by an organization should be considered when evaluating how best to spend resources on a new learning program. ADVISOR’s feasibility analysis helps an organization discover the most economically effective delivery option or combination of delivery options. Delivery options under consideration run the gamut: instructor-led training; off-the-shelf or customized e-learning; virtual classroom; print; video-teleconference; etc.

ADVISOR actually does the work to analyze plausible delivery options after users define requirements related to content, audience, environment, hardware, and software. Behind each of these requirements is a detailed set of questions designed to capture assumptions. For example, the audience requirement records the client organization’s assessment of the target audience’s resistance to change; their level of motivation; computer skills; reading skills; ability to travel; etc.

After completing the requirements assessment, all delivery options are ranked according to a numerical rating that aggregates the assessment data. Delivery options above a minimum score level are highlighted as “recommended.” Delivery options that cannot meet one or more of the requirements defined as “critical” receive a zero score. Users can drill down and see which specific data points helped contribute to the aggregate score or the minimum score level and make adjustments if the rating is intuitively inaccurate. This adjustment capability is one of ADVISOR’s most useful features, making it in fact an expert system based on the client’s explicit decision criteria. The more you tweak the information that ADVISOR uses in its logic, the more you turn the tool into a representation of the collective decision-making intelligence of your organization.

ADVISOR’s feasibility analysis also takes into consideration the option of using multiple delivery modes to create a single learning event or activity (e.g., blended learning). It accomplishes this by defining a standard approach to dividing a course into component parts that can then be analyzed as discrete chunks of learning independent of the aggregate “course.” ADVISOR’s approach is to divide a course into “modules” designed to deliver one of five learning outcomes:

• Knowledge, comprehension, and application
• Problem solving
• Psychomotor skills
• Affective-attitude
• Software skills

A project management course, for example, may include the following:

A user would then set specific requirements for each module using the same detailed subcategories as used for the full course level: content, audience, environment, hardware, and software. Plausible delivery options can then be compared at the module level.

Analyzing Costs

After defining plausible delivery options, the remainder of the analysis focuses on assessing direct, indirect, and hidden costs of each option. Direct and indirect costs categories include:

• Program development
• Hardware
• Administration
• Management
• Trainee
• Instructor
• Facility
• Transmission (e.g., conferencing, Internet)
• Maintenance (e.g., course updates)
• Technical support

Each of these cost categories includes a worksheet-style list of sub-items designed to accurately calculate true costs. In some cases, ADVISOR provides data that may be useful in completing the calculations (e.g., estimated number of hours required to develop an e-learning course of a specific length), but the user always has the option to override the data with data from their own experience.

The final element of the ADVISOR cost analysis is a look at hidden costs associated with implementing new learning delivery technology. Again, ADVISOR sets up the framework for a disciplined approach to capturing estimated cost of new technology implementation, administration, and associated change management efforts.

At the end of this effort, the organization can rate plausible delivery options in two ways:

• A “best value” method that gives the highest rating to the option meeting all requirements at the lowest overall cost or
• A “best dollar per point” method that gives more weight to those options scoring higher on requirements factors

In both methods, ADVISOR provides multiple views of the aggregated cost data. You can filter data to look at direct costs or indirect costs or view the distribution of costs over the expected lifecycle of the program. In a blended learning scenario, ADVISOR will highlight a recommended blend based on aggregating cost data from course modules but also allows comparison to alternative blends defined by the user.

ADVISOR Enterprise has some history in helping the defense industry organize its decision-making criteria before making a purchasing choice. It is a tool clearly developed to work in an environment where decisions may be based on complex criteria and where budget scrutiny is intense.

For commercial businesses, ADVISOR will be an effective tool if the organization is willing to put in some effort (e.g., identifying hidden costs). The system’s user interface is functional but will require some time to learn. Although there are some shortcuts to conduct an analysis, the tool is much more valuable if you spend some time embedding your own expertise into its models and calculations. It will also take some time to really begin to understand the level of detail that Jay Bahlis has meticulously woven into the logic of the system.

But even with these hidden costs, ADVISOR may be an excellent investment for many organizations. If your organization is one that spends time developing detailed Requests for Proposals for training programs, you may find that much of the information you use in the RFP process will flow easily into ADVISOR. That leaves the direct cost of ADVISOR licenses, and an entry point for all three ADVISOR modules is $9,000, a sum that could clearly be recouped in a single improved decision.

Recommendation

ADVISOR’s value comes from turning it into your own expert system. Many organizations issue RFPs and analyze the responses as if they were individual, unconnected events. If there is consistency, it is because the same training staff are involved in the process. ADVISOR provides the opportunity to codify all of your experience into a single, sharable resource based on well-grounded research and practices. Kudos to Jay Bahlis for raising the level of professionalism of an inherently difficult task.

From Training Media Review, http://www.tmreview.com. Copyright © 2003, TMR Publications. Reprinted by permission.