TRAINING Media Review
2003, Jeff Merrell
ADVISOR Enterprise 5.4
it’s all about the training budget. The wow factor of a slick new program
might set a roomful of training managers abuzz, but someone’s budget must
take a hit if the program is to have life. And the budget decision-making
process is usually where the buzz goes silent.
problem is that deciding between program delivery Option A, Option B, Option
C, or a combination is still a process based on assumptions. Without a method
to make assumptions explicit, organizations risk letting the buzz back into
the decision process.
Enterprise claims to resolve exactly this dilemma. ADVISOR does not provide a
rearview mirror to analyze the impact of training programs. It instead
supports an organization’s efforts to make decisions between competing
options before spending money. The idea behind ADVISOR is that smart,
consistent, and objective analysis of training alternatives before a program
is developed or purchased will lead to better budget and resource allocation
what becomes clear when you begin to look deeper into the tool is that it also
leads to continuously improving the decision-making process. Every time
training professionals put a bit of data into the system, they are making
their implicit understanding explicit—and recording their experience for the
next decision-making effort. In this sense, ADVISOR is really an expert system.
How it Works
key to understanding the value of ADVISOR is looking at how it imposes
discipline on the decision-making process. Historical data plays an important
role in this process, but the tool is designed to make explicit those implicit
lessons learned from history and then turn them into a useful format to support
decision-making on future programs.
product documentation positions the product as being built around four phases of
training analysis. Each leverages the actual data and assumptions built into the
tool by the client organization.
a rigorous baseline data-gathering exercise
using data-driven analysis to understand differences between alternatives
based on the organization’s own data and experience.
reviewing which options offer the greatest impact
the continuous process of reflecting on analyses as new information enters
this high-level starting point, ADVISOR then delivers a detailed decision-making
workflow that forces the user to make explicit all of the data and assumptions
required to create an objective, comprehensive analysis.
of the logic in the workflow is intuitive:
goals into instructional modules (optional)
effectiveness of plausible delivery options
direct and indirect costs
plausible delivery options
risk and compute hidden costs
the right blend of delivery options
Jay Bahlis—the brains behind ADVISOR—uses a combination of trusted
academic research and actual client feedback to build in an impressive level
of analytical detail. As you go through the steps in building the baseline
data, you walk away with a level of confidence that you are covering all of
the essential details. Existing ADVISOR customers interviewed for this review
support the point and recount stories in which specific details used in the ADVISOR
data-gathering steps led to important insights in choosing between
Delivery Option Decisions
in ADVISOR is a line of thinking that all delivery options supported by an
organization should be considered when evaluating how best to spend resources
on a new learning program. ADVISOR’s feasibility analysis helps an
organization discover the most economically effective delivery option or
combination of delivery options. Delivery options under consideration run the
gamut: instructor-led training; off-the-shelf or customized e-learning;
virtual classroom; print; video-teleconference; etc.
actually does the work to analyze plausible delivery options after users
define requirements related to content, audience, environment, hardware, and
software. Behind each of these requirements is a detailed set of questions
designed to capture assumptions. For example, the audience requirement records
the client organization’s assessment of the target audience’s resistance
to change; their level of motivation; computer skills; reading skills; ability
to travel; etc.
completing the requirements assessment, all delivery options are ranked
according to a numerical rating that aggregates the assessment data. Delivery
options above a minimum score level are highlighted as “recommended.”
Delivery options that cannot meet one or more of the requirements defined as
“critical” receive a zero score. Users can drill down and see which
specific data points helped contribute to the aggregate score or the minimum
score level and make adjustments if the rating is intuitively inaccurate. This
adjustment capability is one of ADVISOR’s most useful features, making it in
fact an expert system based on the client’s explicit decision criteria. The
more you tweak the information that ADVISOR uses in its logic, the more you
turn the tool into a representation of the collective decision-making
intelligence of your organization.
feasibility analysis also takes into consideration the option of using
multiple delivery modes to create a single learning event or activity (e.g.,
blended learning). It accomplishes this by defining a standard approach to
dividing a course into component parts that can then be analyzed as discrete
chunks of learning independent of the aggregate “course.” ADVISOR’s
approach is to divide a course into “modules” designed to deliver one of
five learning outcomes:
comprehension, and application
project management course, for example, may include the following:
||Define and explain the major phases of a project
||Knowledge, comprehension and application
||Break down a high-level project plan into specific tasks
||Use a project management software application to create
a project plan
user would then set specific requirements for each module using the same
detailed subcategories as used for the full course level: content, audience,
environment, hardware, and software. Plausible delivery options can then be
compared at the module level.
defining plausible delivery options, the remainder of the analysis focuses on
assessing direct, indirect, and hidden costs of each option. Direct and
indirect costs categories include:
(e.g., conferencing, Internet)
(e.g., course updates)
of these cost categories includes a worksheet-style list of sub-items designed
to accurately calculate true costs. In some cases, ADVISOR provides data that
may be useful in completing the calculations (e.g., estimated number of hours
required to develop an e-learning course of a specific length), but the user
always has the option to override the data with data from their own experience.
final element of the ADVISOR cost analysis is a look at hidden costs
associated with implementing new learning delivery technology. Again, ADVISOR
sets up the framework for a disciplined approach to capturing estimated cost
of new technology implementation, administration, and associated change
the end of this effort, the organization can rate plausible delivery options
in two ways:
“best value” method that gives the highest rating to the option
meeting all requirements at the lowest overall cost or
“best dollar per point” method that gives more weight to those options
scoring higher on requirements factors
both methods, ADVISOR provides multiple views of the aggregated cost data. You
can filter data to look at direct costs or indirect costs or view the
distribution of costs over the expected lifecycle of the program. In a blended
learning scenario, ADVISOR will highlight a recommended blend based on
aggregating cost data from course modules but also allows comparison to
alternative blends defined by the user.
Enterprise has some history in helping the defense industry organize its
decision-making criteria before making a purchasing choice. It is a tool
clearly developed to work in an environment where decisions may be based on
complex criteria and
where budget scrutiny is intense.
commercial businesses, ADVISOR will be an effective tool if the organization
is willing to put in some effort (e.g., identifying hidden costs). The
system’s user interface is functional but will require some time to learn.
Although there are some shortcuts to conduct an analysis, the tool is much
more valuable if you spend some time embedding your own expertise into its
models and calculations. It will also take some time to really begin to
understand the level of detail that Jay Bahlis has meticulously woven into the
logic of the system.
even with these hidden costs, ADVISOR may be an excellent investment for many
organizations. If your organization is one that spends time developing
detailed Requests for Proposals for training programs, you may find that much
of the information you use in the RFP process will flow easily into ADVISOR.
That leaves the direct cost of ADVISOR licenses, and an entry point for all
three ADVISOR modules is $9,000, a sum that could clearly be recouped in a
single improved decision.
value comes from turning it into your own expert system. Many organizations
issue RFPs and analyze the responses as if they were individual, unconnected
events. If there is consistency, it is because the same training staff are
involved in the process. ADVISOR provides the opportunity to codify all of
your experience into a single, sharable resource based on well-grounded
research and practices. Kudos to Jay Bahlis for raising the level of
professionalism of an inherently difficult task.
|ADVISOR Enterprise 5.4
|Ease of Use
|Value of Purpose
|Value for the Money
From Training Media Review, http://www.tmreview.com. Copyright © 2003, TMR Publications. Reprinted by permission.