DERA - Ministry
of Deference
Grant Reference:CU003
- 0000000860
PROJECT:
Consultancy
Report on
'Investment
Evaluation of
IT/IS: Research
and Application'
for DERA/Ministry
of Defence
Executive
Summary
The literature
shows that the
increasing
percentage of
capital
investments in
Information
Technology (IT)
and Information
Systems (IS)
without
commensurate
returns is
forcing a
reconsideration
of how to
justify
expenditure on
technology-driven
business process
change. However,
such
organisations
are reporting an
inability to
adequately
evaluate these
investments.
A
better
understanding of
the impact IT/IS
has on
organisational
performance can
help an
organisation
better utilise
resources and
improve its
position
vis-à-vis its
competitors.
Viewed in simple
system dynamics
terms,
evaluation
provides the
basic feedback
function to
managers as well
as forming a
fundamental
component of the
organisational
learning
process.
Finally,
evaluation
provides the
benchmarks of
what is to be
achieved by the
information
systems
investment.
These benchmarks
can later be
used to provide
a measure of the
actual
implementation
success of the
information
system.
Much of this
difficulty
centres around
the range and
complexity of
managerial and
technological
knowledge needed
to handle the
multiplicity of
justification
and
implementation
paths. IT and IS
investment
yields a mixture
of strategic,
tactical and
operational
benefits and it
is argued that
there are a
variety of
project types
each needing
different
approaches to
their
evaluation. In
each case gains
may be
financially
quantifiable,
non-financially
quantifiable and
intangible. The
full range of
hidden costs,
both direct and
indirect, are
also equally
difficult to
quantify.
Appraisal
processes are
not "one stop
decisions". From
their sources,
often at the
roots of an
organisation,
proposals go
through stages
of development,
filtering and
aggregation
before they
become part of
the capital
budget.
There is no
single simple
technique that
can reasonably
be applied to
IT/IS
evaluation. Both
ratio and
discounted
economically
based assessment
techniques loose
their appeal of
simplicity and
objectivity as
more and more
factors are
added to cover
risks and
intangible
benefits. On the
other hand
strategic
assessment
techniques omit
the essential
element of
financial
viability
necessary for
corporate
survival. The
more
comprehensive
portfolio and
integrated
assessment
techniques are
complex to apply
and remain
heavily
dependant
subjective
judgement.
Clearly, placing
investment
evaluation in a
positive light
is synonymous to
best practice
management
however, much of
the normative
literature
suggests that
organisations
often adopt a
less than
structured
approach to the
justification
process. Many
companies pursue
one of the
following
strategies (i)
Refusal to
undertake IT/IS
projects whose
benefits and
costs are not
easily
financially
quantifiable;
(ii) Invest in
IT/IS projects
as an act of
faith; or, (iii)
Use creative
accounting as a
means of passing
the budgetary
process.
There are
numerous
management
concerns about
investment
appraisal.
However,
regardless of
the complexity
involved it is
important to
justify IT/IS
investments
before
committing time
and money to
implementation
and risking
failures leading
to loss and
competitive
disadvantage.
Competition for
scarce
organisational
resources often
forces choices
to be made
between
heterogeneous
investment
proposals
without knowing
the full impact
each may have on
organisational
performance.
Under pressure
the decision
processes are
more often
characterised by
limited
commitment of
organisational
resources,
myopia
surrounding
technology
related change
and the need to
show quick
financial
returns. This
short term view
can prejudice
investment
decisions
against staged,
or modular,
system
development - a
practice which
is almost
totally ignored
within the
literature on
appraisal
mechanisms.
This analysis is
supported by
evidence from
five case
studies
examining
reports of
real-life
information
systems
investments.
These show the
variety of
justification
approaches that
these
organisations
used in practice
and highlight
their
weaknesses.
The first case
study is of a
local authority
where middle
management took
a strategic
decision to
introduce an
office
automation
system to
integrate their
disparate
departmental
systems, working
environments and
business
cultures. Only a
small-scale
requirements
specification
study was
undertaken,
there was no
monitoring of
the
implementation
stage, and no
post-implementation
review was
conducted.
Associated costs
such as training
and support were
also omitted
from the
original budget.
As a result the
selected product
that lacked some
of the
fundamental
requirements and
this problem was
not detected or
remedied either
during or after
implementation.
The lack of
adequate
evaluation meant
that the system
lacked the
expected impact,
the true costs
and benefits of
the system were
never fully
understood and
that lessons for
future
investments were
missed.
The second case
looks at a new
pensions system
in an insurance
company. This
development
coincided with a
restructuring of
the IT
department and
throughout there
was a lack of
effective
project
management.
Although the
project arose
from a user
driven steering
committee there
appeared to be
very little, if
any, attempt at
justifying the
investment.
Changing
committee
membership and a
lack of
consensus
between
individuals and
departments lead
to the repeated
adoption of
compromise
solutions.
Although the
change factors
(internal
restructuring,
new market
opportunities
and changing
regulations)
were realised,
they were not
managed to
prevent
excessive,
incompatible and
infeasible
demands being
made on and
accepted by the
IT department.
The inevitable
result was
disillusionment,
delivery of low
quality of the
systems with
cost and time
overruns.
The third case
study was
concerned with a
system to
provide
Electronic Data
Interchange
(EDI) between
two Canadian
financial
organisations.
The investment
was evaluated
entirely on a
financial basis,
although softer
benefits were
mentioned to
strengthened the
arguments for
adoption. The
decision to
proceed was
based solely on
an economic
feasibility
analysis and the
ex ante analysis
covered only
hard costs and
benefits.
Despite a
projected
payback period
of one year, the
company made a
significant loss
in the first
year of system
operation.
Although there
were flaws in
the
calculations, a
more critical
oversight was
the failure to
assess the
broader impact
of the system.
The low level of
integration
created new
manual
processing tasks
and others, that
it had been
assumed would be
eliminated, had
to be retained.
The fourth case
shows a
successful
project in a
bank using IS to
leverage the
business
opportunities in
the 'home
banking' market.
The report deals
with ex post
evaluation to
assess whether
the planned
benefits were
actually
realised and
actively
managed. This
evaluation
addressed three
distinct levels:
financial
evaluation,
quality
assessment, and
strategic
appraisal. This
emphasises the
range of
perspectives
from which an
information
system can be
assessed. Some
lend themselves
naturally to
financially
techniques,
while others
cannot
legitimately be
measured in this
way. Since some
perspectives may
also show
benefits that
fall into both
categories this
case shows how a
single
evaluation
technique would
be inadequately
to assess all
the benefits and
drawbacks of an
investment.
The last study
refers to the
process of
customer order
fulfilment and
two
collaborating
organisations
aiming, by
redesign and
interconnection
of their
systems, to
reduce the
average time to
process an
order. An
in-depth study
of the processes
was conducted
using Business
Process
Modelling and
Simulation. This
was motivated by
the need to
generate the
data that would
feed a formal
cost/benefit
analysis in an
objective
manner. It also
allowed
exploration and
extension of the
initial
assumptions
about projected
savings and
costs. The
application of
simulation
proved to be a
valuable
mechanism for
realising the
real business
value of the
proposed
IT-based
solution.
Individuals were
able to see and
assess the costs
and benefits
associated with
the proposed
change giving
them confidence
in the
technology
without the
risks, costs
disruption of an
implementation
project. It also
showed that the
adoption of an
IT-based system
alone would not
improve the
order fulfilment
process to the
extent expect.
The methods used
in this last
case showed
several
beneficial side
effects.
Firstly, it
brought
participants
from different
levels in both
companies to
work together
towards a
solution that
would be of
mutual benefit.
Second,
facilitated
communication
between business
and IS
specialists.
Thirdly, it
structured the
debate and
speeded reaching
a consensus. In
many respects
this illustrates
the multi
faceted nature
of IT/IS
evaluation and
the strong
relationships to
the views of
particular
stakeholders in
any particular
project.
£
Value:
£22, 243
Investigators:
Professor Zahir
Irani,
Dr Tony Elliman,
Professor Guy
Fitzgerald
