PRODUCT-BASED MIS EVALUATION
Since the focus of the product-based evaluation is on the product
(information support) or the output from the system, the evaluation may be
termed as effectiveness evaluation. For assessing the effectiveness of output
from MIS, the following model may be used.
Model structure The information attributes may be identified as
components of a general model for evaluation of MIS effectiveness in an
organization. Some of these attributes are listed below.
(i) Timeliness
(ii) Relevance
(iii) Accuracy
(iv) Completeness
(v) Adequacy
(vi) Explicitness
(vii) Exception-based. Mode! Implementation
Various types of outputs/reports, being generated by MIS of the organization
can be evaluated for their effectiveness in terms of the attributes of the
management information. The attributes of information have been listed in
the structure of the model as mentioned above. To employ this model,
managers at different levels of management of the organization may be
asked to rate the outputs/reports on each of the information attributes.
To get responses, a five-point scale may be used on which the respondents
(users of information systems) may be asked to rate the effectiveness of
MIS in terms of these information attributes. The rating is based on the
number of the reports/outputs which observe the information attributes. For
example, a five-point scale may be prepared to get an evaluation of the
number of reports received by the managers in terms of 'Timeliness', as
given below.
The scale thus prepared is to be administered either through a mailed
questionnaire or through a personal interview and the scoring may be done
by assigning a numerical value of 0 to the least favourable location on the
scale, 1 to the next favourable, and so on.
COST/BENEFIT-BASED EVALUATION
In cost/benefit evaluation, a thorough study of various expected costs, the
benefits to be expected from the system and expected savings, if any, is
done. It is an economic evaluation of the system, in which costs to be
incurred for developing, implementing and operating a system are to be
justified against the expected benefits from the system. In other words,
cost/benefit analysis determines the cost-effectiveness of the system.
For undertaking cost/benefit evaluation, various estimates of costs as well as
benefits expected from the system are to be made. In developing cost
estimates for a system, several cost elements are considered. Among them
are initial development costs, capital costs, operating costs, etc. Similarly
expected benefits from the system are considered. The benefits may be in
terms of reduced cost, better performance/decisions, etc. The various
categories of costs and benefits are measured and included in cost/benefit
analysis. A brief description of all these cost elements and benefits is given
below.
Initial development cost
Initial development cost is the cost incurred in developing an information
system. Various elements of development cost include project planning cost,
feasibility study cost, design cost, conversion cost, implementation cost
(including user training cost, testing costs, etc.). In other words, total
development cost is considered one-time cost and is termed as initial
development cost.
Capital cost
Capital cost is also one-time cost. It is the cost incurred in facilities and in
procuring various equipment, including hardware, etc., required for the
operation of the system. Facility costs are expenses incurred in the
preparation of the physical site where the system will be implemented. It
includes wiring, flooring, lighting and air-conditioning cost. The cost on space
required for office, storage and computer room, if not hired, is also included
in the facility cost. Hardware and equipment cost relates to the actual
purchase or lease of the computer and peripherals.
Annual operating cost
Annual operating cost is the cost incurred in operating the system. It
includes computer and equipment maintenance cost, personnel cost,
overheads and supplies cost. Computers and equipment are to be
maintained and thus some cost is incurred, known as Annual Maintenance
Cost (AMC). Similarly, personnel are required to operate the system.
Personnel cost includes EDP staff salaries and other benefits (provident fund,
health insurance, vacation time, pensionary benefits, etc.). Overhead costs
include all costs associated with the day-to-day operation of the system; the
amount depends on the number of shifts, the nature of the applications, and
capabilities of the operating staff. Supply costs are variable costs that
increase with increased use of paper, ribbons, disks, etc.
Just as the cost elements, in cost/benefit evaluation, various expected.
benefits from the system are also studied. The first task is to identify each
benefit and then assign a monetary value to it. Benefits may be tangible or
intangible, direct or indirect.
There are two major benefits, namely, improving performance and
minimizing the cost of processing. The performance part suggests
improvement in the accuracy, timeliness, non-duplication, adequacy,
usefulness in information and easier access to the system by authorized
users; which in turn leads to better decisions and allows more time to
managers for planning purposes, etc. Minimizing costs through an efficient
system, such as error control, reduced salary and labour cost and reduced
inventory cost is a benefit that is to be measured for evaluating costeffectiveness of a system.
For identification and categorization of various costs and benefits, the
following concepts are important.
Identification of costs and benefits
Certain costs and benefits are more easily identifiable than others. For
example, direct costs, such as the price of a personal computer, ribbon, etc.,
are easily identified from invoices or from organizational records. Similarly,
direct benefits such as reduction in staff because of the new system or fast
processing of transactions, may be identified. Other direct costs and
benefits, however, may not be well-defined, since they represent estimated
costs or benefits that are not very certain or well-defined. An example of
such a cost is a reserve for bad debt.
Classification of costs and benefits
The various categories of costs and benefits are important to make a
cost/benefit analysis. These categories may be tangible or intangible, direct
or indirect, fixed or variable.
Tangibility refers to the ease with which costs and benefits can be identified
and measured. Cost incurred or to be incurred on a specific item or activity
is termed as a tangible cost. For example, computer cost, consultancy fee
paid to a consultant and employee salary are tangible costs.
Intangible costs are those costs that are known to exist but whose monetary
value cannot be accurately measured. For example, lowered employee
morale because of a new system is an intangible cost.
Like costs, benefits may also be categorized as tangible or intangible.
Tangible benefits such as reduced salaries, producing reports with no errors
are quantifiable.
Intangible benefits, such as high morale among employees, improved
organizational image are not easily quantified.
Costs are also categorized as direct or indirect costs.
Direct costs are those with which an amount in rupees can be directly
associated to any of the items or operations of the system. For example, the
purchase of a computer ribbon for Z 3500 is a direct cost.
Direct benefits also can be identified which could be attributed to the new
system. For example, a 5 per cent reduction in salary expenditure because
of the new system can be classified as a direct benefit.
Indirect costs are the results of operations that are not directly associated
with the system or activity. They are termed as overheads. For example,
safety or security of computer room, electricity, air conditioning and
maintenance, etc., are included in indirect costs.
Similarly, indirect benefits are realized as a by-product of some other
activity or system. For example, newly computerized salary system provides
information on the total amount required for disbursements and total
deductions to be made under various heads such as insurance, provident
fund, recovery from loan advances, etc. Information about the amount
recovered from loan advances becomes an indirect benefit of the salary
system as the management would be able to properly utilize the amount and
thus can earn maximum returns.
Fixed costs are constant costs and do not change, regardless of how well a
system is used. They are only one-time costs such as development cost,
capital and insurance cost, etc.,
Variable costs are incurred on a regular basis: They are usually proportional
to work volume and continue as long as the system is in operation. For
example, the cost of supplies depends upon the size and volume of
reports/processing work.
Fixed benefits are also constant and do not change. For example, 10 per
cent reduction in staff as a result of the new system is a fixed benefit.
Variable benefits, on the other hand, are realized on a regular basis. For
example, the amount of daily time saved of a manager varies with the
number and types of decisions taken.