Page 303 - Mechanical Engineers' Handbook (Volume 2)

294 Analysis, Design, and Information Processing

3. Determining the effectiveness of the support in terms of these criteria
4. Determining the most useful strategy for employment of an existing system and
potential improvements such that effectiveness of the newly implemented system and
the overall process might be improved

Figure 8 illustrates a partial intent structure or objectives tree, which contributes to system
evaluation. The lowest level objectives contribute to satisfaction of the 10 performance ob-
jectives for systems engineering and systems design outlined in Section 3. These lowest level
elements form pertinent criteria for the operational system evaluation. They concern the
algorithmic effectiveness or performance objective achievement of the system, the behavioral
or human factor effectiveness of the system in the operational environment, and the system
efficacy. Each of these three elements become top-level criteria or attributes and each should
be evaluated to determine evaluation of the system itself.
Subcriteria that support the three lowest level criteria of Fig. 8 may be identified. These
are dependent on the requirements identified for the specific system that has been designed.
Attainment of each of these criteria by the system may be measured by observation of the
system within the operational environment and by test instruments and surveys of user groups
involved with the operational system and process.

Algorithmic Effectiveness of Performance Objectives Achievement Evaluation
A number of performance objectives can be cited that, if achieved, should lead to a quality
system. Achievement of these objectives is measured by logical soundness of the operational
system and process; improved system quality as a result of using the system; and improve-
ments in the way an overall process functions, compared to the way it typically functions
without the system or with an alternative system.

Behavioral or Human Factors Evaluation
A system may be well structured algorithmically in the sense of achieving a high degree of
satisfaction of the performance objectives, yet the process incorporating the system may
seriously violate behavioral and human factor sensibilities. This will typically result in misuse
or underuse. There are many cases where technically innovative systems have failed to
achieve broad scope objectives because of human factor failures. Strongly influencing the
acceptability of system implementation in operational settings are such factors as organiza-
tional slack; natural human resistance to change; and the present sophistication, attitude, and
past experience of the user group and its management with similar systems and processes.

Figure 8 Objectives tree for evaluation of deci-
sion support system.