Page 301 - Mechanical Engineers' Handbook (Volume 2)

292 Analysis, Design, and Information Processing

1. To identify a methodology for evaluation
2. To identify criteria on which the success of the system or process may be judged
3. To determine effectiveness of the system in terms of success criteria
4. To determine an appropriate balance between the operational environment evalua-
tion and the control group evaluation
5. To determine performance objective achievement of the system
6. To determine behavioral or human factor effectiveness of the system
7. To determine the most useful strategy for employment of the existing system
8. To determine user group acceptance of the system
9. To suggest refinements in existing systems for greater effectiveness of the process
in which the new system has been embedded
10. To evaluate the effectiveness of the system or process

These objectives are obtained from a critical evaluation issue specification or evaluation need
specification, which is the first, or problem definition, step of the evaluation methodology.
Generally, the critical issues for evaluation are minor adaptations of the elements that are
present in the requirements specifications step of the design process outlined in the previous
section. A set of specific evaluation test requirements and tests is evolved from these objec-
tives and needs. These must be such that each objective measure and critical evaluation issue
component can be determined from at least one evaluation test instrument.
If it is determined that the system and the resulting process support cannot meet user
needs, the systems design process iterates to an earlier phase and development continues.
An important by-product of evaluation is the determination of ultimate performance limita-
tions and the establishment of a protocol and procedure for use of the system that results in
maximum user group satisfaction. A report is written concerning results of the evaluation
process, especially those factors relating to user group satisfaction with the designed system.
The evaluation process should result in suggestions for improvement in design and in better
methodologies for future evaluations.
Section 5.6 will present additional details of the methodologies framework for evalua-
tion. These have applicability to cases where evaluation is a separate and independent effort
as well as cases where it is one of the phases of the design process.

Operational Deployment Phase
The last phase of design concerns operational deployment and final implementation. This
must be accomplished in such a way that all user groups obtain adequate instructions in use
of the system and complete operating and maintenance documentation and instructions. Spe-
cific objectives for the operational deployment phase of the system design effort are as
follows:
1. To enhance operational deployment
2. To accomplish final design of the system
3. To provide for continuous monitoring of post implementation effectiveness of the
system and the process into which the system is embedded
4. To provide for redesign of the system as indicated by effectiveness monitoring
5. To provide proper training and leadership for successful continued operational use
of the system