Page 289 -

272 Chapter 10 Sociotechnical systems

of the system is unpredictable. This is not a problem in itself but, from a dependabil-
ity perspective, it can make it difficult to decide whether or not a system failure has
occurred, and to estimate the frequency of system failures.
For example, say a system is presented with a set of 20 test inputs. It processes
these inputs and the results are recorded. At some later time, the same 20 test inputs
are processed and the results compared to the previous stored results. Five of them
are different. Does this mean that there have been five failures? Or are the differ-
ences simply reasonable variations in the system’s behavior? You can only find this
out by looking at the results in more depth and making judgments about the way the
system has handled each input.

10.1.3 Success criteria
Generally, complex sociotechnical systems are developed to tackle what are some-
times called ‘wicked problems’ (Rittel and Webber, 1973). A wicked problem is a
problem that is so complex and which involves so many related entities that there is
no definitive problem specification. Different stakeholders see the problem in differ-
ent ways and no one has a full understanding of the problem as a whole. The true
nature of the problem may only emerge as a solution is developed. An extreme
example of a wicked problem is earthquake planning. No one can accurately predict
where the epicenter of an earthquake will be, what time it will occur, or what effect
it will have on the local environment. It is impossible to specify in detail how to deal
with a major earthquake.
This makes it difficult to define the success criteria for a system. How do you
decide if a new system contributes, as planned, to the business goals of the company
that paid for the system? The judgment of success is not usually made against the
original reasons for procuring and developing the system. Rather, it is based on
whether or not the system is effective at the time it is deployed. As the business envi-
ronment can change very quickly, the business goals may have changed significantly
during the development of the system.
The situation is even more complex when there are multiple conflicting goals that
are interpreted differently by different stakeholders. For instance, the system on
which the MHC-PMS (discussed in Chapter 1) is based was designed to support two
distinct business goals:

1. Improve the quality of care for sufferers from mental illness.
2. Increase income by providing detailed reports of care provided and the costs of
that care.

Unfortunately, these proved to be conflicting goals because the information
required to satisfy the reporting goal meant that doctors and nurses had to provide
additional information, over and above the health records that are normally main-
tained. This reduced the quality of care for patients as it meant that clinical staff had