Page 247 -

230 Chapter 8 Software testing

4. Run acceptance tests The agreed acceptance tests are executed on the system.
Ideally, this should take place in the actual environment where the system will
be used, but this may be disruptive and impractical. Therefore, a user testing
environment may have to be set up to run these tests. It is difficult to automate
this process as part of the acceptance tests may involve testing the interactions
between end-users and the system. Some training of end-users may be required.
5. Negotiate test results It is very unlikely that all of the defined acceptance tests will
pass and that there will be no problems with the system. If this is the case, then
acceptance testing is complete and the system can be handed over. More com-
monly, some problems will be discovered. In such cases, the developer and the
customer have to negotiate to decide if the system is good enough to be put into
use. They must also agree on the developer’s response to identified problems.
6. Reject/accept system This stage involves a meeting between the developers
and the customer to decide on whether or not the system should be accepted. If
the system is not good enough for use, then further development is required
to fix the identified problems. Once complete, the acceptance testing phase is
repeated.

In agile methods, such as XP, acceptance testing has a rather different meaning. In
principle, it shares the notion that users should decide whether or not the system is
acceptable. However, in XP, the user is part of the development team (i.e., he or she
is an alpha tester) and provides the system requirements in terms of user stories.
He or she is also responsible for defining the tests, which decide whether or not the
developed software supports the user story. The tests are automated and development
does not proceed until the story acceptance tests have passed. There is, therefore, no
separate acceptance testing activity.
As I have discussed in Chapter 3, one problem with user involvement is ensuring
that the user who is embedded in the development team is a ‘typical’ user with gen-
eral knowledge of how the system will be used. It can be difficult to find such a user,
and so the acceptance tests may actually not be a true reflection of practice.
Furthermore, the requirement for automated testing severely limits the flexibility of
testing interactive systems. For such systems, acceptance testing may require groups
of end-users to use the system as if it was part of their everyday work.
You might think that acceptance testing is a clear-cut contractual issue. If a sys-
tem does not pass its acceptance tests, then it should not be accepted and payment
should not be made. However, the reality is more complex. Customers want to use
the software as soon as they can because of the benefits of its immediate deploy-
ment. They may have bought new hardware, trained staff, and changed their
processes. They may be willing to accept the software, irrespective of problems,
because the costs of not using the software are greater than the costs of working
around the problems. Therefore, the outcome of negotiations may be conditional
acceptance of the system. The customer may accept the system so that deployment
can begin. The system provider agrees to repair urgent problems and deliver a new
version to the customer as quickly as possible.