Page 64 -

ITERATION IN SYSTEMS ANALYSIS AND DESIGN 49
intended to correct (Boehm, 1988). Many problems associated with evolutionary development
include: “ad-hoc requirements management; ambiguous and imprecise communication; brittle
architectures; overwhelming complexity; undetected inconsistencies in requirements, designs, and
implementation; insufficient testing; subjective assessment of project status; failure to attack risk;
uncontrolled change propagation; insufficient automation” (Kruchten, 2000, ch. 1).
Not surprisingly, many caution that evolutionary development is not suited to every situation,
as the idea of continuous iteration makes unrealistic assumptions. Evolutionary methods assume
that projects can be structured according to short-term iterations, face-to-face interaction is always
tenable and superior to formal upstream documentation, and the cost of change remains constant
over the project (Turk, France, and Rumpe, 2005). Issues such as scaling, criticality, and devel-
oper talent will often require hybrid methodologies—or a combination of evolutionary prototypes
with formal and control-oriented methods (Boehm, 2002; Cockburn, 2002; Lindvall et al., 2002).
Also, evolutionary development demands other complementary assets like smart designers or the
availability of enlightened users in order to succeed (Beck, 2002; Boehm, 1981).

Empirical Impacts of “Iterative Development”

Empirical research on “iterative development” is as scarce as the prescriptive research is plentiful
(Gordon and Biemen, 1995; Lindvall et al., 2002; Wynekoop and Russo, 1997). The empirical
research that does exist focuses on the effects of prototyping on project success (e.g., Alavi, 1984;
Boehm, Gray, and Seewaldt, 1984; and others), while neglecting the impact and role of other
iterating representations. Nevertheless, in what follows, we assess the state of empirical research
on iterations over representational artifacts.
Representational artifacts include the documents, data models, and other representations of the
software, including artifacts such as user-interface mock-ups and “throw-away” prototypes. These
representations are addressed quite extensively in the prescriptive literature, but the iteration of
these representations and the effects of those iterations on design outcomes are notably absent.
The primary exception is the research on “throw-away” prototypes. Although many researchers
distinguish between prototypes that occur at different stages and are used for different purposes
(Beynon-Davies et al., 1999; Floyd, 1984; Janson and Smith, 1985), the empirical literature does
not underscore distinctions between these types of prototypes and their outcomes, and when they
see a distinction, there is no significant difference in the outcomes (Gordon and Biemen, 1993,
1995).
As indicated earlier, the notion most commonly associated with “iterative development” is
evolutionary prototyping. Table 4.4 summarizes the expected impacts of evolutionary prototyp-
ing and compares them with empirical findings. It is important to note that a good number of
researchers have found empirical evidence to be inconclusive, and these data are not reported in
our review. Furthermore, many expectations highlight the drawbacks of the evolutionary method,
but these criticisms focus on design outcomes, which are addressed below.
The fundamental reason Basili and Turner advocated iterative enhancement is that problems and
solutions are not well understood, and even if they were, “it is difficult to achieve a good design
for a new system on a first try” (1975, p. 390). Subsequent empirical research found prototyping
to be an excellent method for users and developers to learn about the requirements together (Alavi,
1984; Boehm, Gray, and Seewaldt, 1984; Naumann and Jenkins, 1982; Necco, Gordon, and Tsai,
1987). Prototyping has been found to support communication and problem solving between users
and developers (Deephouse et al., 1995; Mahmood, 1987), and has led to greater user involvement
(Alavi, 1984; Gordon and Bieman, 1995; Naumann and Jenkins, 1982). Improved user participa-