Page 190 - Software and Systems Requirements Engineering in Practice

156 S o f t w a r e & S y s t e m s R e q u i r e m e n t s E n g i n e e r i n g : I n P r a c t i c e

clusters of pain points: in one cluster of failed tests, memory
consumption was excessive; in the other, latency and response time
were too slow. At first, the test results pointed to low-hanging fruit:
obvious design flaws that had obvious fixes. Once these were taken
care of, the tests continued to show deficiencies, but the causes were
much less obvious.
Therefore, we conceived and convened Quality Attribute Testing
Workshops where we brought together cross-functional teams of
architects, implementers, and testers, to dig deeper into the
performance problems, to prototype solutions to the problems, and to
specify internal resource measurements that would better quantify
what a successful solution looked like. Armed with the test findings,
diagnoses, and prototyped solutions, we then conducted Quality
Attribute Design Workshops, where we designed the solutions in detail,
including looking for secondary problems exposed by the solutions
to the primary problems.
To summarize, our ASR testing strategy will
• Use quality-in-use attributes to identify corresponding internal
quality attributes.
• Test the internal QAS by measuring simple scenarios that are
built early.
• Keep the number of ASR test scenarios small, but with
multiple combinations of resource parameters (or other
independent variables).
• Automate the testing early, so that it is repeatable and cheap.
• Use the tests to drive QA Testing Workshops and QA Design
Workshops to improve the quality.
For a more sophisticated approach to testing critical system
qualities, see [Cleland-Huang et al. 2008].

5.7 Case Study: Building Automation System

For the purpose of illustration, consider a company that manufactures
devices for the building automation domain and software applications
that manage a network of these devices. With the hardware being
commoditized, its profit margins have been shrinking. The internal
development costs for the software applications that manage different
devices have also been rising.
To sustain their business long term, the company decides to create
a new integrated building automation system. The intended system
would broadly perform the following functions:

• Manage field devices currently used for controlling building
functions.