Page 325 -

314 CHAPTER 11 Analyzing qualitative data

11.4.2.5 Iterating and refining
Qualitative coding leads to the construction of an evolving conceptual framework.
As you examine raw data and assign codes to elements of that data, you are in ef-
fect organizing the components and constructing an understanding that will grow
and change as you continue. For emergent coding efforts, the addition of new codes
is the emergence of your understanding. However, even theoretically informed ef-
forts may find that a deeper appreciation of the data leads to the realization that the
initial framework is not quite adequate or correct. If this happens, you may wish
to add codes to your codebook, and to reconsider previously coded material in the
light of these new codes. This iterative extension of the codebook and rereview
of material can be time consuming, but it does reflect the evolving nature of your
understanding.

11.4.3 ENSURING HIGH-QUALITY ANALYSIS
Qualitative data analysis is not objective. During the data-coding process, a human
researcher makes a series of decisions regarding the interpretation of individual ob-
servations: Which category does this item belong in? Are these items really members
of the same group or should they be separated? No matter how expert the judgment
of the individual making these decisions, the possibility of some conscious or uncon-
scious bias exists. Given the inherent fallibility of human researchers, how can we
increase our confidence in the results of qualitative analysis? More specifically, how
can we make our qualitative analysis valid and reliable?
Before we can answer that question, we must be clear on what we mean by these
terms. In terms of qualitative research, validity means that we use well-established
and well-documented procedures to increase the accuracy of findings (Creswell,
2013). More strictly speaking, validity examines the degree to which an instrument
measures what it is intended to measure (Wrench et al., 2013). Reliability refers
to the consistency of results (Creswell, 2013): if different researchers working on
a common data set come to similar conclusions, those conclusions are said to be
reliable.
Ensuring reliability and validity of qualitative HCI research is a challenge. For ad-
ditional guidance on improving the rigor of your qualitative research—and, indeed,
on all aspects of qualitative HCI—see the monograph Qualitative HCI Research:
Going Behind the Scenes (Blandford et al., 2016).

11.4.3.1 Validity
Validity is a very important concept in qualitative HCI research in that it measures
the accuracy of the findings we derive from a study. There are three primary ap-
proaches to validity: face validity, criterion validity, and construct validity (Cronbach
and Meehl, 1955; Wrench et al., 2013).
Face validity is also called content validity. It is a subjective validity criterion
that usually requires a human researcher to examine the content of the data to as-
sess whether on its “face” it appears to be related to what the researcher intends to