Page 327 -

316 CHAPTER 11 Analyzing qualitative data

A database can also provide increased reliability. If you decide to repeat your
experiment, clear documentation of the procedures is crucial and careful repetition
of both the original protocol and the analytic steps can be a convincing approach for
documenting the consistency of the approaches.
Well-documented data and procedures are necessary, but not sufficient for estab-
lishing validity. A very real validity concern involves the question of the confidence
that you might have in any given interpretive result. If you can only find one piece of
evidence for a given conclusion, you might be somewhat wary. However, if you be-
gin to see multiple, independent pieces of data that all point in a common direction,
your confidence in the resulting conclusion might increase. The use of multiple data
sources to support an interpretation is known as data source triangulation (Stake,
1995). The data sources may be different instances of the same type of data (for ex-
ample, multiple participants in interview research) or completely different sources of
data (for example, observation and time diaries).
Interpretations that account for all—or as much as possible—of the observed data
are easier to defend as being valid. It may be very tempting to stress observations
that support your pet theory, while downplaying those that may be more consistent
with alternative explanations. Although some amount of subjectivity in your analysis
is unavoidable, you should try to minimize your bias as much as possible by giving
every data point the attention and scrutiny it deserves, and keeping an open mind for
alternative explanations that may explain your observations as well as (or better than)
your pet theories.
You might even develop some alternative explanations as you go along. These
alternatives provide a useful reality check: if you are constantly re-evaluating both
your theory and some possible alternatives to see which best match the data, you
know when your theory starts to look less compelling (Yin, 2014). This may not be
a bad thing—rival explanations that you might never find if you cherry-picked your
data to fit your theory may actually be more interesting than your original theory.
Whichever explanations best match your data, you can always present them along-
side the less successful alternatives. A discussion that shows not only how a given
model fits the data but how it is a better fit than plausible alternatives can be particu-
larly compelling.
Well-documented analyses, triangulation, and consideration of alternative expla-
nations are recommended practices for increasing analytic validity, but they have
their limits. As qualitative studies are interpretations of complex datasets, they do
not claim to have any single, “right” answer. Different observers (or participants)
may have different interpretations of the same set of raw data, each of which may
be equally valid. Returning to the study of palliative care depicted in Figure 11.2,
we might imagine alternative interpretations of the raw data that might have been
equally valid: comments about temporal onset of pain and events might have been
described by a code “event sequences,” triage and assessment might have been com-
bined into a single code, etc. Researchers working on qualitative data should take
appropriate measures to ensure validity, all the while understanding that their inter-
pretation is not definitive.