Page 123 - Comparing Political Communication Theories, Cases, and Challenge

P1: JZZ/KAA P2: KaF
0521828317c05.xml CY425/Esser 0521828317 May 22, 2004 11:55

Designs and Methods of Research

report. The best way to avoid item bias seems to be pretesting, which can
help to detect bias and control it within the main study (Greenfield 1996;
Niedermayer 1997, 95).

Method Equivalence and Method Bias
When the instruments are ready for application, three other levels of
equivalence must be taken into account: sample equivalence, instrument
equivalence, and administration equivalence. These three levels can be
summed up in the term method equivalence.Van de Vijver and Tanzer
(1997, 264) call a violation of equivalence on this level method bias.
Sample equivalence and sample bias: Sample equivalence refers to an
equivalent selection of subjects, interviewees, or units of analysis (for
content analyses). Identical sampling procedures for every country in
question do not suffice to guarantee equivalence, because different cul-
tures can have different distributions, for example, concerning levels of
education. Thus to avoid sample bias, a culture-specific sampling re-
garding the (main) dependent and independent variables is required
3
(Niedermayer 1997, 93, 96–7). Analogically, when undertaking a press
content analysis, for example, the distribution of different types of news-
papers has to be taken into account. Sample bias can only be detected
and avoided by cultural expertise and the use of external data (van de
Vijver and Tanzer 1997, 264).
Instrument equivalence and instrument bias: Instrument equivalence
can be seen as independent of the specific research project. One has to
examine whether there is equivalence in terms of the people in each cul-
ture who agree to take part in the study, as well as whether participants
are familiar with the instruments (e.g., paper and pencil, telephone, or
online surveys) (van de Vijver and Tanzer 1997, 264; “stimulus equiva-
lence” in Niedermayer 1997). At first sight, content analyses seem to be
rather resistant to instrument bias, however, the risk of bias here lies on
the side of the coders and the codebook. Within an international coding
team, different understanding of the codebook and possibly different
tendencies toward extremes in coding may occur (Lauf and Peter 2001;
more general Wirth 2001). This kind of problem can be found analog-
ically in surveys, where culture-specific attitudes to social desirability,
acquiescence, extremes in answering, and so forth, can cause cultural

3 For different ways of sampling see the overview of Niedermayer (1997, 97–100) or any
basic references for methodology of the social sciences (e.g., Schnell et al. 1999). For
an in-depth presentation of sampling procedures see, for example, Cochran (1972).

103