Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: A frequently used question to assess health status is: “In general how would you rate your health?” This paper will focus two mode of administration issues around the use of this question: First, are basic mode effects between telephone and mail administration. Historically, social desirability has been used as the primary argument for why telephone mode respondent’s are healthier than mail, this argument will be considered relative to recent advances in understanding regarding conversational norms associated with telephone interviewing. Two data sources will be used for this analysis, in each study respondents were randomly assigned to either the telephone or mail mode of administration. One comes from a study in a population with feacal incontinence (FI, n ~ 150/mode, RR1 82 percentmail, 86 percentphone), which demonstrates significant mode differences. The other will come from a general population survey of the state of Minnesota (list assisted RDD sampling frame) that is currently in the field (data collection ends in February). The second issue that the paper will look at is based in the use of different modes of administration for different data collection points in a study. Often in clinical outcomes, medical or public health research in which there is an intervention baseline data will be collected using one mode (usually telephone or in-person) with follow-up data collection using another mode (usually mail). This is done due to costs and enrollment conditions placed on the research by IRBs. Using the data from the FI study respondents initially interviewed by telephone were surveyed by mail after a treatment intervention. On average respondents reported a decrease in health status between the two measurement points, while two other measures, that have been shown to be related to self-reported health status: self-reported severity ratings improved, clinical measures of severity demonstrated improvement.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: The Gallup Organization Europe (Gallup) and the European Social Survey (ESS) jointly conducted a mixed mode experiment in Hungary during 2003. This experiment employed a crossover design with two phases of data collections. The survey modes tested were face-to-face, telephone, self-administered paper and web-based self-administered interviews. A ‘quota’ sample of 1987 respondents were randomly assigned to one mode in the first phase and were re-interviewed using the same questions with another mode in the second phase, the web mode was not included in the crossover design. The questions were selected from the ESS and Eurobarometer questionnaire. Two forms of questionnaire with alternative wording and format were fully crossed with survey mode during the second phase of data collection.
Traditionally, the main mode of survey data collection in Hungary has been the face-to-face interview method. The current research examines how other survey modes, telephone interview, self-administrated paper and web-based self-administrated, affect response shifts and item nonresponse rates. The effects of mode and socio-demographic characteristics on people’s changes in responses are tested using multinomial logistic regression. The influences of question sensitivity, alternative response scales, question wording and format on item nonresponse vary by survey modes. Generally, the response shifts between face-to-face and telephone mode does not significant differ from the response shifts between telephone interview and self-administrated paper questionnaires. The overall item nonresponse levels are lower for interviewer-mediated questionnaires, e.g. face-to-face and telephone interviews had higher item nonresponse than self-administrated paper surveys. Alternative formatting in questions improve response rate significantly. Employment status is found to affect people’s likelihood to provide answers to work related questions. In addition, substantial variations exist among respondents with regard to their propensity to response to a particular survey mode.

Publication Type: Paper/Poster Proposal
Abstract: Words convey meaning in interview and self-administered surveys, but respondents to paper and web surveys infer additional meaning from the symbols and numbers used in, and the graphical design of questionnaires. We report the results of several experiments to compare different uses of words, symbols and graphics designed to instruct respondents to report date responses in a desired format. These experiments were embedded in a series of surveys, three web and one telephone, of randomly sampled Washington State University undergraduate students. We find that the manner in which respondents were verbally instructed to report the month and year in the query had a powerful effect on the phone survey; however, on the web survey, the stem change had less effect since respondents were provided additional visual instructions located with the response space, making them more accessible at the time of response. The most dramatic effect from the web surveys is that using symbols (e.g. MM\YYYY) to convey the number of digits respondents should use to report the month and year strongly increased the use of the desired format. In addition, placing the symbols in close proximity to each answer space increased their visibility at the time of response and thereby resulted in more respondents using a four-digit year response format. Our research suggests that verbal language changes have powerful effects in phone surveys, where aural communication is the primary mode of communication. In web surveys where other visual languages can be used to support verbal language, symbols and graphical location can effectively be used to instruct respondents to report answers in a particular format.

Publication Type: Conference Paper/Unpublished Manuscript
Abstract: Exit pollsters are challenged to collect a representative sample of voters by the recent rise of early voting. Beginning in 2000 the media exit poll consortium has conducted phone interviews of early voters to supplement their in-person election day exit polling in states with a sizable share of early voters. In Oregon and Colorado, the phone survey is the exit poll. Other surveys, such as the Current Population Survey, document that early voters tend to be wealthier, more educated and presumably more partisan than their election day counterparts. Yet, exit pollsters find phone surveyed early voters are significantly less partisan than election day respondents. We investigate possible methodological reasons for these mode effects, such as social desirability bias and failure of in-person respondents to complete the self-administered questionnaire. Our preliminary findings show evidence of mode effects in the differential rates of item completion between phone and in-person respondents. Given the increasing popularity of early voting, understanding and resolving these confounds is imperative to ensure exit polls are representative of voters.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: Mixed-mode survey data collection plays an increasing role in modern survey research. Telephone surveys and self-completion--both traditional paper and pencil, as well as the more recent internet-based--questionnaires often complement face-to-face surveys, in an effort to contain costs (e.g., Biemer and Lyberg, 2002). Also, mixed-mode data collection strategies have been used to improve response rates (e.g., Shettle and Mooney, 1999). For example, those who fail to respond to mail-out or internet questionnaires, may be called by telephone to improve completion rates.
The mixing of survey data collection modes, however, includes a number of problematic issues. Recent research evidence clearly indicates that there exist substantial differences in response rates between modes of data collection (e.g., Dillman et al., 2001; Dillman 2000). Thus, the differential response rates across modes of data collection may give rise to differential response distributions in the measures of interest. Yet, even controlling for these differential response rates, among respondents who do participate in surveys, it is clear that differential modes of data collection lead to differential response patterns. De Leeuw and van der Zouwen (1988), for example, report that telephone surveys are less subject to social desirability bias than are face-to-face surveys. Similarly, Tourangeau and Smith (1998) report a substantial degree of mode-induced measurement bias in responses to sensitive questions.
Saris and Hagenaars (1997) propose using the latent class model (LCM) to examine mode-induced measurement error. This paper extends their use of LCMs to examine a number of plausible models for testing specific hypotheses regarding the nature of mode effects. The paper demonstrates the use of LCMs for examining mode effects for nominal and ordinal level measures. Data from a variety of public-release data sources, as well as recently collected mode effect experimental data, are used to illustrate the utility of this approach.