Publication Type: Paper/Poster Proposal
Abstract: Response rates for random-digit-dial (RDD) telephone surveys continue to decline. Yet several studies suggest that nonresponse bias remains constant for response rates in the range of 40% - 70% (e.g., Curtin et al., 2000; Keeter et al., 2000). The 2002-2003 Joint Canada/United States Survey of Health (JCUSH) provides another example of how nonresponse bias remained constant despite efforts to increase response rates.

JCUSH was conducted jointly by Statistics Canada and by the National Center for Health Statistics of the U.S. Centers for Disease Control and Prevention. This RDD survey was designed to collect data regarding health status and access to health care for both Canadian and U.S. residents, using comparable procedures. Toward this end, interviews for both populations were conducted from Statistics Canada’s regional offices. After 5 months of data collection (November 2002 - March 2003), the Canadian response rate was 66%, relative to a U.S. response rate of 45.3%. To achieve greater response rates for the U.S. sample, additional telephone calls and refusal conversion attempts were conducted in April and June 2003. The final U.S. response rate (AAPOR RR4) was 50.2%.

Sampling weights for U.S. data collected prior to March 31st were produced, and resulting estimates for key health indicators were compared with the final weighted survey estimates and with estimates from the 2002 U.S. National Health Interview Survey. (The NHIS estimates were considered the “gold standard” for analyses of bias.) Similar comparisons were conducted for interviews completed with no more than 10 dials (RR4 = 37.0%), interviews completed with no more than 15 dials (RR4 = 43.3%), and interviews completed without refusal conversion efforts (RR4 = 41.8%). Nonresponse bias was constant for nearly all key health status and health care access measures.

Publication Type: Paper/Poster Proposal
Abstract: The National Survey on Drug Use and Health (NSDUH) collects data on Medicare and Medicaid enrollment status as part of a general interview conducted after the core drug use measures have been administered. While the overall prevalence estimates derived from the NSDUH Medicare and Medicaid enrollment questions have generally appeared credible, it became apparent that among people under 65 years old, Medicare enrollment was overreported and Medicaid enrollment was underreported. Among people over age 65, Medicaid enrollment appeared to be highly overreported. These judgments were based on “eyeball” comparisons with estimates from the SIPP and CPS, both of which administered highly detailed modules on health insurance coverage. Expert review of the NSDUH question wordings suggested that inadequate establishment of context (defining terms after using them in the questions), and other syntactic difficulties created excessive demands on working memory. Correction of these problems in time for the 2003 NSDUH resulted in age-group coverage estimates that more closely matched those obtained in the other surveys, which targeted this topic more specifically and in greater depth. These results are discussed within the context of Tourangeau, et al’s (2000) Response Process Model.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: Arbitron’s syndicated radio listening survey is multi mode (telephone/mail) survey that recruits survey respondents for a one-week radio listening diary. In order to maintain proportionality and representation in the survey, Arbitron uses basic differential survey treatments to recruit household into the survey based on geographies and starting sample characteristics. Despite these differential treatments, significant differences in response rates exist across geographies and among respondents.


In previous work, key socio-economic and demographic differences are cited as significant variables in explaining survey participation. Differences in race, ethnicity and income are shown to have significant impacts on survey response rates. However, these broad socio-economic characteristics are typically used independently of data specific to the data collection efforts such as Interviewer characteristics, timing of day and conversion efforts. This paper will bring together a wide range of characteristics in a multi equation model to explain the interaction of socio-economic variables and specific placement variables. The model will help explain these differences in context of a two stage multi mode survey that depends on a diverse survey population in a complex data collection methodology.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: This paper analyzes the effects of the evaluation by respondents’ of their last survey interview on their attitudes towards surveys in general and whether these attitudes have an effect on their cooperation in future surveys. Data from a locally defined random probability sample is used to answer these questions. Firstly, subjects are found to differentiate between three dimensions when evaluating their survey experience. These dimensions are the burden respondents experienced during their last survey interview, how they judged its ‘entertainment value’ and how irritated they felt themselves to be by confusing questions. According to our second finding, only the burden experienced affects how subjects evaluate surveys in general. This association is found to be substantially stronger when the evaluation of the burden experienced in past interviews is cognitively more accessible as measured by the time needed to generate these judgments. This accessibility is found to be affected by the mode of administration of subjects last survey interview. According to our third result, respondents’ with a critical attitude towards surveys in general are more likely in the complete interview to answer ‘don’t know’, to refuse to answer questions and to be judged by the interviewer to be in general less willing to answer questions. However, the predictive power of attitudes towards surveys for respondents’ cooperation depends on the cognitive accessibility of these evaluations: fast attitude judgments are the better predictor for the subjects’ willingness to answer questions. The cognitive accessibility of attitudes towards surveys is found to increase with the number of subjects’ survey interviews in the past. In summary, the burden experienced by respondents in past survey interviews impairs their evaluation of surveys in general and thereby increases the prevalence of non-response in following surveys. In this way, survey researchers themselves partly determine the conditions for their future work.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: During 2001 the Australian national election study incorporated an online element. This innovation resulted in an identical set of questions being asked using alternate survey methods (mail-in and web-based) and thus, the possibility to examine systematically the claims made for and against online polls. This paper examines the data collected from those online and offline polls according to three main criteria: (1) overall accuracy in terms of predicted outcomes; (2) substantive differences between the internet and mail survey respondents in terms of political and demographic characteristics; and (3) whether survey mode had any significant effect on responses to key variables of interest, independent of any political and demographic differences between the two samples. Conclusions are drawn regarding the overall utility of web-based surveys compared with the more traditional mail-in version.