Publication Type: Paper/Poster Proposal
Abstract: Obtaining high rates of student participation in school-based studies has become increasingly difficult in recent years due to such trends as increasing numbers of schools requiring active consent, student and school overburden, and the rise in number of high-stakes assessments. Additionally, high student participation rates are more difficult to achieve for studies that select student participants at random from a roster as opposed to intact classrooms. The success of these studies typically requires the assistance of school personnel to advertise the study and handle logistical arrangements.

The Education Longitudinal Study of 2002 (ELS:2002) is a school-based research study that uses a random student selection process. ELS:2002 faced challenges in securing student participation in its base year data collection. As part of the design of the field test of the first follow-up study, RTI implemented an incentive experiment to increase student response rates. Offering an incentive was believed to not only make students more likely to participate but also facilitate school cooperation. Student respondents in half of the field test schools were offered a cash incentive and student respondents in the other half of the schools were offered a gift card in the same amount. It was anticipated that cash would result in a higher response rate than gift cards; this was the result in the field test. Based on the results of the field test, cash incentives were offered for the main study. Gift cards were offered only for schools that did not allow cash incentives for students. This presentation will report on the utility of using cash and gift card incentives to increase student response rates and the implications for other school-based studies. Analysis will include measures of data quality (in terms of item nonresponse) as well as quantitative data about response rates.

Publication Type: Paper/Poster Proposal
Abstract: This study investigates the impacts of survey mode and incentives used by institutional researchers in higher education as they conduct research. Findings from parallel paper, phone, and web administrations of a student advisement survey will be presented with a view toward the following research questions: 1) Do the modes yield comparable results? 2) How do the modes compare on logistical matters such as turnaround time, cost, efficiency, and response rates? 3) What impact do incentives have on response rates?

One common issue that nearly all institutional researchers in higher education and that researchers across the country face is determining the most effective means whereby they can acquire needed data. In a higher education setting the portfolio of these means may vary from campus to campus. Numerous ways to gather data exist. Many use traditional paper and pencil surveys, some use phone surveys, and more and more are utilizing web technologies to survey their campus populations.

This study will facilitate a number of purposes. From a content standpoint the data collected will inform the university as to student attitudes and experiences concerning both academic and career advising. Using these data the university advising council will make modifications as needed in order to create a university instrument to be administered annually. Methodologically, this study will help answer a number of critical research questions: 1) Do the modes yield comparable results? 2) How do survey modes compare on logistical matters such as turnaround time, cost, efficiency, and response rates? 3) What impact do incentives have? 4) Do multiple invitations for a web survey help or hinder the process? Essentially the methodological dimensions of this study are intended to provide insight into the tradeoffs between paper, phone, and web surveys as administered in a homogeneous campus environment.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: The National Survey on Drug Use and Health is an ongoing face-to-face household survey of approximately 150,000 households and 67,500 persons each year. The survey, which was called the National Household Survey on Drug Abuse prior to 2002, covers the U.S. civilian noninstitutionalized population aged 12 and older. In 2002 the survey began offering respondents a $30 cash incentive. The offer of an incentive was initiated within the context of other methodological developments in the survey. These other methodological changes included the name change and an increase in interviewers’ adherence to study protocols brought about by enhanced emphasis during initial training and retraining. These changes resulted in a significant increase in response rate. Moreover, the increased response rate was achieved in conjunction with a net decrease in costs incurred per completed interview. This paper presents an analysis of response rate patterns by geographic and demographic characteristics, as well as interviewer characteristics. Changes in the sample composition and possible effects on estimates of drug use prevalence are discussed. Potential implications for other large-scale surveys are also discussed.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: Experiments with respondent incentives often suffer from serious limitations. These can be limitations of design, such as small sample size or deviations from random allocation, but more commonly they are limitations of analysis and interpretation. Studies may: address only effects on response rate rather than on non-response bias; fail to consider effects on weighting and therefore standard errors; fail to make comparisons with alternative strategies to increase the accuracy of survey estimates, such as increasing initial sample size. Additionally, little attention has been paid to the medium and long-term effects of survey incentivisation. These are particularly pertinent to attrition in panel surveys.

Incentivisation can have a number of goals but an important one is reduction in non-response bias. In panel surveys this primarily relates to the later attrition sample. Using data from a large scale, multi-wave, multi-mode (CATI and mail) experiment on a UK government panel survey of young people we test the effectiveness of incentives in bias reduction over more than one wave.

We find persistent and strong effects for response rate, though these vary by mode and wave. The effects on bias reduction are strong for a few items but moderate for most. And they largely disappear after weighting for non-response. This leads us to conclude that, for this survey at least, incentives offer little benefit over weighting and expenditure might be more effectively applied to increase initial set sample size. Additionally, incentives have a negative effect on data quality, in terms of the number of missing items.

We argue that unconditional incentive payments, although the most effective in terms of response rate, can be an inefficient use of resources. Larger conditional incentives, applied later in the survey process, could be more efficient. Such incentives, however, raise practical problems for administration, ethical and public relations problems for survey sponsors (especially government sponsors) and possible expectation effects. We also conclude that uncertainty about the use of incentives will remain until research is better grounded in theory, informed by previous findings.

Publication Type: Conference Paper/Unpublished Manuscript
Review Method: Peer Reviewed
Abstract: In an attempt to increase response rates, many household surveys are turning to mixed-mode designs whereby instruments are designed to be administered via mail and/or telephone. Survey response rates may be further enhanced through the use of incentives. Little is known about how well these efforts work in surveys of low income and minority populations such as those enrolled in Medicaid. The purpose of this paper is to evaluate the effect of pairing a mixed-mode methodology with a prepaid $2 cash incentive in a survey of Medicaid enrollees stratified by race and ethnicity.

A total of 9,350 enrollees were randomly assigned to receive a mail survey with no incentive or a $2.00 bill. The sample was split into two random samples. The first sample was a simple random sample (SRS) of Medicaid enrollees. In the second sample, enrollees in certain strata (American Indian, Black, Hispanic, Hmong, and Somali) were over-sampled. The response rate within the SRS after the mail portion was 39 percent in the non-incentive group and 47 percent in the incentive group. These response rates increased considerably with the use of telephone follow-ups. The non-incentive SRS response rate increased to 59 percent and the incentive response rate increased to 64 percent. Similar increases in the overall response rates were observed in the racial and ethnic strata as well. In the SRS, the response rate for those who received the $2 incentive was 9 percentage points higher after the first mailing, 8 percentage points higher after the second mailing, and 5 percentage points higher at the end of data collection compared to those who did not receive the incentive. While the differences between incentive conditions are more pronounced after the first mailing (with p-values <.01), all differences were remained significant (with p values < .05) after the completion of the mail mode of the survey. The inclusion of the $2 incentive had a similar effect across the different racial and ethnic strata, with the exception of the Hispanic enrollees.