Effect of a Post-paid Incentive on Response Rates to a Web-based Survey

Up: presentation-increasing-response-rates-incentives

Effect of a Post-Paid Incentive on Response Rates to a Web-Based Survey

Reading: Brown, Julie A., Carl A. Serrato, Mildred Hugh, Michael H. Kanter, Karen L. Spritzer, and Ron D. Hays. 2016. “Effect of a Post-Paid Incentive on Response Rates to a Web-Based Survey.” Survey Practice 9(1): 1–7. doi:10.29115/SP-2016-0001.

KEYWORDS:

The results of this experiment indicate that a post-paid incentive can significantly increase the response rate to a web-based survey.

Overview:

• This study tests whether offering a $5 post-paid incentive (cash or Target e-certificate) increases response rates to a web-based survey about healthcare experiences.
• Conducted with patients from two Southern California Kaiser Permanente medical centers
• Compares an incentive group to a no-incentive group.
• The results show that a small post-paid incentive boosts response rates without affecting answer quality or completion time.

What They Did

• Background: Previous research shows prepaid incentives increase response rates in mailed surveys (e.g., Church 1993: 19% boost), but web survey results are mixed—some find no effect or even lower responses with incentives. This study explores whether a post-paid (after completion) incentive works for web surveys.
• Experiment Setup:
  • Participants: 3,592 adult patients from two Kaiser Permanente medical centers, selected because ~30% used the My Health Manager patient portal (for emailing doctors, viewing records, etc.). All had visited or emailed one of 30 primary care providers in the past year.
  • Groups: Randomly split:
    • Incentive Group (1,795 patients): Offered a $5incentive(choiceofcashorTargete-certificate)aftercompletingthewebsurvey.Mailsurveycompletersgot$5 cash.
    • Control Group (1,797 patients): No incentive.
  • Survey Process:
    • Content: A patient experience survey (likely based on CAHPS, assessing healthcare interactions, e.g., doctor ratings, use of online tools).
    • Delivery: Invitations sent via email from RAND’s system (appearing as kp.org), with a survey URL and unique code. The incentive group’s email added one sentence about the $5 reward.
    • Reminders: Up to three emails (initial, 12 days later, 30 days later). Non-responders got a paper survey by mail after 57 days.
  • What They Measured:
    • Response Rate: Percentage completing the survey (web or mail).
    • Completion Mode: Web vs. mail completions.
    • Time to Respond: Days between invitation and completion.
    • Item Non-Response: Percentage skipping survey questions.
    • Survey Answers: Differences in responses (e.g., use of email for appointments, doctor ratings).
    • Incentive Preference: Cash vs. e-certificate choice among web respondents.
    • Costs: Per-unit cost of cash ($8.31)vs.e-certificate($7.49), including labor, postage, etc.
  • Analysis:
    • Compared group characteristics (age, race, etc.) using chi-square and t-tests.
    • Tested response rate differences with t-tests and completion rates with Fisher’s exact test.
    • Checked survey responses and item non-response for differences.

What They Found

• Group Characteristics:
  • Both groups were similar in age (mostly 55+), education (most had college), plan membership (~65 months), emails to doctors (~4.4/year), doctor visits (~2–3/year), and doctor ratings (8.8/10).
  • Incentive group had more Asians (8% vs. 5%, p = 0.005) and fewer Black/African Americans (2% vs. 4%, p = 0.001).
• Response Rate:
  • Incentive group responded more: 57% (1,023/1,795) vs. 50% (898/1,797) for control, a 7% boost (p < 0.0001).
  • Both medical centers showed similar increases (details not shown).
  • Mode: 72% of responses were web-based, 28% mail, same for both groups (p = 0.879).
• Time to Respond:
  • No difference: 5.76 days (incentive) vs. 5.83 days (control).
• Completion and Item Non-Response:
  • Incentive group had slightly higher completion rates (99% vs. 97%, p = 0.014).
  • Item non-response was similar (no specific rates given).
• Survey Answers:
  • No differences in responses (e.g., 44% vs. 49% used email for appointments, p = 0.115; 95% in both groups said doctors used computers, p = 0.672).
• Incentive Preference:
  • Among 739 web respondents in the incentive group, 56% chose cash, 25% e-certificate, 16% refused, 3% no data (cash preferred, p < 0.0001).
• Costs:
  • Cash: $8.31 per unit (incentive + labor, postage, envelope).
  • E-certificate: $7.49 per unit (incentive + email labor).
  • Cash cost 83 cents more but was more popular.

What It Means

• Main Points:
  • A $5 post-paid incentive increased web survey response rates by 7%, consistent across two medical centers.
  • It didn’t change response time, survey answers, or item non-response, meaning higher response didn’t alter conclusions.
  • Patients preferred cash over e-certificates (56% vs. 25%), likely due to cash’s flexibility, though e-certificates were cheaper ($7.49vs.$8.31).
• Why It Worked:
  • The $5 incentive, though small, was enough to motivate patients, possibly due to trust in Kaiser Permanente or the survey’s relevance (healthcare experiences).
  • Unlike prepaid incentives, post-paid rewards avoid trust issues with online payments (e.g., PayPal doubts).
  • The patient portal’s engaged users (30% active) may be more responsive to email invitations.
• Tips for Web Surveys:
  • Offer small post-paid incentives ($5 cash or gift cards) to boost response rates, especially for engaged populations.
  • Provide cash options, as they’re preferred, but e-certificates save costs.
  • Use multiple email reminders and a mail follow-up for non-responders.
  • Ensure trusted branding (e.g., kp.org) to encourage participation.