A Randomised Controlled Trial of Email Versus Mailed Invitation Letter In A National Longitudinal Survey of Physicians.
Abstract Background: Despite the low cost of using email to distribute surveys to medical practitioners, email invitations have been associated with lower response rates, potentially increasing response bias and reducing external validity. We examine if there is a difference in response rates from using email rather than a mailed invitation letter in a nationally representative longitudinal survey of qualified physicians. Methods: We use a parallel randomised controlled trial during the 11th annual wave of the Medicine in Australia: Balancing Employment and Life (MABEL) longitudinal survey of doctors. Participants were from previous waves of MABEL and newly invited in Wave 11. The control group was invited using a mailed paper letter (including a paper survey plus instructions to complete online) and three mailed paper reminders. The intervention group was approached in the same way apart from the second reminder when they were approached by email only. The primary outcome is the response rate and the statistical analysis was blinded.Results: 18,247 doctors were randomly allocated to the control (9,125) or intervention group (9,127), with 9,108 and 9,107 included in the final analysis. Using intention to treat analysis, the response rate in the intervention group was 35.92% compared to 37.59% in the control group, a difference of -1.66 percentage points (95% CI: -3.06 to -0.26). The difference was larger for General Practitioners (-2.76 percentage points, 95% CI: -4.65 to -0.87) compared to other specialists (-0.47 percentage points, 95% CI: -2.53 to 1.60). For those who supplied an email address, the average treatment effect on the treated was higher at -2.63 percentage points (95% CI: -4.50 to -0.75) for all physicians, -3.17 percentage points (95% CI: -5.83 to -0.53) for General Practitioners, and -2.1 percentage points (95% CI: -4.75 to 0.56) for other specialists. Conclusions: For qualified physicians, using email to invite participants to complete a survey leads to lower response rates compared to a mailed letter. Lower response rates need to be traded off with the lower costs of using email rather than mailed letters.