Solved – How to avoid and handle survey non-response

Question

One objective of a survey can be to understand the proportion male vs. female users. To the extent a specific gender correlates to particular use cases of a technology, product design, product/feature prioritization and marketing can be differently managed.

A survey of existing users will inherently have error bars based on sample size, frequency of sampling, etc. As I understand it, when asking the question 'are you male or female?' as part of a demographic survey where all fields are required, an additional selection bias is introduced into the survey.

Specifically, if a survey is sent to 100 users, of which 50 respond, the proportion of responders that are male may not represent the proportion of 100 customers that were initially surveyed.

Assuming there is not a existing sample set of known males and known females whom to survey in order to estimate the responder bias, what are some approaches to prevent or to correct for this error?

Answer 1

In some cases it is possible to make adjustments and "correct" survey non-response. The topic is i.a. discussed in this post. However, your situation seems to be a bit special. You have a list of users, but no information about their profiles.

The best thing to do about non-response, be it in your particular case or in a more general case, is to avoid it. When designing a survey a couple of measures can be taken. What follows is just some general advice. You have to see what can be applied in your particular situation.

• When designing the questionnaire you have to stick firmly to the "KISS" principle: keep it short and simple! Only ask for the items an characteristics you really need. Only ask for the detail you really need. For instance, Rather than asking an exact birthday, ask for the age in years, rather than asking for a city, ask for a broader region, ...
• Make sure that your questions are intelligible and rather straightforward to answer
• The collection mode can also have an impact on the response rate. Some people might claim that face-to-face interviews yield the best rates. However, depending on the topic, respondents might feel more at ease with more anonymous online or telephone surveys.
• Incentives may increase the response rate. The respondents can either be paid for taking part or they can, or a prize draw can be organized among the respondents.
• If interviewers are used, they have to convince the persons to respond.
• Similarly, follow-up and reminders are very important. If after a first contact, a person did not categorically refuse to take part in the survey, one or more further attempts should be done.
• Confidence is crucial. If you want to obtain some more confidential information, or information that is perceived as such by the respondents, you have to be trustworthy. You need to guarantee that confidential data will be treated as such.