Limitations of Questionnaires and Web Experiments

Web-based surveys and questionnaires are a vital epidemiologic technique which provides vital information about the state of public health and illness. They are a popular method of collecting data, which is often less costly and time-consuming than face-to-face interviews, mailed paper questionnaires or automated telephone menu systems. Questionnaires and Web experiments do not come without their limitations, which must be addressed to obtain valid and reliable results.

A questionnaire can be affected by response bias. This is the tendency of respondents to answer questions based on their opinions and not on research goals. Moreover, questionnaire design can influence responses in different ways: for example the language used in the questions can affect whether respondents comprehend and interpret the questions in the same way (reliable) to determine what you’re interested in (valid) or can answer with accuracy (credible).

A lack of enthusiasm or involvement with the questions may also cause respondents to be less inclined to provide honest answers. Additionally, a lack of incentive or compensation may make it difficult for respondents to take the time to fill out an application.

Online questionnaires can be an issue for certain experimental designs, such as studies of reaction time or positioning. It is difficult to control and measure variables across participants because of the different settings for browsers operating systems, browser settings, and the size of screens.

The bottom line is that Web-based surveys can only be accessed by those who have keyboards and Internet literate. This excludes a large part of the population. It is also difficult to Web researchers debriefing participants after the window for their experiment has ended.