Asking Questions So Respondents Can Answer

Throughout the many workshops on measurement, evaluation, and ROI I have conducted a common question emerges when we discuss data collection -- how do you get people to respond? This question relates particularly to self-administered surveys and questionnaires. Answering this question is sometimes challenging as 'getting people to respond' is quite a broad theme. But to put it into context of the many discussions that have ensued during these workshops, the question relates to:

• Receiving an acceptable response rate (non-response error)

• Receiving reliable responses (measurement error)

In an previous post I briefly mentioned non-response error along with another type of error, coverage error. So in today's post, we'll address measurement error.

What is measurement error?

Simply put, measurement error reflects the subjectivity or bias inherent in responses to questions. Error always exists in question responses, unless you are answering a math question where 1+1=2. If answered incorrectly, it is not as much error as it is just plain incorrect. To put it in equation form:

Observed Response = True Value + Error

This means that the response to a survey question to a question (observed response) is the sum of the true answer plus the amount of error due to either chance (e.g. the respondent is in a good or bad mood) or the way in which the measurement is taken. Measurement error has a direct relationship with the reliability of the responses received from a survey instrument. While we cannot remove all random error, there are steps we can take to mitigate that error, improving the probability that what people say on a survey is what they mean.

How do you mitigate measurement error?

One simple solution to the measurement error issue is to ask people questions that they can actually answer.