I love polls. Maybe it’s the data geek in me but I just love looking at the results of a well done poll or survey. Polls and surveys can provide a wealth of information that can help guild decision makers on policy, product features and whether or not to improve a roadway. The challenges with surveys lies in their construction and response rate. Once you have figured out those two critical pieces, the rest is pure data bliss (see below for the first Daily MBA survey. Results will be posted once it’s closed).
The Power of Polls
Polls are a powerful tool simply because they can gage the mood, interests or preferences of your target market or group. There is no better way (other than what they buy or sign up for) to figure out what a group of people prefer. Critical to a good poll or survey is how the questions are constructed, it’s length and who responds. A poorly constructed poll will give misleading information. A long survey will have a low response rate (in general) and if you get the wrong people to respond, your data will be skewed. All of these issues can be mitigated if you just follow some simple rules for great surveys.
The Basics of Good Polls and Surveys
A good poll will give you critical information about your target group. This data is invaluable when setting a policy or figuring out what marketing message works. Constructing an accurate and relevant survey requires that the survey or poll author know the objectives, what information is needed to make the decision and understands the target respondents. A survey that asks questions that respondents will struggle to answer will not produce relevant or even consistent data. Good polls encourage respondents to answer and get them to answer by respecting their time. Consider the following tips when building your poll or survey:
Know the objectives: The reason for the poll is critical to understand and agree on. Write down on a piece of paper the poll’s objectives and always refer to the objectives as you construct it.
Survey length: Most people find it hard to fill out a lengthy survey unless they are getting paid. It’s best to keep the time to fill out under 5 minutes.
Write simple, clear questions: The simpler the question, the better. Making a question complex or adding layers to a question will just confuse people. Each question should be focused on a clear subject and response.
Understand the respondent: Getting into the head of your respondents will make it a lot easier for them to complete the survey. It’s best to write down a brief description of your respondents (right below the objectives) and always keep them in mind.
Correlation data: Try and add questions that can be correlated to other polls or generally accepted data (such as Census data or similar survey data). This will give your poll more creditability.
Consistent look and feel: Keep the font, spacing, colors, tone and style the same throughout the survey. Keep the prose simple, clear and absent of jargon.
Question flow: Questions should build upon each other and be self contained if possible (e.g. Not refer to another question on a different page). It’s also helps the flow to introduce the survey and thank people for taking it.
Pick the right question type: Question type (see below) is vital to both data analysis and people’s willingness to respond. Spend time considering the proper type of question for maximum benefit.
If you keep these simple tips in mind, you will be able to create surveys and polls that will give you great data and allow you to make better decisions.
Selecting Question Types:
There are several different types of questions one can ask in a survey or poll. Typically, polls are more of a yes/no or which do you preferrer (like a presidential candidate) than a survey or questionaire were there may be multiple choice or please rank questions. The following list of question types will guild you toward understanding how they might fit into your overall polling or survey strategy:
Yes/No: This is the most basic type of question that generally has the highest response rate. For example, the question will ask if you agree or disagree with a statement like Do you like chocolate?
Open ended: The respondent can fill it whatever they want (like a comment). These are harder to analyze and most respondents don’t answer them since it takes more time to think and compose a sentence.
Multiple choice: Multiple choice is probably the second most common since it gives a list of choices (like food, candy or candidates) and asks respondents to pick one.
Rank choice: Ranking questions take a little more time and effort to answer. They are really good at getting to what preferences respondents really care about.
Matrix & Rating: Matrix questions have several categories or properties of something (like a car) and respondents are asked to rate what they think. Typically, these questions are used to determine what a customer perceives as the most important feature of something.
The above set of questions make up over 95% of all polls and survey’s you will create or encounter. There are other factors that can go into selecting a question like the logical flow of the survey. For example, you may want people to answer a set of questions if they answered yes or no to a previous question. This survey logic can be complex and takes a little more effort and practice to get right. For our discussion, we will simple avoid survey logic and focus on linear surveys.
The Perils of Poorly Done Polls
A survey with poorly constructed questions, a horrible format or one that is too long will produce poor results. It’s important that your survey be well constructed and avoid the following pitfalls:
Avoid loaded questions: Loaded questions imply the preferred answer within them. For example: How horrible is the parking situation? That’s pretty loaded compared to: How do you feel about the parking situation?
One topic, one question: Never mix topics within a question. That will just confuse respondents. For example, How horrible is the parking and what should be done about it? Those should be separate questions.
Keep the question length manageable: Each question should be simple and short. The more you have to explain the question, the lower the response rate. Strive for a simple, one sentence question.
Be consistent with matrix and ranked order: If you have several ranked order questions or matrix questions, keep the same headers or rankings. Not doing so will probably confuse respondents.
Avoid jargon: If you must use jargon, define it. Some abbreviations or slang words may mean different things to different people.
Keep consistent definitions: Avoid changing the definitions of items or switching context of a word. Remember that simple is better. You want your respondents to think about the answer not the question.
Make the survey the right length: Most people are busy and will not spend 20 minutes responding to a survey. So, keep them crisp and short. There are occasions (like formal research) where respondents get paid to fill them out. In that case, strive to make your survey or questionnaire gather as much data as you can while still respecting peoples time.
Avoid the double negative: Double negatives confuse people and are just poor design. Always rewrite a double negative statement to make it crystal clear what is being asked.
Following these simple tips will make your survey or poll much more effective and meaningful. Remember that the whole point of polling is to gather data to make informed decisions. Just remember that the proverbial garbage in, garbage out is especially true for polls and surveys.
Polls are Powerful if Done Right
The main take away from conducting polls and surveys is that the quality of your collected data is directly proportional to the quality of your survey. A poorly constructed survey will naturally give you poor data. Remember that the next time you develop a poll or survey. If you just remember that simple fact then all of your polls and surveys will produce high quality data.
An Example Survey
Now it’s time to take a survey since that is the best way to demonstrate how survey’s work. So, I created the first Daily MBA Reader Survey using Survey Monkey. The survey will take you less than 5 minutes to complete and I would really appreciate you filling it out. Once I get everyone’s response, I will then do the data analysis and share it with you in another post. As you do the survey, keep in mind what we discussed and see how close I got to the perfect survey.
References
Purdue Online Writing Lab: Creating Good Interview and Survey Questions
Survey Systems: Survey Design
Richard A. Krueger “Tips for Designing a survey questionnaire”
Post on Sample Size and Error Calculator from Creative Survey Systems
