by Susan Hopkins
Surveys are a classic tool in the planning profession. Every project needs a survey. They’re a cheap and easy way to gather public feedback. Right?
Not exactly. If you’ve created a survey, you know it’s more complicated than it seems. Have you ever distributed a survey and received twelve responses? Have you ever had to analyze thousands of responses to an open-ended survey question? These are just some of the challenges we have seen. At this week’s lunch and learn, the Highland team took a deep dive into surveys—why we do them, how we design them, and how we can make them better. We evaluated some of our past work, revisited methods, and explored ways to innovate. Everyone shared some tips with the group. Here are some highlights:
Andre’s tips: Use a conversational tone (i.e. “Now we want to know about you.”) and make sure your language is not overly complex. Most material for popular consumption, such as newspapers, is written at a 6thgrade level. You can find your “Gunning Fog” index online. The Gunning Fog index measures how many years of education someone needs to understand your text. Hemingwayapp.com also checks the complexity of your writing. To make your survey more conversational and interactive, consider a chatbot survey on Facebook Messenger.
Jen’s tip: Pre-test your survey. Ideally, you want to pre-test your survey with five to 10 people who are representative of your target population. They will identify confusing or offensive questions and additional options that should be included.
Sue’s tip: If you think you’ll get more than 30 responses, don’t use open-ended questions or the “Other, please specify” option. It will be time consuming to analyze and the value of that information is questionable. If you’ve done your homework and pre-tested your survey, you should already know the universe of responses. If you’re looking for more in-depth information than you can get from a survey question, consider a different technique (such as a focus group or an interview).
Speaking of surveys, “Opt-In” is an online panel Sue created to gather feedback and public opinions on a variety of topics impacting the future of communities in Western New York. Topics include transportation, parks, economic development and quality-of-life. We distribute a short survey every few months and post the results on our blog. You can sign-up here.
Christopher’s tip: Consider thepower and pitfalls of offering incentives to take a survey. While monetary and non-monetary incentives can be a great way to boost responses, it’s important to make sure we’re not encouraging biased and/or low-quality results.
Tanya’s tips: Don’t be afraid to think a little differently about survey questions. Be targeted, honest, and transparent. Address questions to a specific audience based on their interests and how they may be impacted by a project. Include a frame of reference so respondents know where you (and your client) are coming from. Make sure you understand ultimate decision for the project and opportunities for public influence before you develop a survey. Also, timing is key. Make sure people have enough time to answer the survey so that their input is actually useful—i.e. before decisions are made. Bang the Table offers more insight on this.
Post-script: the Gunning Fog index for this blog post is 12.55, which means someone would have to be a high school senior/college freshman to understand it during their first reading.
