What makes a good survey?
The best surveys start with a clearly defined purpose. Begin your survey project with a clear vision for how you plan to use the survey data - what questions are you trying to answer? What decisions will you hopefully be able to make? What decisions can others make? And, is perception data the right kind of information to help make those decisions?
Once your purpose is clearly defined, the next challenge is to develop the best instrument or set of questions for your survey. If you’re developing a new survey, you’ll want to work with a survey expert who knows how to develop high quality survey questions that are free from bias and can reliably get you the information you need to serve your purpose. Survey writing can seem easy; it’s tempting to write questions on your own without the guidance of an expert. However, most individuals find out too late (i.e. when they have already administered their survey and are beginning to look at the data) that they’ve overlooked mistakes in their survey writing that impact the quality of their data. Simply put - if your survey questions are bad, your data and analysis will be poor.
If possible, rather than developing a new survey, choose an existing survey instrument with established validity. Validity means that the survey instrument measures what it is supposed to measure. You can rely on the fact that the data will be accurate. Validity is established by using the instrument over and over and studying the results. You can only determine the extent to which a survey is accurate by using it, and using it a lot.
Education Elements’ “Tripod’s 7Cs” survey — named for the 7 indicators of exceptional teaching practice – has, for example, been administered since 2001 to millions of students, tens of thousands of teachers, and thousands of schools across the country. Every time the surveys are used, we learn more about how to improve the accuracy of the data they produce, and now the instruments are in their 18th generation. If you can find a survey instrument with established validity that aligns with the purpose of your project, choosing this survey over developing your own will improve the likelihood that you get valuable and accurate data.
Once you’ve selected or developed the best survey instrument for your purpose, you’ll want to make sure you can analyze the data in ways that answer your research questions. If you’re trying to understand how student perceptions differ by grade bands, you will need to know if the students responding to your questions are in elementary school, middle school, or high school. (Not to mention the fact that you’ll want surveys that ask questions in an age-appropriate way for each grade band.) Some survey platforms provide powerful ways to prepopulate this information for each survey participant, which cuts down on the length of the survey and typically leads to more accurate information. And, if this isn’t an option, you’ll want to include demographic questions in your survey so that participants can self report this information.
Finally, you need a thoughtful survey administration and promotion plan. Even the best written surveys will fail if no one takes them. To make sure you get the highest response rates possible, consider the following:
- How can I let my stakeholders know about this survey? Will I reach the most people through social media? Word of mouth? Email or newsletters?
- What do I need to do to motivate them to take the survey? Typically, you’ll want to share a compelling reason to take the survey that matters to each specific stakeholder group, and an honest statement of the amount of time it will take to complete.
- What survey administration will make the survey the most accessible to the people I hope to hear from? Will a completely online survey work or do you need to use paper as well? Can your survey be taken on a cell phone? Does the survey need to be translated? Is the survey accessible for younger students?