Have you ever started a survey project with one idea in mind but realized you have something different once you get the results?
It’s easy for this to happen. After rounds of feedback and the (seemingly endless) iteration, where you initially plan to head may not be where you end up.
If this is something you struggle with — or simply want to avoid — I suggest incorporating “big picture questions” (BPQs) into your next survey.
Quite simply, BPQs are a list of 5 – 7 high-level questions you want your survey to answer. Once you document these, you use them as a guidepost throughout the entire process to help you stay on track.
Here’s how they work.
Project kickoff
Think about your BPQs when during the planning phase.
After you have identified your research niche, draft your BPQs to give your topic more substance and direction.
For instance, let’s say you’re creating a survey to study how marketers are using survey-based research. (Meta, I know.)
Your next step would be to draft your BPQs, which identify the specific things you want to learn.
In our example, the BPQs could be questions such as:
- How are marketers building in stories/insights when writing survey questions?
- How are marketers sourcing respondents for their surveys?
- What types of quality controls are marketers using to improve the credibility of their data?
- Are marketers repurposing their research? If so: how? What’s working? What could be improved?
- Where do marketers struggle with survey-based research?
- Do marketers consider their research to be effective? What are those who are have success doing differently?
Your job is to share these questions and get consensus from your team. I often ask: “If our research answers these questions, would you consider the project a success? If not, what else do you want to learn?”
Once everyone agrees to BPQs, they will serve as the guideposts throughout the rest of the project.
Question design
You’ll next use your BPQs when drafting your survey questions. Take each BPQ and create survey questions that will help you uncover the insights you’re looking for.
Continuing with our example, let’s look at this BPQ: What types of quality controls are marketers using to improve the credibility of their data?
You can turn this into a several survey questions, such as:
- Does your survey include screening questions in which you disqualify people who aren’t the right fit for your survey?
- How many write-in questions does your survey typically include?
- How many attention-check questions does your survey typically include?
- Are you cleaning your data?
- If yes, what process do you use?
- If no, why not?
Survey review
You may be tempted to put you BPQs aside after drafting the survey, but it’s important to revisit them when the survey questions are almost final.
Here’s why: Your survey will likely evolve as you get feedback. Questions will be added, and others will be cut . . . and where you intended to go may not be where you ended up.
My suggestion is to map your final questions back to your BPQs. Simply list your BPQs in a doc and then list relevant survey questions underneath. You’ll then reference this “BPQ map” during the final step.
Findings
Lastly, revisit your BPQs when you are presenting your findings. Go back to the BPQ map you created in the previous steps and dig into the data to see what you have discovered. I then revisit and answer each BPQ when sharing the findings.
From my experience, BPQs work really well if you use them throughout the process because you:
- Identify which questions are most important to answer
- Continually verify your survey will answer these questions
- Answer these need-to-know questions when presenting your findings