Interviews, focus groups, or surveys: which should you use?

Sarah Durham: Welcome back to the Smart Communications Podcast. I’m Sarah Durham and I’m joined today by Laura Fisher, who’s a Senior Strategist here at Big Duck. Welcome back, Laura.

Laura Fisher: Thanks for having me.

Sarah Durham: For those of you who’ve been listening to this podcast for awhile, Laura has been on the show a few times. Most recently we recorded a podcast about how doing interviews can help you get better insights and we thought today we would kind of expand on that and another article that Laura wrote on our blog, which is called interviews, focus groups and surveys, three research methods to help you understand audiences and we’re going to unpack that in a little bit more detail today so that if you’re doing your own research, you’ve got a few more tips and tricks up your sleeve. So let’s dig in. Before we talk about these methodologies, let’s talk a little bit about context. Why would a nonprofit communicator or other person need to do this kind of research?

Laura Fisher: We typically see research being most useful at two different phases in a project, potentially at the beginning of a project when you’re just starting out and trying to learn a bit about your audiences. So from a communication standpoint, your audiences, their motivations, their perspectives are going to be key to whatever it is you’re trying to do. So if you’re launching a campaign or going through a rebrand before you begin talking to audiences and doing some research among them can be helpful to jumpstart that project and get their perspectives. The second place it might be helpful is midway through a project using research as a form of testing. So we see this with organizations a lot. If they are testing a new brand, maybe a name or a logo or a tagline or connecting a campaign and testing a concept or a theme for that campaign, putting it in front of your target audiences to get their feedback and it can help you make decisions and help you understand if this new branding or new campaign concept is going to resonate with the people that you’re trying to reach.

Sarah Durham: Okay. So let’s talk about three different types of research you map out in this blog. And by the way, we’ll link to this blog and also to the podcast about conducting interviews in the show notes. But walk us through high level what you talk about in this article and the contexts in which each of these methodologies might be most useful.

Laura Fisher: The three methodologies I lay out are interviews, focus groups, and surveys. Interviews and focus groups are both what we call qualitative research, which means you’re really digging into perceptions and motivations and the feelings of an audience member as opposed to something more quantitative, which might be numerical data. So for interviews and for focus groups, you’re more having individual conversations. An interview for example, is a conversation with one person exploratory with the audience group you’re trying to get to know. So these come in handy. If you are trying to understand and experience a motivation or behavior that might have connected someone to your organization. Maybe understanding why they donate or why they volunteer. A focus group is pretty similar. It’s just with a larger group. So typically between five to eight people, all with a common connection to your organization. And you can use focus groups for similar reasons, to really unpack motivations and perspectives and experiences. And in a focus group you can really in real time see themes emerge because multiple people are talking to you about the same topic and you can see trends over time. For surveys, these are typically most helpful when you’re trying to ask a lot of questions of a large number of people. You can ask, you know, up to 20-25 questions in a survey. I have a list of thousands and thousands and cover a lot of topics. So while with interviews and focus groups, you might be digging into a few topics very deeply, a survey. You could talk about your communications, your brand, and more in a number of questions and really dig into various perspectives on your communications. So we tend to think about interviews and focus groups and qualitative research when you’re really trying to dig into a topic and meaty way and surveys as being more of a high level way to get a lot of insights very quickly.

Sarah Durham: So, we’ll talk through a couple of examples and places where these might be more or less useful for you, but before we do that, I want to ask you a question that I imagine some of the people who are listening to this are going to have, which is can I really do my own interviews or my own focus groups? I’m sure a lot of people have seen TV shows where focus groups are done by professional facilitators with two way mirrors and all of that. If somebody is not an expert researcher, can they do this on their own?
Laura Fisher: Definitely. I think there are a ton of resources out there for creating a research process on your own and really is as simple as writing some objective, non-leading questions, getting the right people in a room and asking those questions. You as the researcher, because you work in the organization, will have to play a more objective role than you might typically. So you should be approaching it from a purely research standpoint, not letting, you know, your role at the organization, get in the way of asking clear and objective questions. But I think if, for many of our clients in nonprofits, time and capacity and budget is a constraint and there’s no reason that you, the communicator, cannot conduct some of the research on your own. You could also consider having someone else at your organization that does not work in communications and doesn’t regularly interact with donors or volunteers. Conduct the research on your behalf if that helps you feel like you’re getting more objective opinions.

Sarah Durham: Yeah, I agree. I think DIY research is a little bit like exercising, like it’s better to do some rather than none and if you can’t afford to hire a pro or you don’t have a volunteer who’s really an expert, definitely better to do what you can do on your own then skip it. There is a little bit of jargon that you’ll see come up also when you start to search for interviews, a lot of times professional researchers will call interviews IDIs, in depth interviews, which you know I’ve always thought was a little bit, a little bit of malarkey cause it’s really just an interview.

Laura Fisher: That’s true.

Sarah Durham: A lot of the interviews we do here are just done by phone and they’re you know, half hour long, maybe an hour maximum and developed with a facilitator’s guide. So to your point, questions are developed in advance. They’re written to be non-leading and also the sort of ground rules for the conversation like what is or isn’t going to be confidential, whether or not you’re taking notes, who’s going to see those notes? Those kinds of things are good to think through in advance. Right?

Laura Fisher: Definitely. I think having a script is really key to interviews and focus groups because it, for one, gives you a guide and talking points to start a conversation and make sure everyone feels comfortable having that conversation and to following a script, especially if you’re doing a number of interviews can help to eliminate bias and make sure you’re asking everybody the same questions and giving everyone a chance to weigh in on the same topics.

Sarah Durham: And I think we dug into that a little bit also in the other podcast we’ve got. So if you’re about to embark on a lot of interviews that will be good to listen to. And we’ll link to it again. Before we move past interviews, let’s talk about an example or two of where interviews are handy or where you found that they were the right type of research.

Laura Fisher: As I said before, interviews are especially helpful for unpacking motivations and experiences and really understanding why the connection someone has with your organization is important and powerful. So I have found it particularly useful in messaging work that we do. So for example, we’re working right now with a rare disease organization on crafting new donor messaging. And in that process we’re talking to 25 different donors to hear about their connection to the organization, why they give, what parts of their mission they care most about where else they give, and why they might give to other organizations to really unpack who that person is and what their experience and motivations are for giving to the organization. And then using that and finding themes across all 25 of the interviews to craft messaging that they can then use with a much larger donor audience. So in that case, interviews are really helpful to dig into a specific topic and action, which is giving.

Sarah Durham: And why would you do a focus group instead of interviews?
Laura Fisher: Especially in the context of donating, sometimes a focus group and having a number of people in a room talking about why they give can make people uncomfortable. They like to have one-on-one conversations. So I find focus groups to be more useful when you’re talking about a nonsensitive topic or when you’re testing an identity of some sort. So an example of when we’ve used focus groups is with an education organization doing an awareness campaign and they had a campaign theme, a hook, which was messaging and a visual application and we gave them some options and they really wanted to hear from the people that it was going to be put in front of to see which resonated the most. So we did a focus group where we showed those concepts to a group of about six different people and got their feedback in real time to what they liked, what messaging resonated, what visuals resonated, that sort of thing. And then we could use that to help the organization make a decision about which one to move forward with more broadly. So in that case it was especially helpful because we had work to put in front of a few people. They had the ability to react in real time and then we could build on conversations and perspectives that people were bringing after they saw actual visual and writing work play out. So the focus group has been particularly helpful in those testing contexts.

Sarah Durham: Yes, so that’s a great example of a testing contacts. And I can think of one that we worked on when we were collaborating with the strategic planning firm. We were doing a strategic plan together and we did some focus groups for an arts organization, a Brooklyn based arts organization. And we had focus groups with artists in different communities. Again, not a sensitive topic for people to get together and share their feelings about the organization or about the work. Although there is a dynamic in focus groups that is the sort of either group think or group dynamics that you do have to manage. So what comes up in the group dynamic?

Laura Fisher: It is true that when you have a group of people together talking about anything, people are going to build on each other’s perspectives. Perhaps be biased by the conversation that happened before they speak. It might mean something as simple as the fact that they just, you know, say I feel the same way as the person who spoke before you or that they don’t share an opinion because they’re in a room with a lot of people. So one way we combat that in focus groups, making sure that you give everyone an alternative way to share their feedback. So we often will have a piece of paper, or if it’s a virtual focus group, we share our email addresses, things like that. So if someone feels more comfortable sharing and writing or they want to add to something that they didn’t feel comfortable sharing out loud as an example or something like that, they can do so in writing and share it privately. So that’s just a quick way to make focus groups a little bit more inclusive and kind of combat some of that group think or shyness that can happen in group settings.

Sarah Durham: You can also set some norms at the beginning of the focus groups about hearing all voices or asking perhaps, you know, very loud and dominant personalities in the focus group to take a break and let somebody else talk or something like that. I’ve seen that be a little bit of a wildcard. It kind of just depends I guess on who’s in the room. Okay. So let’s talk a little bit about surveys. What’s an example of a context or two when a survey is particularly helpful?

Laura Fisher: As I said before, surveys are especially helpful when you have a lot to ask of a large number of people. So we were doing a brand and communication study with another health organization and they have a list of, you know, 50 to a hundred thousand people that they haven’t heard from in terms of communications preferences and their views and perceptions of their brand in a long time. So we did a full assessment of their brand and their communications. And a big piece of that was a long survey that included a lot of questions about both brand and communications. Everything from which statement about our organization is most motivating to where do you like to receive your communications? So we got to ask a long list of questions from a lot of people that they hadn’t heard their perspective in quite a long time and then could synthesize at a very high level themes that emerged on their email list. So while an interview might have given us more in depth information about fewer people, we got to see a snapshot of tens of thousands of people that they hadn’t been able to see in a long time. So if you’re feeling like you don’t really know who is on your email list or you have no idea what channels people prefer to hear you on, a survey can be a really good way to get a lot of information about a lot of people very quickly.

Sarah Durham: It’s really helpful also to sort of see that pie chart that quantitative data gives you about percentages of people who’ve responded a certain way. Before we started recording, I was remembering with Laura a project we worked on years ago where we actually used a survey to test two different logos that a client was considering and I was very skeptical of that. I thought that something quantitative about something as kind of emotional as a logo would be problematic, but it really worked well. The organization was kind of undecided, not sure. We sent an email to their alumni, which was a very large list to weigh in on these two visual directions and there was a clear winner. There was an open ended field for comments and people share a lot of comments, which was a lot of work for the people synthesizing those insights to sift through, but really helped make sure that it was something that they got input from, from a very broad range of people in their community. So let’s say you’ve done your interviews or your focus groups or your surveys, maybe you’ve done some combination of those things. We frequently do use, you know, multiple modalities of research depending on the project. How do you synthesize those insights and how do you get over or work with the biases of whoever is actually conducting the research?

Laura Fisher: Synthesis can take time and that time it takes depends a lot on the research methodology you use. Going through surveys, as Sarah said, you get to see pie charts and percentages and they can be a little bit easier to analyze because you get a quicker snapshot of how everyone’s feeling about every question you asked. Interviews and focus groups, we tend to take very diligent notes or even record those and then sift through them. The way that we tend to do it is to have one person sift through the notes and collect themes. Once go back and sift through it and collect themes, add on to themes you’ve already collected again and then have a second person review those notes and identify their own themes, push back on themes that they didn’t see highlighted as much as you might have noted. Just to make sure that whoever is reviewing on the first time is not bringing their own biases to what themes they emerge by having two synthesizers review it. You make sure that you’re getting really objective results about the actual themes that emerged the most times from the script and the recording of the interviews and focus groups. So that’s a great way to eliminate one person’s bias by layering in a second researcher to with you.

Sarah Durham: Yeah, I have seen that be really powerful and transformative in our work in two ways. The first is that two different people do see or hear different themes and ideas, but also I think for the first person, if you, for instance, conducted those focus groups or did those interviews yourself as you’re doing them, you do start to form ideas and insights and it is very easy as you try to synthesize those insights to reveal your own preferences, to start to build the case that you want to build. And what I think is really powerful when somebody else has to review them and synthesize them is that you know that’s going to be checked. So you hold that tendency a little bit more at Bay but also the people that you are presenting those insights to or those findings to tend to believe them a little bit more because they aren’t just, you know, Laura did all this research and therefore she thinks X and it’s all about her. It’s a little bit more objective or it’s been pressure tested a bit and I think that that’s particularly useful if you’re doing research that you’re going to present to your executive director or your board or to a funder where the validity of that research needs to be maybe a bit more rigorous and less personal. There is another step that I see you and your colleagues on our strategy team do to that I think when you’re doing your own research in house at a nonprofit is worth elevating and that is the step of after the research is done before you get into recommendations, sharing back those synthesized insights. So when we do the research, I think you often have a first presentation, which is just a kind of, this is what we heard synthesis, and then a second presentation, which is the recommendations. Is that correct?

Laura Fisher: That’s correct. So we tend to do all of the research for several months, come back with a set of findings and insights, and those are typically just a synthesis of the themes that we saw and a little bit about what we think that might mean. We’re not jumping to this means you need to start, you know, sending 500 emails this year or whatever it may be. It would be something like your list doesn’t feel like they hear from you enough. So it’s kind of that middle step of reporting back on what we heard and adding our own layer of communications insights. And we’d like to do that because it helps to, to Sarah’s point earlier, when the recommendations do come around, it makes a lot more sense. It’s about taking someone on that research journey with you. If we conducted 25 interviews and did a survey to 50,000 people, you’re going to want to hear that and you’re going to want to hear the results of that. And oftentimes with the clients we work with, this is the first time they’ve, you know, heard directly from their audiences in awhile. So we find that that step of checking in and sharing back what we heard can be really powerful for the organization and help them even beyond communications. Sometimes the quotes that they see or the information they get in the survey help them in other places in their organization as well, not just the communications and marketing they’re doing.

Sarah Durham: It’s also a nice way to, I think, highlight the difference between strategies and tactics. Because I think if you jump right from doing research into making recommendations, those recommendations are often tactical, like send more email. But when you stop to say your audiences want to hear from you more, there are many strategies to solve that problem. Email might be one tactic you could use, but there may be multiple ways you could do that. And I think it’s a nice way to kind of Mark the journey of how you arrive at those recommendations with some of the tactics that might emerge. There’s another piece that comes up a lot and we try to layer very proactively into our work that we want you to think about too and that is how to make sure that the research that you conduct is inclusive and equitable and Ally Dommu, Big Duck’s Director of Strategy, wrote a blog about that which we’re going to link to in the show notes. But Laura, this is something that you’ve got a lot of practices around. What tips or tools do you think are useful to bear in mind to make sure the research process is inclusive and equitable?

Laura Fisher: For us I think bringing the inclusivity and equitability into the research process is all about the voices that you seek out and center in the research process. So that might look a couple of different ways. It might be that- do I make it as equitable as possible? You’re not just hearing directly from board members. You’re hearing from volunteers, you’re hearing from program participants, you’re hearing from a number of different people who engage with your organization in different ways and not just those who might have influence to make sure that you’re hearing from all different voices. Similarly, making sure that you have a diversity of voices that you’re listening to as well. So demographically diverse, we try to implement things like screener surveys before we do interviews or focus groups to cast a wide net and try to collect a number of different people that are connected to in different ways and demographic makeups to identify the people that we want to have in a focus group or an interview and make sure that, as much as we can, we are talking to a diverse set of people and a diverse set of connections to your organization. That’s not always possible and sometimes from our clients, you know, we aspirationally want to have a more diverse email list or something like that. So we try to help clients seek out that as well. And it’s helpful I think to think about when you’re embarking on a research process, how you can not only be representative of what your current board or your current email list looks like, but what aspirationally you might be looking for.

Sarah Durham: So there are probably millions of resources that if you Google things like how to do focus groups or how to do interviews or something that come up. One of the resources we like a lot here is a book called Just Enough Research, which you can buy on Amazon. I can’t remember the author’s name. We’ll try to link to it in the show notes, but Just Enough Research is a very handy book. If you’re trying to do your own research and talks about some of the things we’re discussing today. Are there any other parting tips or tricks you want to elevate?

Laura Fisher: I would just add a resource that I use a lot is a Survey Monkey guide to writing good survey questions. For anyone who hasn’t written a survey before or hasn’t in a very long time. It does a great job of sharing how to use different question types, how to make sure you’re writing an objective question. That kind of information might also be useful for writing an interview script or a focus group guide so we can link to that as well. It’s a very helpful sort of starter kit for embarking on a survey.

Sarah Durham: Great. Yeah, and I think one of the insights that’s emerging for me is I listened to you talk is that when you spend a lot of time doing research for a living, as you do, you build a network of tools and resources and skills and those allow you to build your confidence and to feel more certain that the research you’re doing is done well and as valid as it can be, but you don’t have to go that deep. Right? It’s better to do some research and do your best and for an in-house person with limited time to do that, just being thoughtful and methodical is probably the first and most important place to begin.

Laura Fisher: Definitely, and I would just add that sometimes even doing five interviews is enough to hear themes from a certain group. The book just enough research is very true that even a little bit can go a long way, especially if you’re starting from not having many research practices happening at all. So even starting with five interviews can really share some themes that you might not have noticed before.
Sarah Durham: Yeah, and I feel like I’ve said this before on this podcast, but my experience has been that any project benefits from research that sometimes, you know, research is the first thing that gets cut from the budget. There’s just not enough time or there’s no money to do it. But anytime we’ve done research, it has always been helpful. There’s always something that emerges that you say, wow, you know, I didn’t know that, or that’s so valuable. And sometimes what emerges is a little bit unexpected. Like everybody’s really on the same page about this, or nobody’s on the same page about this. Everybody sees it really differently. So we hope that this podcast has inspired you to take a step back before you embark on your next big communications project and ask how much research should I do going into this to understand the context or the landscape, what kind of testing might I do and what’s the most efficient and effective way to get that research done so that it’s done well, done equitably, and your organization can really benefit from it. So Laura Fisher, thank you for joining me.

Laura Fisher: Thank you for having me.