How can interviews help you get better insights?

Sarah Durham: Welcome back. I’m Sarah Durham, and I’m joined today with a member of Big Duck’s strategy team, Laura Fisher. Laura joined us in 2015. Welcome back, Laura.

Laura Fisher: Thanks for having me.

Sarah Durham: Laura is an expert researcher and planner. And she works with a number of organizations conducting research to help craft smart communications strategies and plans for them. So, I asked her to join me today to talk about interviewing and its value overall to the process of doing research, specifically with communications. But I think actually a lot of what she has to say is applicable even beyond that.

Let’s start at the top though, Laura. What’s the value of research in the first place?

Laura Fisher: Sure. So, research is a great way to hear directly from the people that you are, in our case, going to be communicating with. So, sometimes we start a communications project or we are working on some piece of communications and we kind of just go with our gut and go with our assumptions. But research is really a way to hear directly from the people that you’re going to be communicating with on what they care about, what they need, what their behavior is, so that you can form smarter strategy that’s really focused on what your audience wants and what your audience needs. So, you might think we know our audiences really well, and oftentimes we do, but research is a great tool to hear directly from them so that you can form the communications strategy that’s totally tailored to your audiences.

And, as Sarah said, we’re focused on using research to inform communications strategy, but it’s definitely something that could be used across all different types of work in your organization. So, if you are on the program side of things, you could certainly talk to program participants, get feedback on a program, get ideas about a new potential program, and hear from them about ideas that might inform how you design a new program, or how you change how you’re running a particular program that’s already ongoing. So, it’s definitely something that could be used across various departments in your organization.

Sarah Durham: One of my observations over many years of working with nonprofits is that nobody thinks research is a bad idea.

Laura Fisher: Right.

Sarah Durham: But very few organizations budget for it or allow time for it, particularly in the US. It seems, actually, to be more common to use research in the EU, at least from some of the work we’ve done there. But I’ve also seen, time and time again, that the organizations that do actually take the time to do some research, even if they do it themselves and do it in a kind of down and dirty way, the insights that they gather from that research, I think without exception, are game changing.

Laura Fisher: Oh, yes.

Sarah Durham: I’ve never worked on a project where, at the end of doing some research, the organization said, gee, that was really a waste of time. All it did was confirm what we already know. Always along the way there seems to be some sort of insight, or aha moment, or something.

Laura Fisher: I would definitely agree. And I think research, it’s not something that’s totally evergreen. You should be doing it more in an ongoing basis. But often when we do research with organizations, like let’s say we’re doing a survey, and we send a survey to your email list about communications preferences, that information is one, often the first time that the organization is getting information about how their email list likes to be communicated with generally. And they use that and the insights from that for years. I’ve had organizations that we work with remember a data point from a survey and use it to inform something they do a year later. So, it really is useful and lasting insights that you can use for years to come, even after the initial research is conducted.

Sarah Durham: And before we dig into talking specifically about interviews, you just said something that sparked, I think, an important seed of an idea about how frequently to do research. I have seen that, in some of the largest organizations, these are huge organizations with large comms teams and large budgets. Typically, what they will do is their communications teams will allocate a certain amount of money every year for research and there might be certain polls or processes that they use every year. But then, periodically, they’ll invest in maybe a deeper dive.

Other organizations, some of which are very large and some of which are small, tend to only do research episodically. They do it maybe as part of strategic planning, maybe as part of a rebrand, maybe as part of an analysis of the potential of their audience or their membership. And, in those cases, in my experience, they budget a lot of money, maybe 50 thousand dollars, 75 thousand dollars. We saw one organization, large organization, budget several hundreds of thousands of dollars to do a really deep dive into research.

The challenge with that is that it goes stale. That if there’s a big election, a big game changer in your space, a new technology or tool that’s a game changer, some of that research is going to get thrown out. So, I like your point about making sure to try to do it consistently and try to budget for it.

Laura Fisher: Yes, definitely. And I think, for those organizations that are more time strapped or budget strapped, it might be more strategic for you to do research in the more episodic way on a project to project basis. So, before you start a big project, doing some research, doing a little bit of evaluation at the end, that can still be really useful and really insightful for how you set strategy for that particular project. Even if you can’t invest money in research year over year, thinking about how you can do research in small ways to inform a big project that you’re about to work on.

Sarah Durham: So, let’s get into interviews finally.

Laura Fisher: Sure.

Sarah Durham: Why use interviews? What’s the value of interviewing?

Laura Fisher: Yes. So, interviews are helpful for a couple of reasons. For one, they’re a qualitative research strategy, which means that they give you some of that softer data. They give you quotes, they give you storytelling directly from the people that you’re speaking with, as opposed to just the numeric data or quantitative data. So, qualitative data can be useful because you can hear stories, and input, and motivation, and behavior directly from the people that you’re talking to. So, interviews are a great way to just have a candid and open conversation with someone.

They also are helpful because they can be done relatively informally. So, that’s something that I think is particularly useful within the nonprofit space, because you could conduct a pretty informal interview with a couple of board members at a board meeting. You could, at an event, have a quick conversation with a donor that’s structured and a little bit more of a strategic way and use those insights, as opposed to something like running a survey or conducting a focus group.

There are some research strategies out there that are harder to pull together. When done well, interviews can be used in a lot of different contexts and in a lot of more on-the-spot contexts where you can really take time to gather some data on the spot from someone that you might just have in the room, as opposed to going through what might feel like a really formal research process.

Sarah Durham: So, how do I know who to interview, and how do I know how many interviews I should do?

Laura Fisher: Sure. The best practice in the interview world is to try to do five interviews with a similar type of person. So, let’s say you really want to interview participants at a recent event you had. You had a 5K and you want to know how their experience was and what you might do differently next year. I would say interview, ask five people who participated in that event. We suggest five because, after five interviews, you’ll really start to hear trends in what people are saying. You’ll start to see, oh everyone said they really loved getting the emails in advance, but they thought that it would have been nice if they got a follow up email that said how much money the event raised, or something like that.

You’ll start to hear trends in the things that people say, if you talk to enough people. That said, we don’t have all the time in the world. And that, if you will do up to ten or 15 interviews, you’ll start to hear repetition.

Sarah Durham: I want to jump in and say that, in my experience when I’m doing interviews for a particular slice of an audience, five is a really helpful number in that if the organization is aligned or if there’s something that is kind of a universal truth, you do hear it. You see the pattern in those five people. But I’ve also had experiences where all five people are all over the place. And I’m curious also, then what do you do? Do you keep going?

Laura Fisher: In that case, I would probably suggest one, taking a deeper look at who you are talking to, because it might be that there’s more variation in the audience group than you might think. And two, probably conducting a few more. Conduct five more and see if you can start to see some patterns emerge in what folks are saying.

Sarah Durham: Okay. So, you were talking about the different types of people to interview.

Laura Fisher: Yes. So, there are two different types of interviews. The first is what we call a stakeholder interview, which is interviewing somebody who’s really close to the organization, who knows the organization really well. That might be the CEO, that might be a staff person, that might be a board member. Those interviews are really in service of understanding the organization and understanding organizational direction, ideas from people close to the organization. Those are really useful to use in contexts like if you’re about to start a branding process and you want to get information from folks who are really close to the organization about who they think you are that might inform some of the branding work that you do. So, they are really great for hearing from those closest to you and using that to inform some of your direction.

A user interview is really more focused on the target audience. So, what I mean by that is the people that are actually getting your content, coming to events, the people that you’re directly communicating with. So, talking to users is going to be a great way to understand who your audience is, what they like about how you communicate with them, what motivates them, who they are, why they take action, why they engage. That’s going to give you the really good content about who your audience is and how you can form strategy around reaching them.

So, in my experience, those user interviews are going to be really helpful, especially if you’re thinking about running an event, or running a campaign, or something where the users and the audience are directly involved and you want to get their feedback. You want to hear what could make the experience with your organization even better for them.

Sarah Durham: I have an example of a user interview aha moment from a number of years ago, probably about a year after Facebook was no longer just being used on college campuses and were starting to be used by anybody. We were working with an organization that was developing programs for rabbis and cantors for religious leaders in the Jewish world. And they were debating should they be on Facebook? What was the best way to communicate with these rabbis and cantors? And so, we conducted a series of user interviews with people who they were hoping to reach.

We asked them a number of questions, but among the questions we asked them was about how they liked to be communicated with by this organization. Did they want email, etcetera? And then, we asked questions like, do you use Facebook? Do you use this? Do you use that? And the organization went into this process fully expecting that the people they were trying to communicate with were going to prefer more traditional, print, and phone calls, and things like that. And it turned out what the user interviews surfaced was that rabbis and cantors at that time were actually early adopters of social media. And it was because the people that they work with, the people that they serve were using Facebook. So, because their communities were on Facebook, they had very quickly found that Facebook was a great way to engage with their community. And that was the kind of insight that I think the organization just wouldn’t have been focused on had we not done it.

Do you have an example of a stakeholder situation where sometimes the stakeholder interview surfaces something that was unexpected?

Laura Fisher: The best examples of stakeholder interviews are those that we conduct before we start a branding process for the organization. So, oftentimes we will talk to folks who are close to the organization before we start working on a branding process with a client. And that is for us to really get to know that organization and to hear from some people who are really close to the work about their goals, their hopes and dreams for that brand, for who the organization is at its core, so that we can do our best to represent the core of the organization through its brand.

So, a good aha moment or a moment that really makes us pause a lot of times in research in doing those stakeholder interviews is, especially when there is a disconnect, to your earlier point, among who the target audience is. So, I will do a couple of stakeholder interviews and find that some people think that the brand is really in service of fundraising, and that donors are your key audience, and that’s who the brand really needs. That’s its primary audience. But others really think that they need to engage staff and board in the brand and they’re a primary audience.

So, that’s a disconnect between folks who are closest to your organization, who know you well about who the brand is ultimately in service for, who your primary audience is. And that’s something that then we’ll unpack together with the organization and sort of hash that out and figure out priorities. But it’s very interesting to hear from those directly in the organization to see where those disconnects are, to see where that alignment is, and then figure out how you build off of that.

Sarah Durham: Yeah. And an insight like that, I think, could have profound implications also for what you do next.

Laura Fisher: Mm-hmm (affirmative).

Sarah Durham: If people are really, really aligned, then that’s great. That means you don’t have any big obstacles and the path forward is pretty clear. But, when people are very misaligned about what the work is in service of, in your example, you might have some work to do just to get everybody on the same page before you can actually go and do the rebrand or go and do the other work.

Laura Fisher: Right, exactly.

Sarah Durham: When you are deciding who to interview, how can you bring a diversity lens to the process? How do you make sure that you’re representing people with varied backgrounds, and perspectives, and stakeholders, from an equity point of view?

Laura Fisher: Sure. So, it depends on who your audience is. So, sometimes we will do what we call a screener survey before we conduct interviews, which basically asks a wide pool of people if they would be willing to be interviewed. And that asks them questions about things like demographics and engagement with the organization so that you yourself have the ability to choose who you want to interview and get a more diverse pool.

That’s not always a possibility with some organizations. Some organizations have less diverse bases to begin with. And, in that case, I think the best option to try to hear a wider perspective is thinking about broader market research that you can do. Sometimes if we don’t have the diverse pool we want to hear from on our own email list, it’s required to sort of go broader and do bigger research with more of a nationally representative sample or something like that.

So, we don’t always have access to that within our own database. And if that’s a really important part of who the organization is and the direction that you’re going, thinking about how you can hear from those voices, it does require a bit more work to find those voices when you don’t have them immediately accessible to you on your email list. And that’s where something like bigger market research can be a helpful tool.

Sarah Durham: And I would just encourage you, as you design your process, to think about that, to think about are you hearing all the voices of the communities that you serve? Are you hearing all the voices of the stakeholders that represent your organization, so that you’re confident that you’re gathering the best insights and the most diverse insights?

Laura Fisher: Definitely.

Sarah Durham: Okay. So, somebody who wants to embark on conducting interviews is going to, hopefully, begin by identifying who are their stakeholders and who are their users, and then determining how many in sort of the sub categories of that. So if, for instance, you’re taking the stakeholder audience, would we say that the board is a category of stakeholders and we should interview five of them, plus five staff people, plus five volunteers, plus five donors? Or would we say that all those groups collectively are stakeholders and we should just interview five of them? How do you decide?

Laura Fisher: I think it depends on the project that you’re working on and the goal of that work. So, because time is hard to find working in a nonprofit, I think to combine some of those groups is okay. It’s important to get a diversity of perspectives. And, if conducting up to 20 interviews before you’re starting off on a project is just too big of a task for you, doing five interviews with a mix of program staff, board members, the executive director, that’s okay. You want to hear a variety of perspectives, and they should all be connected to the organization in a similar way, which is that they’re close to your work, they know your work really well. If the way that they are engaged with you and the relationship they have with you is slightly different, that’s all right.

I think in the world of user interviews and when you’re really talking to that target audience, being a little bit more aware of the type of group you’re talking to is important. So, if you were to do interviews with donors versus interviews with volunteers, those are two very separate audiences, and you probably want to talk to five or so of each of those groups because the way they engage with you is so different. So, I think putting a little more pressure on yourself to think about the relationship to the organization when you’re doing user interviews is important. When it comes to those stakeholder interviews and folks who know you so well, it’s okay to have some different types of relationships in that group of five.

Sarah Durham: And Laura wrote a really helpful blog that we’ll link to in the show notes that provides a number of tips for conducting interviews and tackles some important topics like how not to ask a leading question, or things like that, that I think are really central to the practical application of this. The other piece that we haven’t really dug into here, and maybe this is a good topic for a separate podcast, but I just want to touch on it, is sort of where is an interview or conducting research with the user appropriate, versus when would testing perhaps be a better way to do something?

And what I’m thinking about, for instance, is that, if you ask your users, let’s say your clients, should our organization change something, should we change a program, or should we change our name, or something like that. For an outsider, change can be perceived as a bad thing, and sometimes it’s hard for people to imagine it. But, if you put in front of that person an alternative, so instead of asking the question, should we make a change, you said, should we stay the way we are or should we do this? And you point to something really tangible that is the expression of that change. I’ve seen a number of examples where that kind of approach, which is more of a testing approach, gives much more tangible and useful results, because then you’re not just sort of struggling with the fact that people find change difficult or they can’t imagine what you’re talking about. How do you see that?

Laura Fisher: I think what you’re getting at is just two different points in the process. So, interviews and conducting research is helpful to do, as I said before, at the start of a process. So, if you’re thinking about changing your name, I would probably avoid asking, should we change our name, to the people that you’re talking to. You should ask their opinions on the current name, what’s working about it? Thinking about who the target audience is, is it resonating? What do they like about it? What’s working? What’s not? That’s going to give you some insights and some information about how people are viewing the name now, without asking them directly if you think it should change. They might bring that up themselves. They might not. But they’re going to give you some objective responses to the current status of the name that you can use to inform your own decision.

Testing might come further down the line, and it’s still a good practice, a good research practice to keep in mind. That’s something where, when you do have options to present to folks, you can put them in front of them and give them the time to respond and let them say, I think this one is more effectively communicating your X, Y, and Z. I think this one is not doing as good a job. So, I think those are both important research practices, but they come at different points in the process.

Sarah Durham: Great. So, in addition to what’s in your article, are there any parting suggestions or tips you have for people who maybe have never done their own interviews and they’re beginning to design a process that would have interviews as a key research moment?

Laura Fisher: Sure. I think the best advice is just to try a couple of them because interviewing people for the first time and conducting any type of in-person research for the first time can be a little bit intimidating or a little bit odd to have that kind of candid conversation with someone. But going in there remembering that you are meant to be objective and just really a data collector, I think, is really important. Because that’s going to help you ask your objective questions that are going to get the best answers from people.

Going in there and being totally open to the type of feedback that you can get from people is going to help them feel like they can trust you and say anything that they want to. So, just trying a few out with the researcher hat on and just trying your best to get the most objective responses from people that you can.

Sarah Durham: And I think what you will find in Laura’s article is that, if you take the time before you do that first interview to really craft a thoughtful facilitator’s guide for yourself, and design the questions really specifically, and make sure you’ve kind of done your prep, those first interviews are going to be a lot easier, because you’re going to have a script to follow. You will have done some heavy lifting in preparation, which I think makes the process a lot easier too, than just sort of popping on the phone and winging it.

Laura Fisher: Yep. Definitely.

Sarah Durham: Laura Fisher, thank you for joining me.

Laura Fisher: Thanks, Sarah.