- The Interviewist
- Posts
- How to Validate, Support, or Elucidate Survey Data with Interviews
How to Validate, Support, or Elucidate Survey Data with Interviews
Becky Lawlor shares best practices on conducting interviews on survey data to inform reports and thought leadership.
The Interviewist
A note: The Interviewist is shifting to a monthly cadence. You can now expect a new issue/episode on the last Tuesday of each month. Thanks for reading!
![]() | Becky is the Founder and Chief Research & Content Officer at Redpoint, an agency that partners with B2B tech firms to elevate their market presence and brand recognition through original research and authoritative insights. Prior to Redpoint, Becky spent about 17 years as a freelance B2B tech content marketer, copywriter, and consultant. She has also worked as a public information officer for the town of Avon, CO and in communications for nonprofits. |
🌎️ Salt Lake City | 🎤 25 years interviewing |
The following has been edited for clarity and length. Listen to/watch the full interview on the Interviewist YouTube channel.
Sarah: How are you involved with interviewing these days?
Becky: My content marketing agency creates thought leadership content. A big part of any thought leadership is getting insights from the thought leaders within an organization, as well as industry thought leaders. So even when we're not doing survey-based or quantitative data-type reports, the content we're doing often lends itself to needing a lot of interviews with both internal or external subject matter experts.
I’ve interviewed hundreds of folks myself at this point, but I also have a team of writers that I oversee, and sometimes they're the ones doing the interviewing.
Redpoint produces a lot of thought leadership content based on surveys and quantitative data. How do you use interviews to understand or supplement survey data?
One way you could use interviews is to pre-inform your survey. If you talk to a couple of folks who are in the industry or who match the target audience you're going to survey, you can test out your survey or your hypothesis to make sure that you're going to get what you think you're going to get.
Once we have survey data, you can use interviews to validate that data. For example, I had one survey for a legal tech company, and we asked about a compliance issue. One thing we asked was: What are the consequences? Are people getting jail time or fines? The data said yes—but that's really shocking! You don't hear about jail time being an issue in corporate for not being compliant, right?
So in that case, it was really nice to then interview some industry experts and consultants who work with these companies and compliance. They could say, yeah actually, I have personal, individual experience where I've seen what the data is saying—I've actually seen it play out.
So, especially in a case where the data is really surprising, you can use an interview to validate that a person's real-world experience matches that data. Or maybe you just don't even understand why the data is the way it is. And again, you can go to industry experts and have them weigh in as to, “Why do you think we saw this result out of the data?”
Do you prioritize talking to external experts versus client SMEs?
Getting industry experts and outside experts is its own challenge. So some of it just depends on the reality of whether or not we can get those. That depends on my clients’ relationships and connections in the industry.
A lot of these original research reports can really elevate internal thought leaders. A lot of my clients will use the research when they go to trade shows and events for their internal SME to speak using the data. So if that's the real goal—to elevate our internal thought leaders—then we're probably going to be interviewing internal thought leaders.
If it's more of an industry authority play, then we will probably try to get more outside, external industry experts.
Do you ever offer incentives for external sources?
I do if I have to go out and find them on LinkedIn. And you’ve got to offer enough—you have to look at the level you're trying to source from. So if you're trying to source CMOs, you've got to offer $100 to $200 Amazon gift cards for their time.
If you have any kind of relationship or connection, you can do it without incentives. But if you're out there just cold outreaching, you’re probably going to need to incentivize.
The $10 Starbucks gift card probably won’t cut it.
The other way to really incentivize people is to let them know how you're going to use the content and how it's going to build their thought leadership and reputation as well.
So even if I'm offering a paid incentive, I like to let the source know: This is going to go out to my client’s email list of X people, they're going to publish it here, they're going to try to get PR in these publications. I want them to know how they're going to benefit and how it's going to elevate them.
How do you prepare experts for these interviews without overwhelming them with data? How do you avoid asking leading questions?
I have to be careful about how many people we interview, because there's only so much space in a report—they're already dense with all the data. You're trying to make them digestible for somebody who has a short attention span. So I try to stick to three to five interviews total.
Hopefully, you have different perspectives, different backgrounds, different expertise. So you look at their expertise and background, and find the areas of your data that would make the most sense for them to weigh in on.
I usually only give them about five to seven questions max. I'm just giving them five to seven data points and presenting, here's what this data found around this topic. But I don't ask, “Do you agree with this?” I'll just say, “Here's the data that we have. What are your thoughts on this? Why do you think this data is the way it is?”
I’m careful not to use leading terms like, Do you agree? Do you disagree? We just want their input, feedback, and industry expertise on why the data might have come out the way it came out.
If it's an internal SME and they're going to use the report more in-depth anyway, I might just share the whole slide deck with them. I will still have some questions, but in those cases, I might be more open to something like, “You looked at all of the data, is there anything in there that I didn't ask you?” It's a question I like to ask in general.
Do you have any other advice for conducting interviews around quantitative data?
It starts with looking at the data itself. You may ask 30 questions in your survey, and you’ll have your hypothesis. But often, the best stories and the hidden nuggets of insight are in segmenting or cross-tabbing your data. Looking at things like, if we had somebody in a C-suite role, and somebody as an individual contributor, maybe the C-suite thinks everything is grand, and the individual contributors think life sucks down here. We're overwhelmed.
There are a lot of different ways to look at the data. This doesn't happen all the time, but I think it’s important to do that deeper level data analysis. It's hard for a lot of folks if you don't have a data background, or you're just trying to run your own survey, and lot of vendors don’t do it.
But you've invested so much in the data, and those hidden stories are where I also want to do the interviews, because that's where the interesting stories come out.
Do you recommend that writers take data analysis courses? Or is there a way to beef up those skills to better identify insights to ask experts about?
I don't know. I have a Master's in creative writing and a Bachelor’s in English, and I am not a mathematician by any sense. When I was learning the whole end-to-end research process, I was trained in basic data analysis. But I really quickly was like, I shouldn't do something that's not my zone of genius. I should bring in somebody where that's their zone of genius.
Writers come into these projects at different stages. They’re handed the data, and hopefully, all of that analysis has been done for them, and they're getting a nice deck.
But if they're not, if they're just getting Excel spreadsheets—it's time to advocate. Either the client needs to do that work first to equip the writer to do their best work, or the writer can work with a data analyst—but they should put that work in their budget.
If you're a writer who loves data and loves nerding out on that stuff, then maybe you want to do those data courses. But I think that's not the reality for every writer. And I don't think you have to be a data analyst to write a research report, but what you do need is to have the data served to you in a way that you can understand it.
Do you have any interviewing inspirations, muses, or role models?
I love Terry Gross. I think she's an amazing interviewist, she always has thoughtful questions and great ways to draw insights out of folks.
🎤
Want more? Check out our full, uncut interview below.