Mixed methods are further optimized when guided by the dual process theory of thinking, which describes how two psychological systems operate in decision-making. System 1 is automatic and instantaneous, while System 2 is conscious and effortful. Researchers often target System 1 in survey research to uncover people’s instinctive preferences and beliefs.

In a recent 538 article about political issues that Americans care about, quantitative survey questions were used to arrive at the finding that the economy and inflation far outweigh other issues as a top concern. TL;DR conducted a survey of U.S. Consumers around the same time and asked respondents the same question, but in an open-ended qualitative format. We believe that drawing on the dual process theory helped us arrive at more nuanced and richer insights through this mixed methods approach that we will explore in this blog post.

In May of 2022, TL;DR surveyed 983 U.S. consumers between the ages of 18 and 65 and asked them to weigh in on multiple issues, including what they thought were the “biggest issues that America needs to solve in the near future”. We coded their open ends based on themes that emerged from the data rather than using a set of predetermined categories. We then created a separate coding scheme to align with the categories reported in the 538 article.

Our open-ended format leveraged System 1 differently than a guided question would, since our respondents didn’t have to search a category list, consider each option, and rule out alternatives, all the while abandoning that which they already had as top of mind concerns. Additionally, the format answered the “how” behind the “what” people thought. And while we arrived at a similar top takeaways as the 538 article, our open ends allowed us to discover the variability and nuance within economic concerns, such as housing, wages and debt.

TL;DR’s approach was able to account for opposing views within a single theme using coded subcategories. Unsurprisingly, political partisanship evoked the most divisive responses. One on side of the matter, people mentioned concerns such as “the president we have right now”, “we need a leader that everyone trusts” and simply “Biden”. On the other hand, people named “Donald Trump”, “Manchin. What’s up with him?” and “Republicans”. This contrast would not have been possible to detect if we just looked at frequency counts of the general category of “political partisanship” alone.

Another benefit to TL;DR’s approach was that fact that our study yielded a much larger share of “Other” responses (27%) compared to 538’s 2%. Issues such as hunger, inequality and infrastructure comprised half of this robust category. This finding suggests that participants are more likely to choose a named category over an “Other” in a guided format, which prevents a deeper dive into other specific and important topics. The corollary here is, when open ends (or preceding Qualitative research) are used in combination with closed-ended questions for things like pain points and preferences, we arrive at a more accurate picture of those consumer insights and thus better inform future actions we might take to solve for them.

All in all, coding a large set of open ends allows for more natural and nuanced data that better reveals people’s top-of-mind thoughts on a particular issue or topic. It also triggers System 1 to name what is truly important and unveils a more robust story than constraining participants’ responses within a provided list. By drawing on principles from behavioral science in our question design, we can avoid inadvertently priming respondents to answer a certain way via the options we give them, and thus avoid bias in our data. Open ends analyzed using mixed methods addresses this problem and provides additional layers to navigating the consumer’s cognitive landscape. The implications for B2B and B2C are endless, and the right research provider will get you those exciting insights regardless of the topic you care about.