
A simulation product is only as trustworthy as the evaluation behind it. But how does your simulation product give you the context you need on that evaluation?

[Predictive Answers](https://insightscompany.io/predictive-answers), our simulation product, answers research questions by simulating the segments in your audience. Predictive Answers uses your own research sources – interviews, surveys, support tickets, CRM data – and attaches citations and a **confidence score** to every answer, giving you full transparency that other simulation tools don't offer (more on that later).

Behind the scenes, we pressure-test the simulation from Predictive Answers at increasingly demanding levels. One of the most challenging tests is how well we can predict an **individual person**, idiosyncrasies included. If simulation holds up there, all of the segments above it gets easier. So we ran a study at that individual level to see how we held up across different question types – on questions types that the twins are best suited to answer and extended to use cases beyond those.

On our consumer-signal framework, our framework of the questions that matter most for consumer research, the twins hit **82%**. Pushed beyond the framework into hard-to-predict territory, they still scored as high as **76%**.

The gradient from this study maps directly onto how the confidence scoring in Predictive Answers works. Confidence tracks evidence strength, and this study illustrates why it must.

Here's the study and what it taught us about how to evaluate a simulated answer.

## The Experiment: A Two-Instrument Simulation Protocol

### Instrument 1: The grounding interview

We built digital twins of 10 real people, each grounded in nothing but a 45-minute AI voice interview on Persona. Notably, this is **half the length** of the 2-hour interviews used in [Generative Agent Simulations of 1,000 People](https://arxiv.org/abs/2411.10109), the landmark Stanford study on generative agents. One of our questions was whether interview quality and instrument design could substitute for raw interview length.

Each of our 10 participants completed a 45–60 minute AI voice interview run on Persona, conducted entirely by our AI interviewer. The interview guide is our own adaptation of established life-history methodology, extended with consumer sections academic protocols don't cover. In one conversation, it covers life story and turning points, relationships and daily life, political and religious worldview, health and coping, money and work, future aspirations and values, and everyday consumer habits – from the brands they buy to what drives their purchase decisions.

The AI interviewer asks adaptive follow-up questions based on what participants reveal, which means every transcript captures not just facts but *decision logic* – how each person actually reasons about money, risk, and values.

### Instrument 2: The held-out evaluation battery

Later, the same participants completed a decision survey the twin never saw during construction – spanning identity, personality self-ratings, and behavioral scenarios: surprise windfalls, guaranteed-money-versus-gamble choices, investment allocations, job offers, privacy trade-offs, and civic attitudes.

Every number below is **out-of-sample**: the twins were never tuned against the people or the questions being scored.

## The Consumer-Signal Framework: 82%

At the center of the evaluation is a framework we developed for the question types consumer research asks most – the decisions that drive purchasing, loyalty, and values-based behavior. Across ten questions, it covers how people spend and manage money, what they're willing to pay for, how they handle financial risk, and the values and worldview that shape those choices.

STAT: 82% | Twin accuracy on the 10-question consumer-signal framework, across 10 participants

One twin scored a perfect 10 out of 10 on this set. Three more scored 9 out of 10.

This is the territory that matters if you're using simulation for consumer insight – and it's also, not coincidentally, the territory a life interview is best equipped to reveal. People talk about their money habits and values constantly, in stories, complaints, and plans. The framework works because it targets attitudes people have rehearsed their whole lives.

## Beyond the Framework: How Twins Handle Questions They Weren't Built For

The framework tells you where simulation is strongest. But we also wanted to know how far the twins could stretch – so we deliberately extended the battery into two categories of questions that are harder to predict, or only loosely related to what a life interview reveals.

**Category 1: Forward-looking attitudes.** How participants feel about AI in daily life, whether it will reshape their careers, whether universities should allow it in coursework, their general risk preference, and where they sit on a five-point political scale. These are real attitudes, but they're *constructed* attitudes – people are still forming them, and they shift with framing and mood. Even with these added, the twins held at **76%** across the expanded 15-question set.

**Category 2: Fine gradations.** Questions where the answer is a matter of degree: how much confidence in the press – some, or very little? Do you worry *somewhat* or *strongly*? Stable corporate job, or negotiate first? When the twin misses these, it's almost always by one adjacent notch – predicting "some confidence" when the person said "very little." The direction is right; the calibration is off by one step. With both hard categories included, accuracy across all 21 questions was still **70%**.

STAT: 82% → 76% → 70% | Framework accuracy, then with forward-looking attitudes added, then with fine gradations added

For a twin built from a single 45-minute conversation, holding 70%+ while answering questions it was never designed for is the more surprising result. It suggests the interview is doing real work: capturing not just facts a person states outright, but the reasoning patterns that let a twin extend to territory the conversation never directly covered.

## Why the Gradient Exists

This pattern echoes a much older finding in survey research. Political scientists John Zaller and Stanley Feldman's "A Simple Theory of the Survey Response" argues that people don't walk around with fixed, pre-packaged answers stored in their heads. Instead, they construct an answer on the spot, out of whatever considerations happen to be accessible at that exact moment – a recent experience, a mood, whatever framing the question happened to use.

This is one cause of a phenomenon that researchers are familiar with: even the same participant can give different responses to the exact same question.

If that's true for the real person, it shapes what simulation can do: the more stable and rehearsed the underlying attitude, the more predictable the answer. Money habits and values are rehearsed daily. A stance on self-driving cars might be constructed for the first time when the question is asked – by the participant and the twin alike.

INSIGHT: When asking a question to digital twins, it's important to know which tier your question lives in – or to use a tool like Persona with confidence scores built in. Simulation is already reliable where attitudes are stable and rehearsed – habits, values, spending psychology. It's weakest where real people are themselves inconsistent.

## The Hardest Test: Open-Ended Voice Responses

If structured survey prediction is already difficult, open-ended voice responses are harder by an order of magnitude. There's no constrained answer space, no option list to anchor to, and the evaluation itself is inherently subjective – we used an LLM judge to score each twin response against the real participant answer on stance, content, and style, which captures meaning rather than exact wording, but introduces its own uncertainty. These scores should be read as directional, not definitive.

We believe open-ended prediction is among the hardest challenges in digital twin research with a limited dataset. A twin needs to not just hold the right opinion, but express it in the right voice, at the right level of specificity, touching the right details – all from a single 45-minute interview. That's a high bar.

Which is why the results surprised us. When we asked the same participants free-form questions in a follow-up interview – about the government's role in the environment, how they feel about the media, their career values, and their spending habits – the twins scored **75–85%** on those topics consistently across participants. The same question types that scored highest on the survey scored highest in free conversation too. The gradient isn't an artifact of the multiple-choice format. It reflects something real about how stable certain attitudes are, and how reliably a good interview surfaces them.

## Why Confidence Scores Work in Predictive Answers

Most research questions aren't about one person – they're about audiences. [Predictive Answers](https://insightscompany.io/predictive-answers) simulates those audiences from your own first-party research – interviews, surveys, CRM, support tickets – letting you ask any question and get a cited, confidence-scored answer in seconds.

At the segment level, the idiosyncratic misses we measured on individual twins wash out while shared patterns compound, which is why 82% on individual twins is the floor, not the ceiling, for what Predictive Answers delivers.

The gradient from this study maps directly onto how its confidence scoring works:

- **Confidence tracks evidence strength, and this study shows why it must.** Our results map exactly where simulation is reliable: Consumer-Signal Framework answers predicted at 82%; constructed attitudes at 76%; fine gradations at 70%; one-off specifics lower.
- When Predictive Answers scores an answer, it's measuring the same thing – how rich, direct, and consistent the supporting evidence is. A question about spending psychology backed by twenty interview passages scores high. A question the corpus barely touches scores low, visibly, instead of hiding behind a fluent answer.
- **Contradictions lower confidence – because they should.** Our weakest twins came from interviews where self-description and behavior pointed in different directions. Predictive Answers treats contradictory evidence the same way: it penalizes the score rather than picking a side silently.
- **Grounding beats generic.** In our calibration work, prompts grounded in interview evidence consistently outperformed stripped-down ones. Generic synthetic respondents – built from LLM training data and internet averages instead of your customers – lack that grounding entirely. Predictive Answers grounds every segment model in first-party research: your interviews, surveys, support tickets, and CRM data.
- **Every claim cites its evidence.** In our experiment, each twin prediction cited the specific interview passage behind it. Predictive Answers works the same way: every simulated answer links to the exact interview clips and survey responses supporting it, so you can check the reasoning yourself.

This is a small-scale example – 10 participants, one demographic segment – and the results should be read as directional, not as a definitive or scientifically validated benchmark. Even so, we think they offer a useful signal for any brand or research team weighing synthetic research and digital twins against the alternative of not testing at all.

CTA_BOX: See simulation grounded in your own research | Predictive Answers builds cited, confidence-scored segment models from your interviews, surveys, and CRM data – so you know what your customers would say before you ask them. | Book a demo | https://insightscompany.io/predictive-answers


We pressure-tested digital twins at the hardest level. They hit 82% on the consumer research questions – and what that means for how you should evaluate a simulated answer.

We Tested Digital Twins on Unseen Consumer Questions. They Hit 82%.


## About Moonbow

Moonbow is a skincare brand built on the principles of Traditional Chinese Medicine (TCM). Founded by twin sisters, Heather and Dr. Felice, the brand brings Eastern healing philosophy into modern skincare — a cleanser, serum, and moisturizer that bridge traditional medicine and contemporary beauty science. As the brand was preparing for its first retail launch, they needed packaging that works for consumers who have never heard of the brand, or even of TCM.

## The Challenge

Moonbow had designed packaging that reflected who the brand is: Chinese characters, an Eastern-inspired color palette, messaging rooted in TCM philosophy.

But the team suspected a risk that plagues every close-knit founding team: they had been looking at the same packaging for so long that they could no longer see it the way a new customer would and understand if the messages came through clearly.

Their strong existing community made the problem harder, not easier. Moonbow's Instagram followers and loyal customers already understood TCM. They already knew the product, the founders, and the philosophy. Asking them for feedback would have returned answers that didn't represent the customers the brand needed to win at retail.

> "We'd spent so much time with this packaging that we knew every detail by heart. But that's exactly why we realized we needed to step back and understand how a consumer, someone experiencing Moonbow for the very first time, would actually perceive it."
>
> — Heather, Co-Founder, Moonbow

The questions were specific and high-stakes: Should the cap be brown or red? Should the front panel lead with a Western ingredient like hyaluronic acid, or with the TCM story? Were the Chinese characters and brand language legible and compelling to someone encountering Moonbow for the first time? And critically — would the word "medicine" in "Traditional Chinese Medicine" confuse or concern shoppers picking up the product for the first time?

> "Consumers only see the front panel first. The challenge is creating enough intrigue for them to pick it up, explore the other panels, and feel confident enough to purchase."
>
> — Heather, Co-Founder, Moonbow

## The Solution

Moonbow partnered with Persona to run a qualitative study with consumers intentionally outside their existing orbit — skincare consumers who had no prior exposure to TCM and who had never heard of Dr. Felice or the brand. These consumers would be more like the average consumer who might discover Moonbow on the shelf.

The study was designed to show participants packaging elements one at a time. At each step, participants were asked what they noticed, what they liked or didn't, and whether they'd pick up the product.

> "Our core community already understands Traditional Chinese Medicine. But our goal is to reach the people who don't, those who have never heard of TCM or don't know who Dr. Felice is. Traditional Chinese Medicine is still unfamiliar to many consumers, so our challenge has always been: how do we distill 3,000 years of wisdom into something that's simple, approachable, and easy to understand?"
>
> — Heather, Co-Founder, Moonbow

## The Results

### Eight Packaging Changes. One Clear Direction.

The research delivered concrete answers across every key packaging question — and surfaced insights Moonbow hadn't anticipated.

Moonbow learned which cap design was preferred, and they saw that Western ingredient callouts, like hyaluronic acid, performed better on the front panel than the brand had assumed.

But the most unexpected finding came from the word "medicine" in "Traditional Chinese Medicine": consumers found it confusing to reconcile something described as both "traditional" and "medicine." TCM was newer to this audience than the team had realized.

That single insight prompted a meaningful rethinking of how Moonbow communicates the brand — not by changing what the product is, but by educating more clearly and meeting consumers where they are.

STAT: 8 | specific packaging changes implemented based on Persona research

Moonbow ended up making eight distinct changes based on the research: selecting a brown cap over red, moving Western ingredient callouts like hyaluronic acid to the front panel, adding a "Doctor Founded" marker to establish credibility, adding a QR code linking to the brand origin story, removing cursive text participants couldn't read, adding a pronunciation guide and translation for the Chinese character on the packaging, deepening the color palette for shelf presence, and adding educational soundbites about TCM philosophy.

### The Asian Aesthetic Was an Asset — Not a Risk

One finding validated the cofounders' brand narrative and ran counter to a common assumption consumer brands make when entering mainstream retail: Moonbow saw they should lean into their heritage, not away from it. When participants encountered the Chinese characters and Eastern design language, they responded positively — the aesthetic added legitimacy, not confusion.

> "When people looked at the packaging, they were drawn to its distinctly Chinese design. Rather than feeling unfamiliar, it gave the brand authenticity and reinforced that our roots in Traditional Chinese Medicine are genuine."
>
> — Heather, Co-Founder, Moonbow

### More Detail – and More Speed – Than Anything They'd Done Before

Compared to Moonbow's previous research methods — Instagram polls and internal Google Sheets sent to their existing community — Persona delivered a different order of information entirely. And for a small team moving quickly toward a major retail launch, the turnaround time mattered as much as the depth of insight. Persona was the rare solution that offered both.

> "Persona is very detailed, and the quick turnaround for me was the biggest value, because we're moving so quickly."
>
> — Heather, Co-Founder, Moonbow

After the study concluded, Heather used Persona's built-in AI tool to pull additional takeaways directly from the data.

> "What I found most valuable about the platform was the overall summary and analysis. From there, I was able to use the AI assistant to dig even deeper and surface additional insights and key takeaways. I also think Persona's user experience and the technology behind it are exceptional. I've already recommended it to several people, and the next time I have a research question, Persona will be the first place I go."
>
> — Heather, Co-Founder, Moonbow

INSIGHT: Persona gives teams the outsider perspective — specifically from the consumers you most need to reach — before the investment is made and the decisions are final.

## Why It Matters

For Moonbow, the impact of this research will show up at retail. The entire study was motivated by a single, durable insight the team knew from experience: a consumer's first interaction with a product begins and ends with what they can see on the shelf. Everything else — the brand story, the science, the philosophy — only gets read if the packaging first earns a second look.

For brands preparing for a retail launch or major channel expansion, this case shows that:

- **There's no substitute for external testing** — Your team is closest to the brand, making it difficult to see how a new customer will experience it. External testing with a tool like Persona is the best way to see how a new audience will react.
- **Qualitative research delivers specific, actionable direction** — Moonbow didn't just get impressions and sentiment. They got eight concrete changes they could implement immediately before print
- **Cultural identity is a strength on shelf** — The research validated leaning into TCM heritage rather than diluting it. What the brand needed wasn't less authenticity, but better education.

CTA_BOX: Ready to see your packaging through your customers' eyes? | See how Persona can help your brand test packaging, messaging, and creative before you go to market. | Book a Demo | https://cal.com/insights-company/30min


How a TCM-based skincare brand used Persona to get outside consumer feedback — and made concrete packaging changes before their retail debut.

How Moonbow Used Persona to Redesign Their Packaging Before a Retail Launch


## About Jupiter

Jupiter is a fast-growing consumer personal care brand that started with a hero anti-dandruff shampoo. As Jupiter enters a new market with a new product offering, they needed to understand what messaging and creative direction would resonate most with their target audience — and they needed to move quickly before investing in the full marketing campaign build-out.

## The Challenge

Jupiter was gearing up for a high-stakes campaign launch alongside their entry into a new vertical this summer. The team had an understanding of their target customer, and they came up with three compelling campaign concepts. But they needed to choose between these three directions based on what would actually resonate with their customer — and this decision would have major implications on budget, including a potential five-figure photoshoot for one of the concepts.

The core questions were clear:

*Which campaign concept would resonate most with their target audience? What kind of creative imagery would actually build mindshare and convert?*

Still a fairly lean team despite considerable growth, Jupiter had never had a formal research budget. They have always prided themselves on understanding their customer and had done in-depth research when they first launched — but ongoing consumer insights beyond those moments felt like something reserved for bigger companies with more resources.

> "We needed a way to get real feedback from real people — quickly, affordably, and without the overhead of traditional research."
>
> — Co-CEO, Jupiter

## The Solution

Jupiter partnered with Persona to run a qualitative study with 20 human participants from their target demographic. The AI-moderated interviews explored reactions to multiple campaign concepts, creative directions, and messaging — giving Jupiter's team direct access to how their customers actually talked about the problems the brand solves.

![Example of a report generated by Persona[border]](/blog/tio-magic-report.webp)

## The Results

### A Creative Pivot

The research delivered a clear winner among campaign concepts — one of the three concepts landed incredibly well with their target customer, with participants sharing in-depth feedback on why.

And importantly: it wasn't the direction the team had originally planned to invest in. The findings from Persona led Jupiter to rethink the creative they would launch with. Given how approachable, realistic, and relatable the winning campaign concept was, it led to significant budget savings that could be redirected into marketing activations that engaged their customers and influencers.

> "We decided to change our photoshoot pretty considerably… instead of doing a $50,000 photoshoot of elevated personas, we cut that budget down 60% and said we're gonna take the balance of that money and put it towards UGC type of photography."
>
> — Co-CEO, Jupiter

STAT: 60% | of photoshoot budget restructured based on Persona learnings

### Beyond Answers: Creative Fuel

The interviews didn't just validate a direction — they became a source of creative inspiration. Video clips of participants in their real environments sparked vivid ideas the team never would have arrived at in an internal brainstorm.

> "One of the women doing the survey had a baby in the background crying. It was so real and there were just these glimpses of our customers' reality that we otherwise wouldn't have seen."
>
> — Co-CEO, Jupiter

The interviews also surfaced specific language that consumers used to describe their everyday experience — raw, unfiltered words the brand team hadn't thought to use in their messaging.

> "Hearing firsthand how people describe their hair — how personal it was, the richness of the descriptions — was quite inspiring and instructive for our team."
>
> — Co-CEO, Jupiter

### An Intuitive, Seamless Experience

From onboarding through final delivery, the Jupiter team found the entire experience of using Persona intuitive and low-friction — critical for a lean team without a dedicated research expert in-house.

> "From the onboarding to the results, everything was completely intuitive. We felt like it was a seamless process."
>
> — Co-CEO, Jupiter

After the interviews were complete, Jupiter's CEO used Persona's built-in AI tool to ask follow-up questions of the data to dig in more deeply — and found that it provided more contextually grounded answers than general-purpose LLMs.

> "What I loved most about the platform was Persona's interactive AI tool. You could ask questions of the AI moderator, asking for further insights or seeing if it could extrapolate beyond our results."
>
> — Co-CEO, Jupiter

![Example of "Ask AI anything" by Persona[border]](/blog/jupiter-ask-ai-chat.webp)

### A First-Ever Research Budget

For the first time ever, Jupiter is now able to earmark a dedicated research line item in their budget — something that wouldn't have happened without the accessibility and cost of the Persona platform.

> "We want to use Persona for everything we do that's consumer facing. Truly. Because it's affordable, we are, for the first time now, earmarking an annual research budget."
>
> — Co-CEO, Jupiter

INSIGHT: Consumer brands don't need six-figure research budgets to make informed creative decisions. A single well-designed qualitative study can reshape campaign strategy, unlock authentic consumer language, and give lean teams the confidence to make bold pivots backed by real data.

## Why It Matters

Jupiter's experience illustrates something many consumer brands struggle with: the gap between creative intuition and consumer reality. The team had strong instincts, but the research revealed that their audience wanted something different from what they'd planned. That insight has proven to be incredibly valuable — not just for this launch, but also in their workflows moving forward.

For growing brands preparing for retail launches or major campaigns, this case shows that:

- **Qualitative research drives real business decisions** — Jupiter didn't just learn what people thought; they restructured a $50K budget based on what they heard.
- **Real-world interviews & consumer language are a strategic asset** — Seeing consumers in their real-world environments and hearing their specific language become building blocks of effective decision-making.
- **Accessible research creates a research culture** — When the barrier to entry drops, teams start building research into their ongoing operations rather than treating it as a one-off luxury.

CTA_BOX: Ready to let your customers reshape your next campaign? | See how Persona can help your brand get real consumer insights — fast and affordably. | Book a Demo | https://cal.com/insights-company/30min


How a consumer personal care brand used Persona to reshape a major campaign strategy — and is now able to do ongoing research for the first time.

Jupiter Chose a Major Marketing Campaign Using Persona


## About Tio Magic

[Tio Magic](https://tiomagic.com/) is a VC-backed startup building AI-generated educational content for children. Their platform creates personalized shows that combine entertainment with learning — imagine "a season of Bluey that teaches mathematics." As they prepared for South Park Commons Demo Day, the founding team needed to validate the best way for parents to use their platform.

## The Challenge

Tio Magic faced a common early-stage startup problem: their immediate network wasn't representative of their target market.

Local conversations with Bay Area parents provided valuable early feedback and helped shape the team's initial thinking. But the founders recognized that they had largely exhausted their immediate network, and that this group — while insightful — didn't necessarily reflect the broader American market they were ultimately aiming to serve. With South Park Commons Demo Day approaching, they needed to learn if they were going down the right path.

A critical question emerged: **Were they building a solution that parents were comfortable with? And how could they understand that on a nationwide scale?**

Traditional user research would take weeks and cost thousands of dollars that startups don't have. They needed to recruit parents nationwide, validate their hypothesis, and understand how users actually interacted with their product — something that wouldn't have been possible at their stage until now.

## The Solution

Tio Magic partnered with [Persona's AI-moderated research platform](/) to conduct 10 interviews with parents across the United States. Each interview explored screen time habits, reactions to the product concept, and user experience with the website.

### Real-Time Adaptive Interviewing

The AI interviewer didn't just follow a script — it adapted in real-time based on participant responses, asking contextual follow-ups the team hadn't anticipated.

> "I remember a specific moment where the AI was kind of weaving its way to a particular question in a really good, smart way. It was smarter than I thought it was going to be and that my team probably could have been, since we just aren't experts in research."
>
> — Marci, Founder, Tio Magic

## The Results

The nationwide research delivered exactly what they needed: validation, direction, and actionable insights.

### Validated Market Demand

STAT: 10 interviews in < 24 hours | Nationwide Parent Research

Tio Magic's interviews with 10 parents completed in less than 1 day. The research confirmed their hypothesis: parents saw the value of the product they were building.

> "I was able to interview a random sample of 10 parents, and it definitely gave us way more conviction that what we're doing is correct."
>
> — Marci, Founder, Tio Magic

![Recruit external participants suited to your study with Persona [border]](/blog/tio-magic-external.webp)

### Landing Page Redesign in a Weekend

Beyond validation, the research revealed a critical behavioral insight that led to immediate product changes.

A finding from the interviews was that users were reading about the product on the landing page, but not using it.

- **Before:** Long explanatory content about the product vision
- **After:** Simple call-to-action to create an episode immediately

> "We had a very deep landing page that discussed a lot. We simplified it a lot within a day of viewing this research. Now if you visit our landing page, it's basically just asking you to make an episode."
>
> — Marci, Founder, Tio Magic

### Actionable Insights Without Hours of Video Review

The AI summary report provided everything needed to make decisions without watching hours of recordings.

> "The thing I used the most was definitely the summary output report. This was already generated, and when I saw it, I was like, perfect — this is exactly what I needed."
>
> — Marci, Founder, Tio Magic

![Example of a report generated by Persona[border]](/blog/tio-magic-report.webp)

INSIGHT: Early-stage startups don't need expensive, time-consuming research to validate product direction. With the right tools, 10 well-targeted AI interviews can provide the conviction needed to make major product decisions and get ready for fundraising conversations with confidence.

## Why It Matters

For early-stage founders, this case demonstrates how AI-moderated research can answer critical validation questions without the traditional time and cost barriers.

Tio Magic's experience shows that:

- **Representative sampling is now accessible at any stage** — 10 diverse, strategically recruited interviews can provide conviction for product decisions and break out of biases in your local market
- **Speed enables iteration** — With research kickoff to results in hours, teams are able to see live results in their product in days
- **Behavior matters more than stated preference** — Watching how users actually interacted with the site revealed the real opportunity — the same principle applies to [churn research](/churn), where understanding what users actually do at cancellation matters more than what they say they'll do

> "My overall take: this is really good. The value-add is extremely clear."
>
> — Marci, Founder, Tio Magic

CTA_BOX: Ready to validate your product direction? | See how Persona can help you recruit nationwide and get answers in days, not weeks. | Book a Demo | https://cal.com/insights-company/30min


How Tio Magic broke out of their local network to validate their product hypotheses with a nationwide sample before a fundraising milestone

Tio Magic Validated Product Direction with 10 Diverse, Nationwide Interviews - in Less Than 24 Hours, using Persona


## About Otter.ai

Otter.ai is a leading AI-powered meeting transcription and collaboration platform used by millions of professionals worldwide. As a leading AI-native company with over 200 employees, Otter has been at the forefront of applying artificial intelligence to workplace productivity since before the GPT era. Their user research team, led by a sole senior researcher with experience at other AI-native brands, needed to scale insights gathering across the entire organization while maintaining research quality. They turned to [Persona's AI-moderated research platform](/) to find out whether enterprise users would actually engage.

## The Challenge

Otter's research team faced a common scaling problem: one researcher supporting an entire 200-person organization with diverse research needs across product, sales, and marketing teams. Traditional 1:1 interviews couldn't scale to meet demand, and the team needed to understand whether their B2B professional user base would actually engage with AI-moderated research tools.

A critical open question loomed:

*Would enterprise B2B customers – busy professionals accustomed to white-glove service – be willing to participate in AI-conducted interviews? Or would they view it as impersonal and decline to engage?*

> "I am the only researcher on the team, and part of the reason I was interested in bringing on AI moderated tools was to be able to do more and bridge the gap. AI-moderated interviewing tools make sense, but it's still an open question how receptive our customers will be."
>
> – Misha Cornes, User Researcher, Otter AI

## The Solution

Otter partnered with Persona to run a study with their B2B users that focused on understanding user expectations around one of Otter's key features.

### Persona's Link Customization

Otter utilized Persona's link customization to create a fork-in-the-road experience: participants could either book a traditional calendar slot with the human researcher OR complete an AI-moderated interview on their own time. This A/B design would reveal genuine user preference rather than forcing adoption.

> "If it's framed as an option — either you can talk to me or you can do this on your own time — I think people might actually choose to do the AI interview."
>
> — Misha Cornes, User Researcher

![Otter.ai research study showing fork-in-the-road option [border]](/blog/otter-fork-in-road.webp)

## The Results

The pilot delivered a clear answer to the B2B acceptance question:

STAT: 15 out of 16 | Participants Chose AI Interviews

The results were a landslide. 15 out of 16 participants opted for the AI interview over a human booking. This proved that for B2B stakeholders, the value of their time and the convenience of an immediate session outweighed concerns about missing a traditional "human touch."

> "The findings from the study were pretty definitive. We put that fork in and received all 15 AI interviews. Only two people chose to talk to me (and one of those actually had to cancel)."
>
> – Misha Cornes, User Researcher

This finding challenged the assumption that B2B professionals would resist AI-moderated research. When given a genuine choice – not forced adoption – busy professionals overwhelmingly preferred the flexibility of completing research on their own schedule.

### Quality of the Interaction

Beyond just completion rates, the quality of the interaction was high. The Persona AI's ability to probe deeper than a standard survey was immediately evident:

> "I was impressed by the follow-up questions. It didn't just move to the next script item. It actually probed into why the user felt that way about the AI 2.0 features. The feedback from the AI interviews was also consistent with what I heard in the human conversations."
>
> – Misha Cornes, User Researcher

Otter also observed that AI interviews sometimes encouraged greater bluntness and honesty than human-led conversations – consistent with findings from other Persona studies.

INSIGHT: B2B customers are ready for AI-moderated research. When positioned as a convenient alternative rather than a replacement, professionals prefer the flexibility of asynchronous AI interviews over scheduling constraints.

## Why It Matters

For research teams at scaling companies, this pilot answers a fundamental question about AI-moderated research adoption. Many B2B teams hesitate to adopt AI-moderated research due to concerns around perception and enterprise readiness. Otter's pilot shows that:

- **Choice matters** – something Persona enables with its unique branding capabilities
- **Enterprise participants are open to AI** when expectations are clear
- **AI interviews can meaningfully reduce friction** while preserving insight quality — a dynamic that's especially powerful for [churn and retention research](/churn), where timing and participant honesty are everything

Persona's unique customization features, which allowed for a fork-in-the-road design, proved especially valuable: rather than assuming user preference, it measured it directly. This approach can help other research teams build internal confidence in AI-moderated methods by demonstrating real user acceptance.

![Persona recruitment settings](/blog/otter-recruitment-settings.webp)

> "AI-moderated interviews feel inevitable — especially when you're trying to scale research with one person."
>
> — Misha Cornes, User Researcher

CTA_BOX: Ready to scale your research? | See how Persona can help your team run AI-moderated interviews at scale. | Book a Demo | https://cal.com/insights-company/30min


15 out of 16 Otter.ai B2B users chose an AI interview over a human call. Here's what that means for enterprise research.

The Insights Company Blog

The Class That Graduated Into AI

We Tested Digital Twins on Unseen Consumer Questions. They Hit 82%.

How Moonbow Used Persona to Redesign Their Packaging Before a Retail Launch

Jupiter Chose a Major Marketing Campaign Using Persona

Tio Magic Validated Product Direction with 10 Diverse, Nationwide Interviews - in Less Than 24 Hours, using Persona

94% of Otter’s B2B Users Chose Persona AI Interviews Over Human Calls

Frequently Asked Questions