Get the Best Out of AI—Without Bias

In the first, second, and third parts of copywriter Chris Silvestri’s series on empathy engineering—his framework for understanding customers using AI personas—he wrote about the process of gathering and analyzing data, making AI customer personas, and testing his marketing messages against his new AI-formulated customers. In the fourth and final part of the series, Chris outlines the different cognitive traps that marketers can fall into when communicating with AI and trying to understand their customers.—Kate Lee

Was this newsletter forwarded to you? Sign up to get it in your inbox.

---

Right now, your potential customers are making important decisions about whether to buy your product—or not. They're scrolling through your and your competitors’ websites, comparing features, and feeling frustrated or confused. And you? You're trying to read their minds through spreadsheets and surveys, tailoring your messaging to what you think will resonate with them.

So far in this series exploring the empathy engineering framework, we’ve gathered foundational real-world data, built our AI personas, and started testing our messaging. In step four, the final part of this series, we’ll refine our AI personas and prompts to ensure our marketing strategy is aligned with the constantly changing realities of our target audience.

Traditional data sources give you part of the picture, but they’re best at understanding the individual, not the whole. So, what if you could actually understand all of your customers—not just one, but thousands of them?

That's what makes artificial intelligence so powerful. It helps us see these human patterns at scale. But here's the catch: AI learns from us, and we humans are full of biases and mental shortcuts that can mess with our judgment. How do we get the best out of AI without letting our human quirks get in the way?

Four cognitive traps to avoid

When we start working with AI, cognitive biases can creep in. These biases—mental patterns that influence how we perceive and interpret information, often leading us to make irrational or flawed decisions—may cause us to misinterpret its insights, overlook important data, or become overly reliant on the machine's “wisdom.”

Let’s consider four cognitive biases that have the biggest impact on how we understand our customers through AI. By understanding overreliance bias, anchoring bias, confirmation bias, and availability bias, we can avoid simple mistakes and better tailor our messaging to our customers’ needs.

1. Overreliance bias: AI knows best—or does it?

We’ve all been there: Confronted with a technical problem, we Google the solution, blindly follow the first set of instructions we find, and then wonder why our computer is making strange noises. It's easy to fall into this trap with empathy engineering, too. Overreliance bias is our tendency to trust AI's recommendations too much, especially when the task is complex or the AI’s reasoning is unclear.

Imagine that I want to test a new landing page headline for our fictional software company TeamFlow. So I prompt our AI persona, “Jane, the efficiency-driven project manager”:

"Hey Jane, take a look at this headline for our landing page:

“TeamFlow: Project management zen. Get your inner peace back (and hit every deadline).” How effective do you think it is at grabbing your attention and making you want to learn more?”

Jane gets back to me with her feedback, and her inner monologue, which you can read between the “<thoughts>” tags:

AI (Jane): <thoughts> "’Project management zen’ and ‘inner peace’—those are definitely attention-grabbing, especially in my world of looming deadlines and constant juggling. </thoughts> I actually really like it! It's different from the typical headlines I see for project management tools, and it definitely piqued my interest.”

I’m thrilled! Jane's response is exactly what I was hoping for. But what if:

• The AI misunderstood the tone? Perhaps the AI missed the slightly ironic tone of the headline and interpreted “project management zen” literally, leading to an overly positive evaluation.
• The AI's data is outdated? Maybe the AI's training data doesn't reflect the latest trends in project management, and the concept of “zen” feels outdated or clichéd to real project managers.
• The headline is actually confusing? “Inner peace” might not be a tangible benefit that resonates with project managers who, lately, are more focused on measurable results.

It’s here that I should question both my own assumptions and those of the AI. I return to my TeamFlow chat, using Google AI Studio with the Gemini 1.5 model, and ask questions to validate or disprove my assumptions, specifically about the word “zen” and on whether or not we need to focus more on tangible benefits. This screenshot shows where I share Jane’s feedback with my TeamFlow AI chat and ask probing questions:

The AI replied with potential issues with the headline based on its ideal customer persona (ICP) data and offered a few suggestions with different angles to test.

Source: Screenshots from Google Gemini 1.5 Pro.

The alternative headlines use different lenses—like control and automation over project management chaos—to speak to customers.

Next, I wanted to see what Jane thought about these alternatives. Do they resonate more with her? Are they more effective than the original one? I fed these options into the ICP chat, and let Jane walk me through her “thoughts”:

Out of all the options we shared with Jane, a clear winner emerged, with the headline that focused more on showing ROI for project managers, like hitting deadlines, delighting clients, and getting time back:

We could take this refined headline option and further iterate on it, and test it with real customers on a landing page.

2. Anchoring bias: Don't settle for the first output

Anchoring bias is our tendency to fixate on the first piece of information we encounter—an initial estimate, a starting price, or, in our case, the AI’s first output. This ”anchor” can then disproportionately influence our subsequent thinking and decisions, even if it's wrong or incomplete.

Let’s say you’re negotiating a new software contract, and the vendor asks for $10,000 per month. Even if you negotiate them down to $5,000, you might feel like you've scored a great deal—even if a fair market price for a similar product might be closer to $2,000. $10,000 was your anchor.

Imagine our AI, after analyzing all the customer data we gathered in part 1, generates a description for Jane that includes this personality trait: “Jane is easygoing, flexible, and doesn't get stressed easily.”

Our research suggests that project managers are often highly organized, deadline-driven, and prone to stress, so this personality seems a bit off. To avoid getting stuck on the AI’s initial suggestion, I generate a few alternative descriptions of Jane's personality that align more with what we know about project managers. In the response below, my TeamFlow chat gives me feedback on the new personality description as well as a few variants:

This variant focuses on organization and attention to detail:

By comparing the three new options side-by-side, we can get a clearer sense of which one best reflects the research data and our understanding of the target audience. I can see Jane demonstrating a high level of organization and attention to detail much more than being easygoing and flexible. These additional ideas help us avoid getting fixated on a single (potentially flawed) initial output and make it easier to make changes based on a more holistic view of the persona.

(Our original personality trait for Jane from our research was: “Detail-oriented, organized, proactive, collaborative, results-driven, excellent communicator, empathetic leader.” So the better your research is, the more accurate your input for the AI will be, and the fewer iterations you’ll need to run through.)

3. Confirmation bias: Don't let AI be your yes-man

Humans are creatures of comfort. We like to be correct, and we prefer information that confirms our beliefs–even when those beliefs are wrong. This confirmation bias is one of the most pervasive and dangerous mental traps we face.

Let's say that after reading a few Reddit threads, we start forming the hypothesis that project managers like to make sense of chaos at work. In the prompt below, we ask our AI persona, Jane, to share what really motivates her, starting with her frustrations:

Jane confesses that moving parts are indeed some of her major pain points. But let's pause and challenge it.

I’m curious to learn what other pain points she struggles with and to understand what she intuitively “knows” others in the field complain about, which might give us an eye-opening outsider perspective. So I ask our AI persona.

By prompting Jane to dig deeper, we extract a broader range of pain points, such as communication breakdowns, scope creep, lack of visibility, and more. We might have overlooked these if we’d just let our confirmation bias guide our decision. Instead, we can use these new insights to inform our messaging, product development, and overall marketing strategy.

4. Availability bias: Don’t let the recent eclipse the relevant

Our brains are wired for efficiency, not always for accuracy. We favor information that's easily recalled: vivid memories, recent events, or juicy anecdotes. The so-called availability bias can lead us to make decisions based on a skewed perception of reality. In marketing, this bias can trick us into overemphasizing recent or memorable feedback, and overlook long-term trends.

Let's say TeamFlow has just launched a new ad campaign on LinkedIn targeting project managers. We're eager to see if it's hitting the mark, so we write: “Hey Jane, you’re scrolling through LinkedIn and come across this new ad for TeamFlow. What’s your reaction? Does it resonate with you?”

Jane replies with her feedback to the ad copy in the prompt.

Jane's response is exactly what we were hoping for: The ad copy would get her to stop scrolling and pay attention, specifically because of the relatable headline about the chaos of project management, AI automation, the benefits mentioned, and the free trial offer. But hold on: Could availability bias be clouding our judgment?

To get a clearer picture, we dig deeper and ask Jane for additional perspectives that she might be aware of from her own “experience.” A more senior project manager or someone who’s used these tools in the past might think differently about our ad:

The AI, drawing on our broader research data, might highlight potential concerns.

A more seasoned project manager might be skeptical and looking to save time, reduce costs, and increase efficiency. By challenging our initial assumptions and prompting the AI to consider a wider range of perspectives, we can uncover potential blind spots in our campaign.

This process of questioning, reframing, and incorporating diverse perspectives is the continuous feedback loop at the heart of empathy engineering.

The human-AI feedback loop

In this series, we’ve been using the P.A.T.H. (Preparation, Articulation, Testing, Harmonization) framework, which can be used as a feedback loop where each stage informs and enhances the next. Harmonization, the final step in the empathy engineering framework (and the focus of this article), is the fuel that drives this cycle. It ensures that your AI personas and the insights they generate stay relevant and aligned with real-world customer behavior, which is always changing. 

Here’s how the cycle works:

1. Preparation: You gather customer data, laying the foundation for understanding your audience.
2. Articulation: You use that data to craft AI personas that embody your ideal customers.
3. Testing: You test your messaging on these personas, gathering feedback and insights.
4. Harmonization: You analyze that feedback, refine your personas and prompts, and then…

The cycle starts all over again.

Source: Graphic courtesy of the author.

The insights you gain from Harmonization feed right back into the Preparation phase. You gather new data (with a better-informed lens to look through), update your personas, and refine your prompts, continuously getting better at understanding your customers and sharpening your marketing strategy. This constant iteration is what makes empathy engineering the solution to a problem I see all the time: companies struggling to keep up with their customers because they lack a repeatable system for gathering and analyzing insights.

By utilizing the P.A.T.H. framework in your organization, you're creating a structured and sustainable process for understanding your customers, refining your messaging, and adapting to change. And by understanding our own cognitive biases and iterating on our work, we can ensure that AI empowers us to be smarter and more empathetic marketers, not just faster ones.

Empathy is a superpower

Over the course of our empathy engineering project, we gathered data and turned it into living, breathing AI personas. We tested our ideas against them. And most importantly, we learned how to check our own blind spots along the way.

Instead of trying to guess what our customers want, piecing together bits of data and hoping you get it right, we now have a powerful system for understanding them better than they know themselves.

When we use AI correctly, it becomes a lens that helps us see people more clearly, understand their struggles more deeply, and connect with them more meaningfully. We become better observers and listeners.

And when we pair AI's ability to spot patterns with our human capacity to truly understand and care about people's needs, the results speak for themselves: Messages that hit home on the first try. Products that solve real problems, not imagined ones. And marketing campaigns that convert better because they speak to actual human needs, not just demographic data.

So start building your AI personas. Have conversations with them. Challenge them. Let them challenge you back. Use what you learn to create marketing that actually means something to people. And use it to get better at the most human skill of all: understanding people.

---

Chris Silvestri is a conversion copywriter for B2B SaaS companies and the founder of Conversion Alchemy. 

To read more essays like this, subscribe to Every, and follow us on X at @every and on LinkedIn.

We also build AI tools for readers like you. Automate repeat writing with Spiral. Organize files automatically with Sparkle. Write something great with Lex.