What I Learned Onboarding Our AI Project Manager

Was this newsletter forwarded to you? Sign up to get it in your inbox.

Every’s consulting team is growing. Right now, we have two potential new hires in a trial period: Jean-Claude, who’d manage our sales pipeline, and Claudette, a visual designer.

You might be surprised to learn that they’re both AI agents. If they’re able to reliably do what we need them to and we bring them on full-time, our team will consist of four human and three agent employees.

Claudie, our first AI colleague, has been with us for two months. Natalia Quintero, Every’s head of consulting, and I rely on her to track where every client project stands and to make sure nothing falls through the cracks, work that saves the team 15 hours per week. It’s hard to imagine operations without her.

Getting her up to speed, however, was neither a seamless nor a linear process. That road is paved with previous iterations of Claudie we had to fire because they were not structured right.

Each Claudie revealed more about what it takes to get an agent to be a reliable co-worker—lessons that have only become more urgent as more companies deploy agents, creating what Every CEO Dan Shipper has called a “parallel organization chart” of AI colleagues, each with a name, manager, and real responsibilities. At Every, we’ve started helping others build the same setup through our hosted agents, called Plus Ones. Claudie was our crash course. Here’s what she helped us figure out.

Define the job before you hire for it

Built in Claude Code—hence her name—Claudie was designed to handle administrative tasks that consumed too much of Natalia’s week. The albatross was maintaining the dashboard that shows the status of all our client work, which meant staying on top of a constant flood of information from Natalia’s email, Google Docs, Google Sheets, meeting transcripts, and her calendar. Before Claudie, Natalia was spending hours that could have been dedicated to strategy and client relations finding data across dozens of sources and manually copy and pasting it in the right tab.

The first step was to give Claudie access to various sources of information and ask her to gather everything she needed before making a single update to a client’s database, which required tracking a dizzying number of moving pieces: action items, client feedback, and names of employees who attended each client session, and on and on.

Claudie required lots of oversight at first. For example, she failed to input details discussed in client meetings and wasn’t presenting data the way we’d like—simple fixes once we realized she just needed access to Natalia’s meeting transcripts and a tool for creating pivot tables in Excel. Each time something went wrong, Natalia flagged it, and we dug in to diagnose the cause.

It’s an easy thing to overlook: Agents can only work with the context and tools you give them. Before you bring one onto your team, get specific about what they’ll be responsible for, and what information they’ll need to actually do the job.

How a 3× founder (acquired by Amplitude) decides his first 10 hires

Your first 10 hires are crucial. Patrick Thompson (3× founder, CEO of Clarify, acquired by Amplitude) created a guide with a precise, three-stage hiring sequence, tips for how to identify force-multipliers and executors, when not to hire, and how to structure a great interview. This guide helps you avoid the hiring decisions you’ll spend six to 12 months undoing.

Download the guide

P.S. We’re hosting a small NYC founders breakfast on 4/16 in Midtown—early-stage operators talking hiring, scaling, and what breaks first. Grab a spot.

Want to sponsor Every? Click here.

Understand how your agent does its best work

At first, we treated Claudie like any other new hire—telling her to find what needed updating and asking her to go do it. An experienced project manager would have hit the ground running. Claudie failed spectacularly.

The problem was the context window, or the maximum amount of text an LLM can access at one time. Claudie was trying to process too much, and information kept getting lost. So we broke Claudie into layers. We built a central orchestration agent that delegates to several fleets of subagents, each responsible for a discrete task: extracting data, identifying needed updates, and making those changes. Results improved but remained unreliable. Key dates regarding client sessions and discovery calls were frequently dropped altogether.

Our breakthrough came when we identified where communication was failing. Claudie’s subagents were gathering data and reporting it back to the orchestration agent. In theory, this should have worked. In practice, a single client update might require reviewing dozens of emails, meeting transcripts, and spreadsheets—too much for the subagents to relay without hitting the context limit. So they started summarizing the information instead of passing everything through, and the orchestration agent was making decisions based on AI recaps rather than the raw source material.

To solve this issue, we instructed the data-gathering subagents to dump everything into a local file hosted on the same computer as Claudie instead of communicating information back. The orchestration agent could then direct subagents to the relevant files to make updates without ever engaging with the data itself. Voilà—context window preserved. Once Claudie started working from raw data instead of summaries, she nailed it.

Agents process information differently from humans. But like humans, they have weak spots that can be mitigated or even solved with the right management approach.

Give your agent a handbook that is required reading

Getting Claudie’s architecture right wasn’t enough on its own. She also needed context about the role and how to do it well.

So we wrote her a handbook, as we would have done if onboarding a human project manager. Built as a project management skill in Claude, it details everything from success criteria to the team structure to when to escalate an issue to Natalia.

With a human employee, you’d hand them the handbook at onboarding and expect them to reference it as needed. Claudie’s hard-coded first step when starting up is to read the handbook to ground her in the specifics of our team and her role within it. We found that when she skipped this—which, when left to her own devices, she frequently tried to do!—performance plummeted.

We treat the handbook as a living document. As Claudie’s role has expanded, we’ve updated it to reflect her new responsibilities. For a human who learns on the job and asks clarifying questions, a slightly out-of-date handbook is no big deal. For Claudie, it’s all she knows.

Don’t be stingy with promotions

Once Claudie’s subagent architecture was stable, we expanded her responsibilities. At first, she updated each client’s dashboard individually. Once we trusted her with that, we had her do them all at once.

Right now, we’re setting Claudie up on her own computer with a Claude Max plan and web server that’s on 24/7, which will give her the ability to run automated jobs at specific times each day and always be available to respond to our messages and requests on Slack. If that goes well, Claudie will graduate from project manager to chief of staff: She’ll monitor, triage, and send emails, pick up tasks in Asana, and communicate a project’s status in Slack.

The criteria for a promotion are the same as they’d be for any team member: strong performance, a clear set of updated responsibilities, and the support and tools necessary for them to succeed in the new role.

Apply your learnings to your next hire

Onboarding Claudie wasn’t quick, nor was it easy. We rebuilt her multiple times from scratch. When we hit hour 50 of trying to get her to work, it was tempting to write off the AI entirely. When we did get Claudie to work, however, it was clear what a mistake that would have been. All we needed was the patience to figure out the right way to harness her brain power so she could deliver.

If an AI worker isn’t performing, the problem is rarely that the model can’t do the job. It’s more likely the way you’ve structured, connected, or instructed your agent. Figure out where you went wrong, fix it, and have them try again.

It’s a lesson I’ll take with me as I onboard more agents. The best thing a manager can do—for a human or an AI—is refuse to give up on a new hire before you’ve exhausted what you could be doing differently, and to believe in their potential.

To learn more about Claudie, listen to Natalia’s AI & I episode on how she automated her job.

Thanks to Laura Entis for editorial support.

Nityesh Agarwal is an engineer at Every. You can follow him on X at @nityeshaga and on LinkedIn. To read more essays like this, subscribe to Every, and follow us on X at @every and on LinkedIn.

We also do AI training, adoption, and innovation for companies. Work with us to bring AI into your organization. Discover Every’s upcoming workshops and camps, and access recordings from past events.

For sponsorship opportunities, reach out to [email protected].

If you’d like to become one of our human colleagues, explore open roles at Every.