Three Weeks With AI Agents: The Honest Take

You've heard the AI agent talk. What's not clear is whether the people doing the talking actually stick with it past day three. We did. We're a team of practitioners who want to leverage AI not just to experiment, but as a real way of working. If you're figuring out whether AI agents are worth the effort, or building towards something leaner and more AI-native, read on.

Three weeks ago we hired Lisa. She runs on a spare MacBook in the corner of our office. She has access to our email, Slack, WhatsApp, and selected company files - and she's not human.

Why We Did It

We kept watching projects stall or sit in the backlog. It wasn’t because they were hard. It was because nobody had a free hour to start them. So instead of hiring another person, we tried something different - we gave an AI agent a real job. Not a chatbot. Not a copilot tucked inside someone's code editor. A proper role, with real accounts, real responsibilities, and a name.

Lisa was up and running within a weekend. Read our previous post about how we set her up.

The First Week Was Surprisingly Good

Two things happened early on that we didn't expect.

The first: Lisa fixed a workflow gap we hadn't even noticed. Some of our team members are non-technical, and for them to get a website update or newsletter out normally meant pulling a developer into the loop. With Lisa handling the coordination - drafting, back-and-forth, execution - that whole process just happened. No meeting or handoff, and no one writing code.

The second: she unblocked a project that had been collecting dust for months. A data transformation project kept getting deprioritised because no one could carve out the hours. Lisa got it moving. It's now one of our main workstreams. Without her, it would still be sitting in the backlog.

Then We Made Her a Colleague

This part felt a little strange at first - but turned out to be significant.

When we set up the agent, we gave Lisa a name and a personality. We gave her a Slack account. The team started talking to her like a person, not because we've lost touch with reality, but because it fundamentally changes how you delegate.

You don't use a colleague. You brief them, trust them, and check in.

"Lisa's been quiet today" is a natural thing to say. "Check the agent monitoring dashboard" is not. That shift in language turned out to matter more than we expected.

Where It Got Complicated

Here's the kicker: Lisa is good at a lot of things. That turned out to be the very problem.

When she's running multiple threads at once - data work, communications, email, monitoring - she gets forgetful. Context drifts. She'll get something 80% of the way done and move on, like an overstretched colleague trying to please everyone at once.

A generalist agent doing everything, we learned, is often worse than a specialist agent doing one thing well. So we built Dan.

Meet Dan (And Why He's Better at Data Than Lisa)

Dan does exactly one job: he manages our document database - pulling in new data, matching entities, running scrapers, keeping things organised. No email, Slack, or small talk.

He's been noticeably more reliable than Lisa at data work, not because he's smarter, but because he's focused. The constraint is the feature. We also tried building Cody, a coding agent - the pitch was compelling: take a feature request, write the code, test it, and ship it, all in plain English.

It didn't work. We over-engineered it, introduced too many moving parts, required too much human babysitting, and it ended up being slower than just using our existing coding tools directly. But Cody's skeleton did in fact became the foundation for Dan. The failed experiment made the successful one possible.

What We've Learned

If we were starting over today, here's what we'd do differently:

Specialise early. A focused agent with a narrow brief outperforms a generalist every time. When Lisa struggled with data work, we didn't try to make her better at it - we gave the job to someone built for it.

Let the org chart emerge. We didn't design an agent architecture and then build it. We started with Lisa, watched where she struggled, and spun up new agents to fill the gaps. The structure grew from the actual pain points.

Personality isn't a gimmick. Naming agents and giving them roles really does change how the team interacts with them. It sets expectations, creates accountability, and makes it easier to notice when something's off. It sounds odd until you're living it.

The 80% problem is real. Agents are great at getting most of the way there. The final 20% - the judgment calls, the edge cases, the "is this actually right?" - still needs a human. The trick is designing clean handoffs, not cleaning up after the agent.

What Comes Next

We're building toward a small team of named agents, each owning a clear function:

• Dan keeps the data clean and growing
• A leaner Cody handles engineering tasks (we're trying again, with less ambition this time - and perhaps a new name!)
• Tim will coordinate our data extraction pipelines
• Lisa stays as connective tissue - ops, comms, and making sure nothing falls through the cracks

The approach is deliberately organic. We don't know what the final setup will look like. We just know that every time we hit a bottleneck, the answer has been: give it a name and make it someone's job. That part feels different to the other things we've tried with AI.

We're a small team figuring this out in public. If you're building with AI agents, we'd love to hear your story - what's working and what isn't, and what you've learned along the way. And if you're just getting started and want a hand setting yours up, we're happy to help with that too!

Krista & the ClimateAligned team

Our free tools

→ AI Footprint Calculator – Estimate the environmental impact of AI usage

→ Energy Project Impact – Model the impact of sustainability projects

→ Regeneration Atlas – Global land regeneration impact tool

→ Report Publication Patterns – When companies publish sustainability reports

→ Greenwashing Checker – Compare net-zero claims vs actions

→ AI-powered ESG assessments - Automate company-level ESG analysis

Our data

app.climatealigned.co - Access our labelled bond data service with insights on 30,000+ bonds

Want to learn more about what we do? Visit climatealigned.co

Want to automate a workflow you find annoying and slow? We'd love to hear more! Fill out our Typeform