How to choose an AI agency without getting burned.

The AI services market in 2026 is the Wild West. Every agency that did WordPress sites in 2019 now has "AI" on their homepage. Every consultant who spoke at a conference once calls themselves an "AI strategist." And every big four firm will happily sell you a $400,000 "AI readiness assessment" that produces a PDF you could have written with ChatGPT in an afternoon.

We've seen it from both sides. We've rebuilt projects that other agencies abandoned. We've seen clients burn six figures on pilots that never made it to production. And we've watched smart operators get suckered by slick decks full of words like "agentic" and "multimodal" that hide one hard truth: the team selling to you can't actually build it.

Here's how to avoid that.

Ten questions to ask

Run these in your first or second call. Any agency worth hiring will answer them clearly without hedging. If you get vague, marketing-speak answers — that's the answer.

1. "Can you show me three projects you've shipped in the last twelve months?"

Not case studies. Not deck slides. Not logos. Actual shipped systems running in production. If they can't name three, they're either too new, too stalled, or too focused on pitching instead of building.

2. "Who specifically will write the code?"

Some agencies pitch with senior engineers then staff the actual project with juniors or offshore contractors you never meet. Get names. Get LinkedIn profiles. Get one intro call with the person who will actually be typing.

3. "What stack will this run on?"

If the answer is a proprietary platform only they can maintain — that's a lock-in you'll pay for forever. If the answer is a standard stack (Python, Node, Postgres, a major LLM provider, open-source orchestration) you can take the project elsewhere if things go wrong.

4. "Who owns the code when we're done?"

The only correct answer is "you do." If there's any caveat, push back until there isn't. No perpetual license. No hosted-only. No "we retain rights to the agent framework." You. Own. The. Code.

5. "What happens when the model gets something wrong?"

Any real AI engineer will answer this in thirty seconds with concrete techniques: confidence scoring, human-in-the-loop routing, eval suites, fallback paths, logging so you can audit failures. If they wave it away with "we'll fine-tune it" or "GPT-5 doesn't really hallucinate anymore" — run.

6. "Show me your evaluation framework."

AI systems without evals are vibes. You want to see how they measure whether a change to a prompt, a model, or a retrieval strategy actually made things better. If they don't have an answer — they're shipping on faith.

7. "How do you handle data privacy and PII?"

The honest answer is nuanced: some data stays in your infra, some hits the model provider's servers with a zero-retention agreement in place, some is redacted before it ever leaves your systems. Vague answers here are a data-protection lawsuit waiting to happen.

8. "What's the total cost of ownership for the first two years?"

Build cost is the sticker. Ongoing cost is the real bill. Ask for a two-year TCO including model usage, hosting, monitoring, and a realistic retainer for iteration. If they can't produce this — they haven't thought about it, and you'll find out the hard way.

9. "Can I talk to one of your past clients?"

Not the one in the case study. A different one. If they dodge, stall, or produce only the same reference every time — assume the roster is thinner than it looks.

10. "What would you recommend we don't do?"

This is the tell. A real practitioner will happily tell you which parts of your idea are bad, which shortcuts are too risky, or which features will cost ten times what they're worth. A salesperson will tell you every idea is brilliant.

Five red flags to watch for

If you see any of these — slow down and ask harder questions. If you see two or more — walk away.

Red flag 1: They won't quote a fixed price or a tight range.

Good AI engineering work can be scoped. "We'll do time and materials and see how it goes" is often a sign that the agency doesn't know how to estimate — or knows the number would scare you.

Red flag 2: They talk about "AI transformation" more than specific systems.

"Transformation" is consulting-speak for "we'll bill you monthly to tell you what you already know." You don't need a transformation. You need two or three specific systems that save hours or make money.

Red flag 3: Their portfolio is all demos, no production deployments.

Demos are easy. Production is the part that's hard. An agency whose case studies are all "we built a prototype" has never shipped anything that had to survive a real user base, a real edge case, or a real on-call rotation.

Red flag 4: They resell a single platform and everything looks like a nail.

Some agencies are really "Salesforce partners" or "HubSpot partners" with AI sprinkled on top. Their answer to every problem will be the platform they get a kickback from. You want a team whose interests align with building the right thing, not the thing that pays them a rebate.

Red flag 5: The proposal is all strategy, no build.

If the first six months are "discovery," "alignment workshops," and "roadmap development" with no working system at the end — that's a consulting firm pretending to be an engineering firm. You can get that deck cheaper on Fiverr.

The single most important thing to get in writing

There's one clause that matters more than any other:

"Client will own all intellectual property, source code, model weights, prompts, evaluation data, and infrastructure-as-code produced under this engagement, delivered in a format that allows client to run, modify, and transfer the system without the involvement of [Agency]."

That's it. That clause protects you from every single bad outcome. If the agency goes out of business — you can hire someone else. If the project goes sideways — you can hire someone else. If they try to double their retainer next year — you can hire someone else. The optionality is everything.

Some agencies will push back on this. They'll say they need to retain rights to a "framework" or a "platform." The correct answer is: fine, but then the framework gets MIT-licensed to us, in writing, as part of delivery. If they won't do that — they're betting on lock-in. That's a bet you don't want to be on the wrong side of.

One last piece of advice

Small is usually better than big for AI work right now. A three-to-five person shop with strong engineering DNA will almost always outperform a 200-person consulting firm at this. The reason is simple: AI engineering is still mostly craft. The senior people who are good at it want to be writing code and shipping things, not managing an army of juniors and filling out timesheets.

When you're evaluating agencies, ask yourself one question: do these people seem like builders? Do they light up when you describe the problem? Do they ask specific technical questions? Do they push back on parts of your idea that don't make sense? Or do they smile, nod, and send you a SOW the next morning?

The builders are the ones you want. Find them. Pay them. Treat them well. And insist on that IP clause.