The 5-Minute Test for Whether Your AI-Powered Agency Sells Judgment or Just Software

Ask any agency pitching "AI-powered" outbound whether you're paying for their judgment or for a wrapper around a tool you'll be able to run yourself in two years, and most won't give you a straight answer. Here's a five-minute test that answers it for you, and it works on any agency you're evaluating, including mine.

The Pipeline Test

I didn't invent this. It's a framework from Amos Bar-Joseph's newsletter The Autonomous Age, from his July 2026 piece "AI just killed the software business. Nobody's said it out loud yet." The mechanic: pull up your last five closed deals. Ask what they actually paid for, not the demo, the outcome. Rewrite one pitch this week without mentioning the product once.

Then see if it still lands. That last part is my own addition to the original test, not his line, but it's the part that actually tells you something. If the pitch collapses without the product name in it, you were selling software wrapped in a service fee. If it still lands, you were selling judgment, and the product was only ever how you delivered it.

What the client actually pays for

I ran the test on my own work before asking anyone to run it on me.

A logistics and freight-forwarding client pays 4,500 ILS a month. Leaning on the tool, the pitch sounds like "12 mailboxes across 6 domains, multi-channel outbound." Strip the product out of that sentence and what's left is knowing which channel actually reaches a freight forwarder's buyer, and what message gets that buyer to reply, sustained month over month at roughly 15 qualified leads. The mailboxes are the delivery mechanism, not the thing being bought.

Same test, different client: a regulated financial-services company, LinkedIn only, 11 months running, a 20 to 30 percent reply rate. Take "LinkedIn campaign" out of that pitch and what's left is judgment about which angle a compliance-heavy buyer will actually respond to, in a world where almost nobody in that world replies to anything. Both pitches survive the rewrite. That's the honest result of running the test on my own numbers, not a guaranteed one.

The pattern shows up bigger than one pitch

Rippling grew 78% year over year at over $1B in ARR as of March 2026, by CEO Parker Conrad's own public numbers, independently corroborated. Workday grew 13.1% in the same period, decelerating from 16.4% and 16.8% the two years before. Rippling runs roughly 300 human SDRs to do it, not fewer. Nobody automated the sales org away and came out ahead. That's one data point, not proof of the whole thesis, but it points the same direction the Pipeline Test is built to expose: the company that kept judgment in the loop outgrew the one leaning harder on scale.

Run it on me too

Take the same five minutes and run it on whoever is pitching you "AI-powered" anything right now, including me. Ask for their last five deals. Ask what the client actually paid for. If the honest answer is "the tool," you're paying for something that gets cheaper and more automated every year, and the retainer should reflect that. If the honest answer is a judgment call, about channel, about message, about which client gets turned away, you're paying for something an AI release doesn't touch.

I sell the second kind. Don't take my word for it. Run the test.