How to Evaluate an AI Tool Launch in 10 Minutes

Ten minutes. That's the entire budget a new AI tool launch should get before you decide whether it earns more of your time. Not an afternoon, not a Slack thread with your team, not a "let me just watch the full keynote." Ten minutes, a checklist, and a decision.

I evaluate launches every week for this site, and the volume as of this writing is genuinely absurd — some weeks there are more AI product announcements than working hours to look at them. The only way to survive that firehose is triage: a fast, repeatable pass that sorts every launch into one of three buckets before you invest anything real.

The three buckets

Every AI launch lands in one of these:

Ship-today. The product exists, you can use it now, and the gap between the announcement and the reality is small. These are rarer than you'd hope.

Beta in a trench coat. Something exists, but it's narrower, slower, or flakier than the launch copy implies. Not fraud — just ambition running ahead of engineering. Worth watching, not worth adopting yet.

Vaporware. A landing page, a waitlist, a demo video with suspicious cuts. The product may exist someday. Today it is a fundraising asset. I've written more about this pattern in the waitlist culture piece, because it deserves its own autopsy.

The 10-minute framework exists to place a launch in the right bucket with minimal effort. Here's the clock.

Minutes 0–2: the landing page sniff test

Open the site and ignore the hero copy entirely. Headlines are written by whoever on the team is best at writing headlines, which tells you nothing. Instead look for three things:

• A screenshot or recording of the actual interface, not an abstract 3D render
• A link that says "docs," "changelog," or "API reference"
• A button that starts the product rather than starting a conversation

If the primary call to action is "Book a demo" or "Join the waitlist" and there is no secondary path into the product, note it and move on. It's not disqualifying alone, but the clock is running.

Minutes 2–4: find the pricing page

Pricing is where launch theater goes to die. A team that has real users has been forced to think about what the product costs, what the tiers are, and where the free plan ends. A team that hasn't will hide pricing behind "Contact us" — or worse, publish a pricing page where every tier says "Coming soon."

Two minutes here is enough to answer: are there numbers, do the tiers map to actual usage limits, and is there a free way to verify the product works? If you want to sanity-check whether a price is in line with the rest of the category, the AI tools pricing index over at ProDealAI is a useful reference point — comparing a launch price against its category median takes about thirty seconds and catches both the delusional and the desperate.

Hidden pricing on launch day, in my experience, is the single most reliable predictor that a tool isn't ready. Teams hide numbers when the numbers aren't decided, and the numbers aren't decided when nobody's paying yet.

Minutes 4–6: the demo versus the docs

Now watch the demo — but watch it like an editor, not a fan. Count the cuts. A demo that shows one continuous session, typos and retries included, is showing you the product. A demo assembled from eight camera angles and a voiceover is showing you a storyboard.

Then open the documentation and compare. Docs are the confession booth of software: they describe what the product actually does, because docs that lie generate support tickets. If the demo shows an agent booking travel end-to-end but the docs cover only "getting started with your API key," the gap between those two is the gap between the pitch and the product.

No docs at all on launch day? That's a bucket-three signal, and a strong one.

Minutes 6–8: who built this, and are they still here?

Spend two minutes on provenance. Not a background investigation — just enough to answer whether this team has shipped before and whether the project has momentum beyond the announcement:

• Does the changelog or release history predate the launch? Products that were quietly iterating before the announcement are real.
• Is there a team page, a company footer, an actual legal entity? Anonymity isn't damning, but it removes accountability.
• Are the founders answering hard questions in public comments, or only the compliments?

That last one is underrated. Watch how a team responds to "how is this different from just calling the model API directly?" A confident team answers specifically. A launch-theater team answers with adjectives.

Minutes 8–10: touch the product or walk away

If you've made it this far, the launch has earned two minutes of hands-on time. Sign up with a throwaway email, run the most basic version of the core promise, and note one thing: did the product do what the headline said, at even 60% of the promised quality?

That's it. You're not benchmarking. You're confirming existence. Bucket one if it works, bucket two if it half-works, bucket three if you never got past the waitlist screen.

The red flags, tabulated

Any single flag is survivable. Three or more and you should close the tab — I keep a running list of these patterns in 7 Red Flags in AI Product Launches if you want the extended edition with the scars attached.

What a pass actually looks like

For calibration, here's a launch that clears triage: the product opens in the browser without a call, the free tier is usable within two minutes, pricing has numbers on it, the docs match the demo, and the changelog shows six weeks of pre-launch iteration. That describes maybe one launch in ten, as of this writing. The ratio itself is useful information — it means your default posture should be polite skepticism, and the burden of proof sits with the launch, not with you.

The mindset that makes this work

Here's my one strong opinion on this: launch-day excitement is close to worthless as a signal, and treating it as worthless will make you better at this than most of the people writing breathless day-one threads. Excitement measures marketing competence. The triage above measures shipping competence. They correlate far less than the industry wants to admit, and the tools that quietly compound — the ones you're still using in a year — usually had unremarkable launches.

Run the ten minutes. Bucket the launch. Move on with your day, and let the watch list do the remembering for you.

If you'd rather have the triage done for you, that's literally what this site is for — the weekly hunt lands every Friday with the launches that survived the framework. Join the newsletter and get the next one.

Keep reading