Time to first win: finding the friction by building an AI workflow
Published: October 15, 2026 · How I AI · Co-author: Laetitia Beyegue (PM)
What does it take to get a new person from "I signed up" to "it worked"? And how do you compare that across products when every review starts from scratch? Those questions turned into a small AI workflow, an onboarding analyzer, that Laetitia (PM) and I built in our spare time. This post is the demo, the rules we ended up with, and the parts that didn't go the way we expected.
What it does
You give it public material: documentation, fee and incentive terms, notes, and screenshots of the signup flow. It gives back:
- a step-by-step map of the funnel
- a friction score for each step, with a reason someone might drop off
- the incentives on offer
- a ranked list of onboarding experiments, each with a hypothesis, a primary metric, and a guardrail metric
The goal isn't to judge a product. It's to find where the path to a first win is longest and what we'd test first.
How it's put together
Three pieces, each doing one job.
- A research step. A prompt that tells Claude to find the official documentation, read it, and write paraphrased notes with the URL and last-updated date for each page.
- A rubric. The same fields for every product: what the person does, what they need to provide, how long they wait, what kind of friction it is, a score from 1 to 5, and any incentive.
- An interactive page. It takes the notes and screenshots, runs the rubric, and shows the funnel, the incentives, and the ranked tests. You can save results and compare products side by side.
The research lives in a Claude Project so every analysis starts from the same instructions. The analysis lives in a small page we can share as a link.
Three things I learned
1. The output is only as good as the evidence
Our first version would happily analyze a company we'd given it nothing about. It said so politely, but it still ran. We added a check that stops the run if the notes don't mention the company, and a clear message about what to do instead. The workflow is more useful now that it refuses to guess.
2. Tag every claim
Every step in an analysis is marked observed (the source says it), inferred (our reading of it), or unknown (the source is silent). Our friction scores are judgment calls, and the tags keep that visible. When we read an analysis later, we can see in one glance which parts are facts and which parts are us.
3. Conflicts between sources are findings
Official pages and third-party guides don't always agree. A guide might describe a stricter process than the company's own page does. We used to resolve that quietly. Now we keep both versions and flag the conflict, because a gap between what people read and what actually happens is useful to know about.
Consumer products vs. developer tools
The method works on both, but what counts as a first win, and what counts as evidence, changes a lot. Here's how the two compare.
| Consumer product or marketplace | Developer tool | |
|---|---|---|
| The first win | A purchase, a first post, a first sale | A first working API call, deploy, or query |
| Where friction hides | Account creation, identity checks, payments, trust | Setup, API keys, install steps, confusing errors, documentation that doesn't match the UI |
| Main evidence | Help center, app screens, fee and incentive terms | Quickstarts, documentation, SDKs, changelogs, sample repos, pricing and free-tier limits |
| What a screenshot shows | A lot, since the flow lives in an app | Little, since the real flow is in a terminal, a console, and code |
| Incentives | Promo credits, fee discounts, referral bonuses | Free tiers, credits, template galleries, community and support |
| Best tests | Funnel steps, incentives, prompts | Shorter quickstarts, templates, clearer error messages, in-product guidance |
How a developer-to-developer comparison differs
- It's easier to verify. You can run the quickstart yourself and time it, so claims can be observed instead of inferred.
- Documentation is the product. Quickstart length, steps to a first success, and whether the documentation matches the real interface become core fields.
- The evidence tags matter more. Developers check what you wrote, so labeling what's observed, inferred, or unknown earns trust.
- The comparison is fairer. Two tools in one category with a clear first success, such as an email API, a hosting platform, or a database, compare directly. Consumer comparisons often mix very different business models.
What stays the same: the evidence tags, treating conflicts between sources as findings, friction scores, ranked tests, and a list of what public material can't show.
What the experiments are
The experiments are onboarding experiments: changes to the path a new person takes between signing up and their first win. Each one is about a single step in the funnel, and each is written so a team could actually run it.
Every experiment has the same parts:
- The step it targets. Which point in the funnel it changes, such as identity verification or setting up payouts.
- One lever. The single thing you'd change, such as moving a step later, cutting a form field, or adding a progress indicator. One lever keeps the result readable.
- A hypothesis. Why we think that change will help, tied to the friction we saw at that step.
- A primary metric. What should move if we're right. It's tied to activation (the first win) or to a leading indicator of it, like the share of people who finish that step.
- A guardrail metric. What shouldn't get worse while we chase the primary one. If we shorten identity checks to lift completion, the guardrail might be fraud or chargeback rate.
- Impact and effort scores. Each from 1 to 5, used to rank the list.
- A type. Systemic experiments change the structure of the path, like reordering steps or letting people try before they verify. Paper cuts remove local friction, like unclear copy or an extra tap. Paper cuts are cheap and quick to ship. Systemic changes usually carry more upside and more risk.
For example, if a seller flow asks for identity and payout details before the person can list anything, an experiment might be: let people create a draft listing before verification. The hypothesis is that people who see their listing taking shape are more likely to finish verification. The primary metric is the share of new signups who complete verification. The guardrail is the rate of problem accounts reaching the marketplace.
The tool doesn't claim the experiment will work. It gives you a ranked list of the best places to look, and the "What I can't see" list tells you which internal numbers to check before building any of them.
What I can't see
An analysis built from public material can't tell you drop-off by step, time to first success by cohort, or how many people hit a review queue. Every result ends with a list of the internal data we'd need to confirm or kill each idea. Without that list, the output reads as more certain than it is.
Try it
The demo is here. The researched examples (Whatnot, TikTok Shop, and a fictional one) run saved teardowns right on the page. Analyzing your own notes needs Claude, so those runs happen in the Claude version of the tool. If you run it on a product you know well, we'd like to hear what the rubric misses. What would you add?