Saraan Asim
auto-tester is an agent I started in June 2026. It logs in to a web app, navigates it and fills its forms like a user would, and produces an interactive flowchart of every distinct user flow. Wherever a real user would get stuck, hit a broken page or reach a dead end, it marks the spot. It’s in early development and isn’t a product yet.
How it runs
A Claude model drives a real browser through Playwright. At each step it looks at the page, picks one action, and the harness records the move from one page to the next. The output is a graph: pages as nodes, actions as edges, and the stuck points flagged.
I wrote the agent loop myself instead of using the SDK’s tool runner. I need to check, deduplicate and log every single step, and a runner that hides the loop takes that control away from me. The agent also gets typed tools for each kind of action, never a raw shell or a generic “use the computer” tool, so my harness can see what each action is before it runs.
The hard part: knowing when two pages are the same
A real app has close to infinite raw states. Every order detail page, every list position, every half-filled form is technically different. If the crawler treats each one as new, it never finishes, and the model bill keeps growing.
So the core of the project is collapsing raw pages into a bounded set of flow-states. /order/123 and /order/456 are the same step in a flow. The login page and the dashboard after login are not.
I split it into two tiers:
- A free structural key. The path with ids collapsed (
/order/:id), a signature of the page’s forms, buttons and links, and a rough page role such as login, list, detail, form or error. For most pages this settles it without calling a model. - A small model only when that is ambiguous. When two pages share a path but look different, or a single-page app changes view without changing the URL, a cheaper Claude model answers one question: is this the same state as that one? I run it as a separate call, so the main loop’s prompt cache stays intact.
Safety
The agent acts on a logged-in account, so it could delete data or buy something. I switched destructive actions off by default, and I only point it at staging or disposable apps, never production.
How I’ll know it works
Before writing the crawler I wrote down what would count as working, so the verdict is a measurement and not a feeling. Every run logs tokens per state, the prompt cache hit ratio, how many states it found before stopping, and how long it took.
The test that decides whether the project continues is a full crawl of a small test app whose flows I have counted by hand. The crawl has to finish because there is nothing left to explore, not because it hit a cap. The cache hit ratio has to be at least 0.8 after warm-up. The number of states it finds has to land within about 1.5 times the real number of flows. If it can’t do that on a simple app, the ceiling is low, and I’d rethink the scope before building more.
What comes next
I’m building the engine first: explore, map, flag where users get stuck. After that comes a React Flow front end for the flowchart. There are no crawl results yet. I’ll write them up when the first full crawl finishes.