What AI agents need before they can audit a site

An SEO agent is a model that proposes audit work: what looks broken, what to fix first, what to tell a client. It can draft that in seconds. The draft is only useful if it can inspect the same things an SEO would check before sending the note.

Most demos skip that part. They start with a prompt and end with a confident recommendation. The writing is the impressive bit. The fragile bit is everything underneath it.

What an SEO agent is

In this post, an agent is not a chatbot you paste a URL into. It is a model sitting on top of a crawl: rendered pages, extracted text, headings, links, indexability, issues, and — if you have them — semantic clusters.

Without that layer, the model is working from a snippet, a sitemap, or whatever it already knew about the brand. It can still sound sure. It cannot show you the page that produced the claim.

That is the difference between a draft you can defend and a draft you have to re-do by hand. The second kind is common. You get a tidy list of "thin pages" or "missing internal links," then you open the crawler and spend the next hour checking whether any of it is real.

In plain terms

If the crawl is the site on a table, the agent is the colleague who can sort the table and write the first version of the email. It is not the colleague who visited the site in their head.

Give it the table and it can group repeats, point at the odd row, and draft the finding with the URL attached. Give it a prompt and a domain name and it will still write something. You will be the one who has to go get the table afterward.

The scribble is cheap. The table is the part that makes the scribble worth sending.

Why the draft starts to wobble

The wobble is familiar. The first reply looks useful. Then you try to act on it.

You cannot see which rendered HTML the model used. You cannot see whether the "thin" page failed to extract. You cannot see whether two pages are competing or just sitting in the same folder. You cannot see whether the internal link it wants already exists in the footer and nowhere in the body.

So you open the crawler anyway. Check the page. Check the template. Check nearby URLs. Decide if the finding survives. At that point the agent has not removed the tedious work. It has created another artifact to review.

The quieter pattern is the reverse. Start with evidence. Then let the agent reason over it.

What counts as evidence

Evidence here is structured crawl output, not a screenshot and a vibe. For each URL, an SEO would normally want to see:

That is not a ranking rubric. It is the minimum an agent needs before "add more content" stops being the answer to every thin-looking row.

On the 105-page software crawl we measured, a privacy policy and a product page could score 0.85 similar. An agent that only saw the score, and not the URLs, could recommend consolidating them. An agent that can open both pages would not. The number is a prompt to inspect. It is not the finding.

Questions that need a crawl underneath

Once the table is there, the useful asks change. Not "find SEO issues." Things more like:

• Why are these pages competing with each other?
• Which product pages have no supporting guide nearby?
• Which internal links would connect a cluster that is already close in meaning?
• What changed between these two crawls?
• Which findings are worth sending to engineering first?

Those questions are not cleverer prompts. They are questions that only make sense when the system can point at pages, clusters, and diffs.

A generic assistant will still collapse different problems into one recommendation. Thin because the render failed, thin because the page is a stub, and thin because three URLs cover the same job are not the same fix. An evidence-aware pass can keep them apart. A paste-the-URL pass usually cannot.

What we keep next to the draft

In Silkra the agent reads the active crawl: rendered content, issues, the link graph, and the semantic groups. The draft is supposed to stay attached to the rows that produced it. That is the bar we are trying to hit. Useful writing, visible evidence, a recommendation you can open.

We are not trying to make the audit automatic. The operator still decides what matters. The workspace should make that decision easier to check, not harder to undo.

The next thing we want to write down is a real before-and-after. Same site, same questions, one pass with only a URL and one pass with the crawl table. Until we have that side by side, the honest claim is smaller: agents draft faster than we do. They do not replace the crawl.