metatron-open-seo/docs/site-audit-pm-research.md

OpenSEO Site Audit: 80/20 Feature Set & Crawl Architecture Proposal (rev. 3, founder decision applied)

Author: PM research synthesis | Date: 2026-06-09 | Inputs: SEOnaut + LibreCrawl code reviews, OpenSEO baseline review, competitor/market research, architecture research, adversarial panel review. Revision notes: panel critiques accepted are folded in silently; where we overrode the panel, an explicit Panel pushback note marks the disagreement. Rev. 3: founder decision — build on the in-house Workers crawler, do not adopt DataForSEO OnPage; "we were blocked" flagging accepted as the blocking posture. §4 now weighs that decision's tradeoffs instead of recommending an engine.

---

1. How site audits actually work (primer)

Every site audit tool is the same three-stage pipeline:

1. Crawl. Start from a URL, fetch robots.txt and sitemaps, fetch pages, extract links, enqueue new same-site URLs (BFS), repeat until a page cap. Two engine choices matter: plain HTTP fetch (fast, cheap, blind to JavaScript) vs headless rendering (sees what Google sees, ~10x cost). JS rendering is table stakes in commercial tools; differentiation is fidelity and whether it's paywalled.
2. Check. Per-page rules (missing title, noindex, broken canonical) plus cross-crawl rules (duplicate titles, redirect chains, orphan pages). The cross-page checks carry most of the value, and they're set-queries over crawl data, not parsing.
3. Report. Severity grouping, plain-English explanations, drill-down to affected URLs, export, and diffs against previous crawls.

Desktop vs cloud. Screaming Frog ($259/yr) and Sitebulb run on the user's laptop: residential IP, no bot-blocking, but no scheduling, RAM-limited crawls (the #1 SF complaint), and the machine must be on. Cloud tools (Ahrefs, Semrush, Moz) crawl datacenter IPs on a schedule — and getting blocked is their #1 recurring complaint: WAFs 403/challenge their bots, producing phantom 404s and false positives.

Why blocking happens. WAFs score IP reputation (datacenter = suspicious) and fingerprint (no JS = bot). It hits every cloud auditor, including ours. The industry playbook: documented UAs, published allowlistable IPs, Cloudflare's Verified Bots program, and ownership verification to justify higher crawl rates (Ahrefs advertises dramatically higher rates for verified owners; the oft-quoted 30→30,000 URLs/min figure is unsourced in our fact base and Ahrefs' verification is its own, not GSC — treat as directional only).

---

2. Where OpenSEO's audit stands today

Honest summary: we have a crawler and a Lighthouse integration, but not an audit product. We extract per-page fields and show filterable tables, but no rule engine names issues; the only "issues" users see are Lighthouse failures. SEOnaut ships ~88 issue types; LibreCrawl ~26; Screaming Frog ~300. We effectively ship ~2 as table styling.

Structural problems, in priority order:

• The link graph is thrown away. Link lists are extracted but only counts persist to D1, so broken links, orphans, redirect chains, and inlink analysis are impossible without schema change.
• Persistence is finalize-only and Workflow-shaped. runCrawlBatch returns full page arrays (links, images) as durable step state (~1MiB/step cap — already borderline on link-dense sites), allPages accumulates everything in 128MB Worker memory, and batchWriteResults writes all rows inside the single finalize step (DB_BATCH_SIZE=100 → a 1M-row link graph would mean ~10,000 subrequests in one step, ~10x the limit). Any link-graph feature requires restructuring persistence first. This was deferred to P1 in draft 1; the panel correctly identified that as backwards — it is now the P0 prerequisite.
• Step budget caps crawl size — ledger, not vibes. Workflows allows ~1,024 steps/instance. Today: 3 steps per 25-page batch → ~341 batches ≈ ~8.5k pages fits (minus discovery/finalize overhead), so the existing 10,000-page clamp slightly exceeds the budget. The sharper constraint is Lighthouse strategy all: 2 steps per 10-URL batch ≈ 1,000 steps at 5k pages — over budget on its own. Politeness delays must live inside steps (wall-clock waits are fine), never as smaller batches. See §4 for the post-refactor ledger and caps.
• No audit MCP tools — keywords, SERP, backlinks, rank-tracker, GSC all have them; audit has zero (verified in src/server/mcp/tools). Ahrefs, meanwhile, already ships site-audit MCP tools (site-audit-issues, site-audit-page-explorer, etc.); Semrush has an MCP server too. We are behind here, not ahead — see §3.4 for what our actual wedge is.
• SSRF gap: url-policy.ts validates only the start URL; discovered links are fetched unvalidated. Must be fixed in P0 for as long as the in-house crawler runs.

Condensed check inventory: status/redirects, title/meta, canonicals, indexability, headings, images = partial (extracted, unflagged; redirects followed silently; header variants like X-Robots-Tag unchecked). Duplicates (30–35% of sites, Semrush study¹), broken links (52% of domains¹), redirect chains (~12%¹), orphans, security headers, hreflang, JS rendering, scheduling/diffs, issue UX, MCP, block resilience = missing. Lighthouse/CWV = strong, better than both OSS tools. (¹ Prevalence figures from market research §3; primary sources to be attached as a footnoted appendix before this circulates externally.)

(Folklore correction stands: the 100k-page lifetime cap settles to actual crawled counts; only failed/stuck audits permanently pay their reservation.)

---

3. The 80/20 feature set

Pricing posture (decides everything downstream). Audit is the most commoditized feature in SEO — Ahrefs Webmaster Tools and Screaming Frog ≤500 URLs are free. Per our SEO strategy (goal = signups), audit is acquisition, not monetization: default 50-page audits are free per site; credits are charged for large crawls, scheduling, and history. The free posture also reinforces the engine decision (§4): with the in-house crawler, a free audit's marginal cost is D1 writes and Workers compute largely inside quotas we already pay for — no per-audit vendor bill, no metered cost on failed runs (DataForSEO charges even for failed tasks).

The prevalence evidence says a credible audit needs ~6 check families, not 300. P0 below runs on the in-house crawler (founder decision, §4); crawler-side hardening is therefore mandatory P0 work, not contingency. Everything except the crawler-side items remains engine-independent by design (schema, SQL checks, UX, MCP) — that portability is the cheap hedge that keeps a future vendor or local-agent engine swap on the table.

P0 — the audit becomes real (~6–8 weeks)

0. Incremental persistence (prerequisite, was P1). Write audit_pages/audit_links rows to D1 inside each crawl-batch step (one extra db.batch per batch keeps subrequests bounded), return only slim summaries (id, url, status) as step state, drop the allPages accumulator. Finalize becomes: multipage SQL step → completion. Bonus: failed audits keep partial results. Also fix the SSRF gap (run discovered links through url-policy).

1. Issue engine + schema. audit_links edge table (source, target, anchor, rel, internal) and audit_issues table. Reporter contract (decided now, per panel): reporters consume a normalized page record — (record: AuditPageRecord) => boolean where the record includes response headers captured at crawl time — not SEOnaut's (page, $, headers) cheerio signature. Cheerio runs once in crawlPage; reporters are DOM-free. This is what makes the engine portable across our crawler, a future local agent, or a vendor feed if the §4 revisit triggers ever fire. Cross-page checks are D1 SQL passes (duplicates = GROUP BY hash HAVING count > 1, per SEOnaut's multipage reporters). Effort: 2–3 weeks including #0's restructure (was "1 week" — panel was right).

2. High-prevalence checks (post-#1; thresholds cribbed from SEOnaut/LibreCrawl):

• Broken internal links + redirect chains/loops: switch to redirect: "manual", record hops, enqueue targets. Decision: redirect hops do not count against maxPages or billing (they're cheap HEAD-sized fetches; metering/progress semantics budgeted at +1 day). Broken-link checks fire only for targets we actually fetched or HEAD-checked — never inferred from absence on partial crawls. ~4 days.
• Duplicates: body-hash group-by catches mirrored/parameterized duplicates only (any dynamic token defeats exact hashing); duplicate titles/descriptions group-by is the workhorse and carries the prevalence claim. Avoid LibreCrawl's O(n²) similarity scan. ~1–2 days.
• Title/meta length, missing/multiple H1, heading order (SEOnaut thresholds). ~1–2 days.
• Canonical + indexability with header fallbacks (X-Robots-Tag, Link: rel=canonical — SEOnaut's standout trick; headers are in the normalized record by design). ~2 days.
• Orphan pages: reorder seeding to sitemap-last (we currently seed sitemap URLs first, the opposite of the SEOnaut pattern we cite, so sitemap pages eat the page budget and link discovery never exhausts), and emit orphan findings only when the crawl completed (queue exhausted below maxPages) — on truncated crawls "no observed inlinks" is true of nearly everything and would mass-false-positive, the exact Semrush health-score noise we mock. ~3–4 days including reordering and gating.
• Thin content + crawl depth: explicit disposition — build in P0 (word count already stored; BFS depth is one column). Page size: P1. ~1 day.

3. Issues UX. Critical/Warning/Info grouping, one-paragraph plain-English explanation per type (adapt SEOnaut's YAML copy), affected-URL drill-down, LibreCrawl's ~100-glob default exclusions (wp-admin, checkout, feeds), and CSV export (previously unaccounted; it's one of the three core report capabilities from §1). ~4–5 days.

4. MCP audit tools — reframed. Draft 1 claimed "a demo no competitor can match." That was wrong: Ahrefs ships site-audit MCP tools today and Semrush has an MCP server. Our wedge is not MCP existence; it's credit-priced, zero-setup, agent-actionable audits: audit any URL on demand for cents with no project setup or $129+/mo subscription, and return per-issue fix instructions an agent can execute (the issue record carries remediation steps, not just labels). Tool shapes designed against Ahrefs' actual, inspectable MCP audit tools: run_site_audit, get_audit_status, get_audit_issues(severity, type), get_audit_pages(filter), each issue carrying how_to_fix. ~4 days.

5. Block detection — moved to weeks 1–2 (was week 5). The panel is right that for our segment (indie founders on Vercel/Cloudflare/Netlify — bot protection there is a one-toggle setting and IP-reputation challenges hit datacenter egress even without it; the week-1 baseline will establish the real rate) blocking is a first-run activation killer, not an ops metric — and our own onboarding audit shows activation leaks are the existential problem. Classify 403s/challenge bodies/cf-mitigated headers as "we were blocked," say so honestly instead of recording error pages, and show owner-targeted allowlist instructions inline (GSC-verified flow, §4). This layer is response-classification, hence engine-independent, and instruments block-rate telemetry from day one. ~3 days.

P1 — weeks 8–14

• Scheduled audits + diffing: extend the existing cron; store per-run issue counts, show deltas ("12 new broken links since last week"). The retention feature and the reason cloud beats desktop. Data retention policy (new): full link graphs kept for the latest 2 runs only; older runs keep issue counts/deltas (a 10k-page graph is ~100–150MB against D1's 10GB cap — weekly large audits would eat gigabytes/year otherwise). Large-audit edge lists optionally archived as compressed R2 blobs.
• AI-crawler accessibility audit — promoted from P2. Which AI bots (GPTBot, ClaudeBot) can reach your content, robots/Cloudflare AI-block detection, is-key-content-in-static-HTML. The panel is right that our own words ("open greenfield, perfectly on-brand") contradicted P2 placement: it's mostly computable from data we already collect, and "is ChatGPT blocked from your site?" is the launch/marketing asset for the whole relaunch — nobody switches auditors for HSTS checks. (Skip llms.txt as a headline; no major LLM vendor consumes it.)
• JS rendering — deferred (consequence of the no-vendor decision, §4). Plain fetch can't render SPAs. P1 ships the honest mitigation instead: detect client-rendered shells (near-empty body text with a large script payload / root-div-only HTML) and flag "this site renders client-side; HTML-level checks are incomplete" rather than reporting false missing-title/thin-content issues. If demand materializes, the revisit path is Cloudflare Browser Rendering as an opt-in, credit-priced add-on for GSC-verified own-site audits (where the owner can allowlist, neutralizing the fingerprint problem) — not a vendor API.
• Security/hygiene checks (HTTPS, mixed content, HSTS/CSP, insecure forms) and external broken links (HEAD checks, per-crawl cache) — demoted below AI-crawler work.
• Cloudflare Verified Bots: reframed as a lottery ticket, not a mitigation. Eligibility requires established, meaningful crawl volume; a new low-volume crawler routinely doesn't qualify, and Web Bot Auth is beta on Cloudflare's side. File opportunistically for our own UA (OpenSEOBot); plan zero dependency on it. The owner-targeted allowlist flow is the only mitigation we count on.

P2 — later

• npx openseo-audit local agent (see §4 option D) — gated on block-rate telemetry, which now exists from week 1.
• Crawl snapshots in R2 ("what we saw" evidence; lightweight SEOnaut WACZ analog).
• Hreflang validation, structured-data validation, page-size checks, link-placement classification.

Skip entirely (explicit dispositions): 300-check parity, crawl visualizations, E-E-A-T scores, multi-language UI.

---

4. Architecture: where should the crawl run?

	A. In-house Workers crawl + mitigations	B. DataForSEO OnPage offload	C. Desktop app	D. Hybrid: cloud + npx local agent
Blocking reliability	Low-medium: no stable IPs, no JS-challenge survival	Medium: still datacenter, but published IPs + named UA are allowlistable	Highest (residential IP, real browser)	High when used; manual trigger only
Scheduled audits	Yes (≤ a few k pages — step budget)	Yes	No	Yes (cloud path); no for the blocked sites D exists for
JS rendering	No	Yes (priced)	Yes	Yes (local Playwright)
Eng effort	Ongoing hardening forever	Low (vendor/SDK/billing integrated)	Months + signing/notarization/3-platform QA	~3–5 weeks (see below)
Cost per 1k pages	Not $0: D1 link writes ≈ $0.30/1k (with indexes) + Workers/Workflows	~$0.125–0.50 base, ~$1.25 JS (list prices from architecture research — re-verify if the §4 revisit triggers ever fire)	$0 marginal	$0 marginal + ingest
Max crawl size	~8.5k today, ~12–25k post-refactor (Lighthouse all binds first; instance chaining removes the ceiling)	Vendor-side, no Workflow ceiling	RAM-limited	Local machine-limited
MCP / billing fit	Good / fine	Good / per-page cost maps to credits	Bad / awkward	Excellent for developers / fine

Excluded options, explicitly (panel asked): Cloudflare Browser Rendering — solves JS but not blocking (egress is still Cloudflare datacenter IPs), adds per-browser-hour cost, and headless-Chrome fingerprints start their own anti-bot treadmill. With the vendor path now declined (see decision below), this becomes the only cloud JS-rendering route (the P2 npx agent's local Playwright is the other) — scoped as a P2 opt-in for GSC-verified own-site audits where the owner can allowlist us first. Residential/ISP proxy egress — rejected on principle and practice: it disguises crawler identity, which contradicts our documented-UA/allowlist posture and many proxy networks' sourcing is ethically murky; cost (~$1–8/GB) is also worst-in-class.

Decision: A — in-house Workers crawler (founder call: avoid DataForSEO vendor costs). The original recommendation was B, contingent on a week-1 spike; the founder has decided against adding vendor spend, and "we were blocked" honest flagging is accepted as the blocking posture. That is a defensible position, but the proposal's job is to price it accurately. What the decision buys, and what it costs:

What going in-house buys:

• No per-audit vendor bill on a free acquisition feature. Every OnPage call is marginal cash out (~$0.006–0.025 per free 50-page audit, $1.25–5 per 10k-page crawl at list prices), and DataForSEO meters even failed tasks — a real footgun we've already hit on the SERP side. Crucially, the comparison must not double-count: D1 storage writes are common to both engines (vendor results would land in the same audit_pages/audit_links tables to run the same SQL checks), so the vendor fee is purely additive. In-house wins on marginal cost at every crawl size.
• No new vendor dependency or pricing-change exposure on the product's centerpiece free feature; full control of crawl behavior, UA identity, and politeness.
• One less integration: no task-queue/webhook polling model, no mapping a vendor's page schema into the normalized record.

What it costs — priced honestly:

1. "Basically free" is true only until the free tier works. D1 Paid includes 50M row-writes/month shared with the entire product. A 50-page free audit writes ~8–15k rows (pages + links + indexes), so ~3–5k free audits/month consumes the whole included allotment; beyond it, $1/M rows — at 10k free audits/month, ~$50–100/month in-house vs ~$60–250/month vendor-side. A ~2x cost win, not zero, and it eats quota headroom from every other D1 feature. Required companion: a TTL on free-tier audit data (e.g., purge one-off free audits' link graphs after 30 days) — the P1 two-run retention policy only covers scheduled audits and doesn't protect the 10GB D1 cap from thousands of one-off free crawls.
2. JS rendering is off the roadmap until/unless we build it on Cloudflare Browser Rendering. SPA-heavy sites (a real slice of indie-founder stacks: Next.js CSR pages, Vite SPAs) get degraded audits. Mitigation: detect-and-disclose (§3 P1), never silently mis-report — and count CSR-shell prevalence in the week-1 telemetry so the JS-rendering revisit trigger has a denominator too.
3. A step-budget ceiling — real, but it's Lighthouse that binds, not the crawl. The ledger: today's 3 steps per 25-page batch supports ~8.5k pages in the 1,024-step budget; after the P0 #0 refactor (1–2 steps/batch), the crawl alone supports ~12–25k pages — the 10k clamp can stand. The binding constraint is Lighthouse strategy all (2 steps per 10-URL batch ≈ 1,000 steps at 5k pages, over budget by itself): cap all at ~2k pages or switch to sampling, and document page caps coupled to Lighthouse strategy. Nor is the ceiling permanent: Workflow instances can chain (the P0 #0 refactor already puts frontier state in D1; a continuation instance resumes it) — roughly a week of cross-instance dedupe work, not an engine swap. The agency/large-site segment is deferred, not foreclosed.
4. Weaker blocking mitigation — and the allowlist fix is unverified on the exact tiers our ICP uses. Workers egress comes from shared Cloudflare IP ranges — we cannot publish a dedicated allowlistable IP, so the GSC-verified fix flow must be User-Agent-based (OpenSEOBot/1.0 via a WAF custom rule; spoofable, tolerable when the owner allowlists their own site). But: on Cloudflare Free, Bot Fight Mode supports no exceptions — a skip-by-UA rule does not bypass it; the owner's only fix there is turning BFM off entirely, a much heavier ask than "add this one rule." Vercel's challenge bypass is plan-gated and may be unavailable on Hobby. Week 1 therefore includes a 1–2 day validation: crawl test zones on Cloudflare Free (BFM on), Cloudflare Pro (Super BFM), and Vercel Hobby/Pro (Attack Challenge Mode), and write per-platform fix copy that is honest where the real answer is "disable bot protection." Verified Bots filing (P1) stays a lottery ticket.
5. The ~1–2 weeks of crawler work inside P0 that the vendor path would have eliminated, plus permanent ownership. Manual-redirect handling, per-host politeness inside steps, sitemap-last seeding, SSRF fix, step-budget restructuring, timeout/retry edges — already counted inside the 6–8-week P0 estimate (not additional to it), but under B most of it would have been deleted, and the hardening treadmill (new WAF behaviors, new edge cases) is ours indefinitely.
6. No fallback engine for blocked sites. Under B, a site blocking us might still be crawlable via the vendor's allowlisted infrastructure. In-house, a blocked site stays blocked until the owner acts. The honest-flagging posture makes this acceptable for owned sites: "we were blocked, here's the fix" instead of pretending.
7. Third-party-URL audits have no remedy at all. The MCP wedge includes "audit any URL on demand" (competitor audits, agent workflows) — but the allowlist flow only works for sites the user owns. For non-owned URLs, "we were blocked" is a terminal state, exactly where vendor infrastructure would have helped. Disposition: scope the claim — on-demand MCP audits are first-class for owned/verified sites, best-effort for everything else, and the tool response says which one it was.

Revisit triggers (set now, so the decision is cheap to reverse). Define "audit blocked" first: start URL blocked, or ≥20% of fetched URLs classified as challenge/cf-mitigated. Then: (a) >15–20% of first-run audits blocked and the fix funnel (block shown → instructions viewed → re-run unblocked, instrumented in week 1) shows owners not converting; (b) sustained demand for >10k-page crawls that chaining doesn't satisfy, or for JS rendering (CSR-shell prevalence from week-1 telemetry is the denominator); (c) DataForSEO/competitor pricing shifts. The reporter contract and schema are engine-agnostic by design, so adopting OnPage later — or the npx local agent (D) — is an adapter, not a rewrite.

Implementation notes for A (from §2's constraints): politeness delays serialized inside steps (wall-clock waits are fine; more steps are not), progress steps collapsed to stay under the 1,024-step budget, incremental per-batch D1 writes (P0 #0), block-rate baseline pulled from existing audit data (status 0/403 rates + PostHog) in week 1 so the revisit triggers have a denominator.

One honest limitation, stated plainly: for sites that block all datacenter crawlers, every recommended option loses scheduled audits — D is a manual local run. The cloud-scheduling advantage evaporates for exactly that segment; our remedy there is the GSC-verified allowlist-fix flow, which restores cloud crawling rather than working around it.

Why not C (the desktop question, directly). Desktop genuinely solves blocking — it's Screaming Frog's model. But scheduled audits die, server-side MCP can't wake a sleeping laptop, and it's a second product (signing, notarization, auto-update, months of work) whose own failure modes — RAM crashes, machine tie-up — are that model's loudest complaints. Draft 1's "~80% of the benefit for ~5% of the cost" for D was invented precision; the honest version: D recovers desktop's crawl-origin benefit (residential IP, optional local Playwright) but not unattended operation, requires a new authenticated bulk-ingest endpoint feeding audit_pages/audit_links (no ingest path exists today; D depends on the P0 persistence refactor) plus a local crawler — the cheerio extraction logic is plain TS and ports, the Workflow orchestration doesn't. Estimate ~3–5 weeks. Panel pushback: the panel suggested that if D balloons to 6–8 weeks the C-vs-D calculus changes. It doesn't: C strictly contains D's costs (local crawler, Playwright, sync) plus installers/signing/3-platform QA, and still loses scheduling and MCP. Worst-case D remains cheaper than best-case C. Scope honesty: D serves the developer half of the ICP; for SEO freelancers (not terminal-native, and whose clients' sites are exactly the WAF'd ones) the realistic remedy is the GSC-verified allowlist flow — which is why that flow ranks above the CLI.

The GSC idea, honestly. GSC verification does not bypass any WAF — Cloudflare neither knows nor cares about Google's ownership records. What it buys: (1) policy cover to raise crawl rate for verified owners (industry-standard move; the Ahrefs precedent is directional, see §1) — but robots.txt respect stays the default with a per-project owner toggle, not auto-ignored: crawl traps and server load are real, and "verified" doesn't repeal politeness; (2) the trustworthy block-fix loop — on a detected block for a GSC-verified site, show the owner the exact Cloudflare/Vercel allowlist rule for our UA (OpenSEOBot — UA-based since Workers egress IPs aren't publishable, see decision above). Highest-leverage mitigation available; OAuth already built; now surfaced in onboarding, not post-P0; (3) later, GSC-enriched audits (true orphans = GSC impressions, zero inlinks).

---

5. Recommended roadmap

Week 1: audit_pages/audit_links/audit_issues schema + engine-agnostic reporter input contract; ship block detection + honest "we were blocked" messaging + telemetry — audit-level blocked definition, fix-funnel events (block shown → instructions viewed → re-run unblocked), CSR-shell prevalence counting, baseline from existing audit data (status 0/403 + PostHog; note raw 403s overcount — classify challenges, don't just count status codes); set the documented UA (OpenSEOBot/1.0 with a /bot info page); 1–2 day allowlist validation on Cloudflare Free (BFM), Cloudflare Pro (SBFM), Vercel Hobby/Pro (Attack Challenge) → per-platform fix copy.

Weeks 2–4: incremental persistence refactor + SSRF fix; crawler hardening (manual redirects, per-host politeness inside steps, sitemap-last seeding, step collapse; page caps coupled to Lighthouse strategy — all capped ~2k or sampled, crawl clamp stays 10k); issue engine + per-page reporters on the normalized record; multipage SQL checks (duplicates, broken links, chains); orphans (sitemap-last + completeness gating).

Weeks 4–6: Issues UX (severity, copy, drill-down, exclusions, CSV export); MCP audit tools with how_to_fix payloads; GSC-verified allowlist-fix flow in onboarding (UA-based WAF rule instructions).

Weeks 6–8: client-side-rendering detection + disclosure; pricing posture shipped (free 50-page audits, credits beyond) with the free-tier TTL/retention policy from §4 cost #1; buffer for the hardening tail that always exists when you own the crawler.

Weeks 8–14 (P1): scheduled audits + deltas with the 2-run link-graph retention policy; AI-crawler accessibility report as the relaunch marketing asset; thin-content/depth checks if not landed; security + external-link checks; opportunistic Verified Bots filing for OpenSEOBot.

Later (P2): npx local agent (new ingest endpoint; gate on block telemetry vs the §4 triggers); Cloudflare Browser Rendering JS audits for GSC-verified sites if demand shows; R2 crawl snapshots; hreflang/structured-data/page-size.

The one-sentence version: keep audits in the cloud on our own crawler (founder call — no vendor spend), give the small ones away to drive signups, spend engineering on a portable issue engine plus agent-actionable MCP output, answer bot-blocking with day-one honest "we were blocked" flagging and owner-targeted allowlist fixes validated per-platform in week 1, accept no JS rendering for now and Lighthouse-coupled page caps — with pre-set, measurable telemetry triggers that tell us if the no-vendor bet needs revisiting, a local CLI later, and a desktop app never.