CrawlerX for agencies: build plan

self-host-telemetry.ts holds a hard-coded PostHog key for the upstream project and skips sending only in hosted mode. Our team-mode instance appears to send anonymous heartbeats to them unless opted out.

Do

Set OPENSEO_TELEMETRY_DISABLED=1 in the production .env and restart. Then remove the module and its wiring, or replace the key with our own PostHog project.

Where

src/server/lib/self-host-telemetry.ts

Done when

Network capture on a running instance shows no calls to us.i.posthog.com with the upstream key.

Customer support requests currently point at the upstream author's email. Links send users to the upstream product, Discord and docs. Create one src/shared/brand.ts (product name, support email, docs and terms URLs, crawler contact URL) and make every reference read from it.

Where

client/features/audit/shared.tsx routes/_app/ai.tsx routes/_app/support.tsx routes/_authenticated.subscribe.tsx routes/_auth.sign-up.tsx client/features/onboarding/OnboardingChatParts.tsx client/features/settings/ApiKeySettings.tsx client/components/AuthConfigErrorCard.tsx shared/ga4.ts shared/gsc.ts server/lib/scrape.ts server/mcp/server.ts server/features/sam/SamChatAgent.ts server/features/rank-tracking/services/scheduledRankChecks.ts server/features/onboarding/OnboardingChatAgent.ts server/referrals/dub.ts

Also

Skills install commands in ai.tsx point at every-app/open-seo. Publish our own skills package or hide that card. Remove the Dub referral code (upstream's partner program).

Done when

grep -ri "openseo.so\|every-app\|ben@" over src returns only tests, comments, and the licence notice. Add that grep to CI.

Production today appears to run vite preview under pm2 (the start script), which emulates the Cloudflare runtime locally. That is fine for an internal tool, not for paying customers. package.json already has deploy:postgres (Alchemy, hosted-prod stage, Postgres). Prove it works end to end on our own Cloudflare account, in a staging stage.

Check

Sign-up, org creation, invite, project create, keyword research, site audit workflow, rank check cron, GSC connect, MCP OAuth. Note every Cloudflare binding used: KV (invitation-send-limit.ts, oauth-provider.ts, progress-kv.ts, serp-locations.ts), R2 (r2.ts, r2-cache.ts), Workflows and Durable Objects (SiteAuditWorkflow.ts, chat agents).

Deliverable

One-page decision: deploy to Cloudflare (assumed) or port off it, with the list of breakages and their cost.

Done when

A staging environment runs hosted mode with our own domain. A second signed-up user gets an isolated workspace.

Search Console and GA4 scopes are sensitive. Until Google verifies the app, external users see an unverified-app warning and are capped. Verification takes weeks and needs a live privacy policy, a homepage on our domain and a demo video. Separately, get written confirmation that DataForSEO allows us to resell their data in a paid product.

Do

Create a dedicated Google Cloud project for production. Write the privacy policy and terms (replace the upstream links in _auth.sign-up.tsx). Submit the consent screen. Email DataForSEO sales about reseller terms and volume pricing.

Done when

Verification submitted with no open questions. DataForSEO reply saved in the repo's docs/ folder.

Every server function must scope by the caller's organisation (see specs/0001). Run /security-review, then write the test that would catch a leak: two orgs, and each server function called with the other org's projectId must fail.

Where

src/serverFunctions/*.ts src/server/mcp/project-auth.ts src/middleware/ensure-user/

Done when

A single parameterised test covers every project-scoped server function and MCP tool. Findings from the review are fixed or ticketed.

Do

Delete the empty root file open-seo@0.1.7 and vite.config.ts.backup. Two files show as modified from line endings only: add a .gitattributes. Tag the fork point. Add an upstream remote and record the sync policy in docs/MAINTAINERS.md. Remove the internal hostname and server paths from docs/SELF_HOSTING_TEAM_MODE.md. Keep LICENSE and add a NOTICE crediting OpenSEO.

Done when

pnpm ci:check passes on a clean checkout.

New table for per-organisation branding: display name, logo, primary colour, footer text. Logo stored in R2. Applied to reports and to the client view, not to the app shell.

Where

New src/db/branding.schema.ts and the Postgres twin under src/db/pg/. Follow server function → service → repository. Run npm run db:generate and keep schema-parity.test.ts green (SQLite and Postgres must both work).

Done when

An owner uploads a logo in Settings and it appears on the report. Only owner and admin can edit it.

ProjectReportService covers GSC, GA4 and the latest audit. Add rank tracking movement, a backlinks summary and AI visibility. Each section already degrades on its own when an integration is missing; keep that. Export: start with a print-optimised route (@media print, A4) since it is the cheapest path. Move to server-side rendering (Playwright is already a dev dependency) only if print output is not good enough.

Where

src/server/features/reports/services/ProjectReportService.ts src/client/features/reports/ReportView.tsx src/routes/_project/p/$projectId/report.tsx

Done when

A 4–6 page A4 PDF with the agency's logo and colours, no CrawlerX or upstream branding, and no API credits spent on render (reuse stored data).

Store a snapshot of the report JSON at share time so a client sees stable numbers and cannot trigger spend. Public route addressed by a random token (store only its hash), with expiry and revoke. Log create and revoke in the activity log.

Where

New report_share table (projectId, tokenHash, snapshot, expiresAt, revokedAt). New public route outside _authenticated. Add actions to ACTIVITY_ACTIONS.

Done when

A revoked or expired link returns 404. The page exposes nothing beyond the snapshot.

One screen listing every client project with the few numbers an account manager scans: rank movement over 7 and 28 days, latest audit health, GSC clicks change, integration status (connected, expired, missing). Sort by "needs attention".

Where

src/server/features/dashboard/services/DashboardService.ts and src/routes/_app/projects.tsx

Done when

Loads in under 2 seconds with 50 projects on seeded data (pnpm seed:projects).

Invitations, password reset and verification currently go through Loops templates that belong to the upstream account. Put a small provider interface in src/server/email and implement it with Resend or SES. Replace billing/loops-sync.ts contact sync (remove, or point at our own tool).

Done when

Invite, reset and verify emails arrive from our domain (SPF, DKIM set). No Loops keys required in hosted mode.

Many client sites sit behind Cloudflare Managed Challenge or DataDome, which the crawler cannot pass. Give agencies something to allowlist: a stable egress IP for the crawler and a user-agent that names us and links to a page explaining it. Publish a short help page with per-WAF instructions. Keep the DataForSEO OnPage fallback as the paid option.

Where

src/server/lib/audit/crawl-request.ts (AUDIT_USER_AGENT)

Done when

The audit failure card on a blocked site links to the allowlist guide and to the paid fallback.

Until self-serve exists, provision each pilot by script: create org and owner, apply branding, set an invoice reference. Set up production basics: nightly Postgres backup with a tested restore, error tracking, uptime check and a status page, and a shared support inbox.

Where

New script beside scripts/seed-projects.ts; runbooks in runbooks/.

Done when

A new pilot agency is live in under 30 minutes by following the runbook, and a restore has been rehearsed once.

The member role exists but is not invitable: spec 0011 blocks it in beforeCreateInvitation, and every invitee is an admin. Add a client role that is read-only and limited to named projects. Model access as a project_member table (userId, projectId), not a JSON list. Enforce it in the project-auth middleware and the MCP layer.

Where

src/lib/org-permissions.ts src/server/mcp/project-auth.ts src/serverFunctions/organization.ts

Done when

A client user sees only assigned projects, cannot run credit-spending actions, and cannot see billing. Covered by the P0.5 isolation test.

Per-project schedule (weekly or monthly), recipients, and whether to attach the PDF or send the share link. A cron creates a snapshot (P1.3) and sends it. Cron hooks already exist in the scheduled handler in src/server.ts and wrangler.jsonc.

Done when

A failed send is retried and visible in the activity log. Unsubscribe link works.

Notify on rank drops above a threshold, audit score regressions, GSC clicks falling, and expired Google connections. Hook into scheduledRankChecks.ts after each run. Preferences table per user and project. Email first, in-app feed second.

Done when

No duplicate alert for the same condition within 24 hours.

Credits are pooled per organisation. Agencies need to know what each client costs, and to stop one client burning the pool. Record projectId on every usage event, show usage by client on the billing page, and add an optional monthly cap per project checked in the same preflight as the balance check.

Where

src/server/lib/dataforseoClient.ts (the single entry point per spec 0002), src/server/billing/subscription.ts

Done when

A capped project fails with a clear "client cap reached" error. Usage-by-client totals match the org total.

Create our own Autumn and Stripe accounts and define the agency plans (platform fee, client seats, credit packs). The 1.28 markup is a constant in src/shared/billing.ts: make it configuration, not a code edit, so pricing can change without a deploy. Stripe checkout already collects business name and tax ID.

Where

src/shared/billing.ts src/server/billing/ src/routes/_authenticated.subscribe.tsx

Done when

Test-mode subscribe, top-up, cancel and invoice all work. Cost estimates in the UI match what is billed.

Signup flow adapted for agencies (agency name, first client project, connect Google), a free-trial policy with a spend ceiling so trials cannot burn DataForSEO money, terms and privacy pages, a pricing page. Rework the onboarding chat copy from "personal SEO tool" to "agency workspace".

Done when

A stranger can sign up, add a client, run a first audit and receive a report without talking to us. Trial spend per organisation is capped and monitored.