55 Commits

Author SHA1 Message Date
Ben Senescu
1a74904b67
Cut worker isolate baseline memory: stub just-bash, lazy-load cheerio (#340)
Production OOM triage: every 'Worker exceeded memory limit' burst hits
unrelated cheap routes right after a deploy — the 128MB limit is per
isolate, and the main worker's eagerly-evaluated module graph is what
crowds it, not any single request.

- Alias just-bash to a throwing stub (worker never uses it): removes
  just-bash + turndown + @mixmark-io/domino from the bundle. The chain
  was pulled in eagerly by @cloudflare/think via the SamChatAgent
  re-export in src/server.ts; SAM only uses its own MCP tools.
- Disable Think's workspace bash tool on SamChatAgent so the stub is
  unreachable at runtime.
- Dynamic-import page-analyzer (cheerio) in the site-audit crawl step
  so it evaluates only when an audit runs, not in every isolate.
- Drop the turndown CJS alias workaround (#339): turndown is no longer
  in the graph at all.

Main eager server chunk: 14,434 kB -> 11,712 kB (-19%); cheerio's
503 kB now a lazy chunk.
2026-07-03 22:23:35 -04:00
Ben Senescu
41d37792e7
feat: SAM — in-app SEO agent with full MCP toolset (#322) 2026-07-03 18:48:18 -04:00
Ben Senescu
a49ec51e95 fix(ai-search): use valid Claude model_name and fail fast on unknown ones (#323)
DataForSEO dropped the Claude Sonnet 4.0 family from its llm_responses
catalog, so model_name=claude-sonnet-4-0 was rejected with 'Invalid
Field: model_name' while still billing the failed task. Point Claude at
claude-sonnet-4-5 and validate every model_name against DataForSEO's
accepted catalog before dispatching the paid call.
2026-07-02 18:15:46 -04:00
Ben Senescu
4e48f8344f Fix production errors: onboarding crash hardening + DataForSEO spend/noise cleanup (#282) 2026-07-02 18:15:46 -04:00
Ben Senescu
868924e1fe refactor(keywords): unify keyword-metric fetching behind one helper (#320) 2026-07-02 18:15:46 -04:00
mattmacrocket
c8a1e17359
fix: surface DataForSEO auth failures as an actionable error instead of "unexpected error" (#53) 2026-07-02 17:38:13 -04:00
Ben Senescu
90a04d1845
Onboarding chat: MiniMax M3 + market-research tools + UX polish (#296) 2026-06-25 12:31:07 -04:00
Ben Senescu
0652f3a8a6
improve onboarding chat ux + migrated to DO
* Add onboarding agent v1 product spec

* Add onboarding agent implementation plan (Project Think)

* Update onboarding plan: chat + seed function (drop Think/Workflows)

* feat(onboarding): data + metering foundation, Project Context store, MCP tool

* feat(onboarding): site read + DataForSEO signal + OpenRouter strategy seed

* feat(onboarding): strategy + streaming chat UI with update_project_context tool

* fix(onboarding): address review — bound free runs, cap chat, share auth+error helpers, harden scrape

* fix(onboarding): use canonical keyword-locations list, not a separate country list

* Improve onboarding strategy chat

* feat(onboarding): refine upgrade rail UI + fact-checked copy

- Rebuild upgrade sidebar: drop the nested card so the rail itself is the
  container (header / plan / features / CTA / progress footer with dividers)
- Remove the 'Free preview' badge + headline pitch; header now reads
  'Previewing OpenSEO' with the site domain beneath
- Tighten copy against the fact sheet: fix monthly-vs-top-up credit wording,
  drop 'live' rank tracking, add money-back + open-source trust signals,
  unify CTAs to 'Upgrade to continue', cut cross-panel feature redundancy
- Replace off-strategy suggested question; add progress bar counter
- FORCE_FREE_PREVIEW flag to always show the preview/limit UI while testing

* feat(onboarding): add 'What do you recommend' strategy chip; revert suggested questions

- Add a highlighted suggestion chip that prompts Sam for the strategy, shown
  only when the user hasn't already used the welcome 'Show my strategy' CTA
- Track strategyRequested so the chip isn't re-offered after use
- Restore the original four suggested questions

* feat(onboarding): add OpenSEO Discord CTA + fact-sheet entry

- Discord link in the upgrade sidebar
- Fact-sheet community entry + system-prompt guidance so Sam can point
  users to the Discord for community/second-opinion help

* chore(merge-ready): round 1 fixes

- scrape.ts SSRF: validate the initial domain via audit/url-policy
  (normalizeAndValidateStartUrl) and re-validate each redirect hop with
  redirect:"manual" (one hop, blocked/private/metadata hosts + DoH rebinding).
  Replace the content-length-only guard with a bounded streaming read so
  chunked/CDN responses can't buffer past MAX_RESPONSE_BYTES. Remove the
  unguarded normalizeDomainToUrl helper. Add scrape.test.ts.
- http-errors.ts: map PAYMENT_REQUIRED AppError to HTTP 402 (was 500), so the
  onboarding chat paywall backstop surfaces correctly.
- OnboardingStrategyChat: replace the hardcoded FORCE_FREE_PREVIEW=true debug
  flag (which forced paid users into the free-preview/paywall UI) with a
  safe-by-default ?preview=1 URL override.
- onboardingStrategy.ts: delete the dead generateOnboardingStrategy export
  (knip) and its now-unused imports; the chat tool path uses runOnboardingSeed.
- chat.ts: rename inner runOnboardingSeed result to fix no-shadow.
- Extract presentational chat sub-components into OnboardingStrategyChatParts
  to satisfy max-lines; reformat Markdown.tsx for prettier.

* chore(merge-ready): round 2 fixes

- chat.ts: validate message role in schema + count total messages (not just user-role) so the free-question gate can't be bypassed with mislabelled roles
- OnboardingStrategyChat.tsx: surface useChat error state with a paywall-aware notice; branch 'Ask about OpenSEO' message text on isPaid
- OnboardingStrategyChatParts.tsx: guard free-preview welcome copy behind !isPaid (paid variant for subscribers)
- onboardingStrategy.ts: reset onboardingRunStatus/onboardingRunAt when the domain changes so a corrected domain can get a fresh free seed

* chore(merge-ready): round 3 fixes

- onboarding chat: count only user-role messages for free-question paywall to match client gate (was counting all messages, firing ~3 turns early)
- ProjectContextStore: drop unused return value/type from saveProjectContextVersion, inline latest-version query into getCurrentProjectContextMarkdown, remove dead toVersion helper and ProjectContextVersion type
- onboarding chat UI: replace 'Why is OpenSEO better than Claude?' suggested chip with 'How does OpenSEO work with Claude?' (Claude is an MCP client, not a competitor)

* feat(onboarding): route post-upgrade to GSC step; drop isPaid from preview chat

- Checkout successUrl now returns to /onboarding?step=3 (GSC connect) instead
  of the strategy chat, with a 'You're in!' success banner introducing the
  remaining GSC + MCP setup steps. Fixes the post-Stripe 'stuck on paywall'
  race since the user leaves the chat entirely.
- The strategy chat is now purely the pre-upgrade free preview: removed the
  managed-access query, the isPaid branching, and the ?preview override. The
  7-question cap always applies (kept as a conversion funnel).

* feat(onboarding): show post-upgrade success as its own step screen

Instead of a banner stacked above the GSC step, render a 'You're in!' screen
using the standard step layout (logo, title, card, Continue) in place of the
GSC step when ?checkout=success is present. Continue drops the param to reveal
the actual GSC step.

* refactor(nav): remove Project settings from account dropdown

Project settings is now reachable only via the project switcher's 'Manage
projects' → /projects → per-project settings. Drops the dead
projectSettingsLinkOptions helper and the now-unused AccountMenu projectId prop.

* refactor(onboarding): remove project-context persistence + MCP tool

Defers the Project Context store to a later PR to simplify this one.

- Delete ProjectContextStore, the get_project_context MCP tool (+ registration),
  the project_context_versions table (schema + migration 0024 + snapshot), and
  the update_project_context chat tool.
- generate_initial_strategy now returns the synthesized strategy to the chat
  without persisting it; claimRun still gates paid spend (one free run).
- Chat system prompt no longer injects saved context; it just grounds Sam with
  the project's domain.
- getOnboardingStrategyState returns only { projectId, domain }.
- Move the agent fact sheet out of docs/ (human docs) to
  src/server/features/onboarding/openseo-fact-sheet.md.

* refactor(routing): move /strategy to /onboarding/chat

Rename the onboarding strategy chat route from /strategy to /onboarding/chat.
_authenticated.onboarding.tsx becomes the index route; the chat is a sibling,
so TanStack auto-creates the shared /onboarding parent (Outlet).

* chore(onboarding): clean up leftovers from the persistence removal

- Drop the chat's onFinish=invalidateStrategyState refetch: now that the
  strategy state is just { projectId, domain } (no persisted markdown), the
  chat can't mutate it, so the post-turn refetch was dead work.
- Fix a stale 'Project Context' doc comment in synthesis.ts.
- Note in specs 0005/0006 that strategy persistence + the MCP read tool were
  deferred, so the docs don't contradict the shipped code.

* feat(onboarding): meter LLM (OpenRouter) spend via Autumn track_tokens

Mirror the DataForSEO metering pattern for LLM cost: a best-effort
trackLlmUsage helper emits a PostHog usage event and records token usage on
Autumn's token-tracking endpoint (REST; not in the autumn-js SDK yet), priced
from the model slug. Wired into the chat stream (onFinish) and the strategy
synthesis call. getOnboardingModel now also returns the resolved model slug.

* feat(onboarding): step-styled site form + account menu on chat page

- Restyle the website/country form to match the onboarding step layout (logo,
  title, helper) and explain why we ask (read the site + pick the search market).
- Extract OnboardingAccountMenu to a shared component and render it on the
  onboarding chat page so signed-in users can reach account actions there too.

* fix(auth): keep verify-email on 'check your inbox' after email sign-up

The post-sign-up redirect always passes ?email=; key the waiting state off it
so a just-signed-up user sees check-your-inbox + resend instead of the sign-in
CTA the verification gate would immediately block, even while the session is
still resolving.

* fix(onboarding): meter total LLM usage across all stream steps

Review caught that streamText onFinish 'usage' is only the last step; with
stopWhen=4 + the strategy tool, multi-step runs under-metered. Use 'totalUsage'
and await the metering so it fires before the stream closes. Also drop the
in-flux cache/reasoning token fields (negligible here).

* feat(onboarding): adopt the 'chat with tools' architecture from agent-onboarding-2

Replace the deterministic seed + synthesis pipeline (and the
onboarding_run_status/run_at columns) with two on-demand tools Sam calls —
read_website and get_seo_metrics — and have Sam write the strategy itself
in-stream, so a mid-stream refresh re-runs cache-backed tools instead of
dead-ending on a 'complete' status. Rename OnboardingStrategy* -> OnboardingChat*.

Preserved from this branch: LLM metering (now via the chat onFinish totalUsage,
covering the in-stream strategy), the account menu on the chat page, the
verify-email fix, and the step-styled site form. Drop columns via migration 0025.

* docs(onboarding): correct spend-bound + stale synthesis comments

Clarify that get_seo_metrics spend is bounded by the question cap + one project
per un-upgraded account (not solely caching, which doesn't cover no-data sites),
and drop 'synthesis' from comments now that Sam writes the strategy in-stream.

* refactor(onboarding): metered LLM via Autumn AI-SDK adapter; drop skipBalanceAssert

Now that every org gets an onboarding_plan with usage credits, onboarding spend
draws down the normal balance — no bypass needed.

- LLM metering: use Autumn's official @useautumn/gateway adapter (withLlmMetering
  wraps the model; correct token-pool pricing for cached/reasoning tokens),
  replacing the hand-rolled onFinish/track_tokens REST plumbing. Point it at the
  existing 'llm_usage' feature (backed by usage_credits + topup_credits) rather
  than a to-be-created 'ai_credits' feature.
- DataForSEO: remove skipBalanceAssert end-to-end (chat metering object, the
  meter() plumbing in dataforseo/client.ts, and the DomainService override type);
  onboarding now asserts balance like every other caller. Kept the email-verified
  + Labs-location gate on get_seo_metrics as the anti-farming bound.

Co-authored with a parallel agent's LLM-metering refactor.

* docs(onboarding): fix stale metering comment + diverged-architecture specs

- DomainService MeteringOverrides comment no longer claims a balance-gate bypass
  (skipBalanceAssert + the onboarding seed are gone).
- specs 0005/0006: correct the update notes — the seed/synthesis pipeline,
  claimRun, and skipBalanceAssert were replaced by the chat-with-tools design;
  flag the bodies as the superseded plan.
- Document the pinned Autumn track_tokens API version.

* feat(onboarding): gate the chat turn on credit balance (LLM included)

Now that every org gets onboarding_plan trial credits and LLM tokens draw from
the same usage/topup balance, assert that balance before streaming — not just
track it. Extract the DataForSEO balance check into subscription.ts
(getUsageCreditsRemaining / assertUsageCreditsAvailable) and reuse it; the chat
throws a friendly PAYMENT_REQUIRED when credits are gone (client shows the
upgrade copy).

* feat(onboarding): make the strategy chat hosted-only

The chat needs the managed LLM + trial credits, so self-hosted has no business
there. Gate the step-2 navigation on hosted mode and add a beforeLoad redirect
on /onboarding/chat so self-hosted lands back in the wizard.

* feat(onboarding): site-form + welcome copy; drop open-source badge

- Site form: 'Tell us about your website.' title, short input labels, no extra
  helper descriptions.
- Welcome message: lead with the upgrade ask + a Discord/email escape hatch.
- Remove the 'Open source — self-host for free anytime' badge from the rail.

* fix(onboarding): show typing indicator during the submitted wait

showTyping gated on the last message lacking assistant text, but right after
send the last message is the user's own (which has text), so nothing showed
until the assistant message appeared. Show it whenever busy and the last
message isn't assistant-text-yet.

* refactor(onboarding): drop redundant email-verified gate on get_seo_metrics

The route guard already requires a verified email to reach the chat in hosted
mode, and the trial-credit balance bounds spend — so the in-tool emailVerified
check was redundant for real users and blocked local/bypass testing. Keep the
Labs-location check (functional).

* feat(billing): meter onboarding LLM spend into the shared credit pool

Both DataForSEO and onboarding-LLM now draw from the same usage_credits/
topup_credits pool via one helper, instead of LLM needing a separate Autumn
ai_credit_system.

- Extract trackUsageCreditSpend (markup -> credits -> monthly/topup split ->
  autumn.track + usage:credits_consume) into subscription.ts; DataForSEO's
  trackDataforseoCost now delegates to it (behavior unchanged, tests pass).
- Enable OpenRouter usage accounting; the chat onFinish sums the real per-step
  cost OpenRouter reports and deducts it through the same helper.
- Drop the @useautumn/gateway adapter, llm-metering.ts, track_tokens, and the
  AUTUMN_LLM_USAGE_FEATURE_ID constant — no ai_credit_system feature needed.

* feat(onboarding): persist the strategy chat in a Durable Object (AIChatAgent)

Move the onboarding chat from a stateless streamText route to an Agents SDK
AIChatAgent Durable Object, so the conversation persists (DO SQLite) and
survives reloads — one instance per project.

- OnboardingChatAgent.onChatMessage ports the system prompt, read_website +
  get_seo_metrics tools, the credit-balance/free-question gate, and the
  OpenRouter cost metering. Billing gates surface as a normal assistant message
  (staticAssistantResponse) rather than an HTTP 402.
- The Worker authorizes every /agents/* connection (resolve session + verify the
  caller's org owns the projectId) before it reaches the DO; the DO derives org
  /domain from the project it is named after. Auth stays on the proven path.
- Client swaps useChat -> useAgent + useAgentChat (WebSocket), keyed by projectId.
- Adds the DO binding + new_sqlite_classes migration; pins @cloudflare/ai-chat
  0.6.1 to match agents 0.12.3.

* chore(onboarding): bump agents+ai-chat to latest; fix review findings

- Bump agents 0.12.3 -> 0.15.0 and @cloudflare/ai-chat -> 0.8.4 (the supported
  pairing; verified MCP, the DO, and the build still compile).
- Thread the per-turn abortSignal into streamText so a user aborting mid-stream
  cancels the billable LLM call (was leaking sub-cent cost on abort).
- Ensure the org's Autumn customer exists in the Worker authorize step before
  the DO checks the credit balance, avoiding a false 'out of credits' gate on a
  brand-new org's first message.

* chore: remove stray reservation-booker-seo-report.html

* chore: drop stale @useautumn/gateway minimumReleaseAge exclusion

The package was removed when LLM metering moved to the shared credit pool.

* perf(onboarding): fetch get_seo_metrics signals in parallel; clarify question-cap

- get_seo_metrics now fetches the domain overview and ranked keywords
  concurrently instead of in series (faster tool turn). Trade-off: it always
  issues the metered ranked-keywords call now, including for no-ranking sites.
- Correct the FREE_ONBOARDING_QUESTION_LIMIT comment: the server re-check counts
  client-supplied history, so the cap is a conversion nudge, not a security
  boundary — the credit balance is the real spend bound.
2026-06-19 18:09:52 -04:00
Ben Senescu
e57768a8ce
Add onboarding chat helper 2026-06-19 13:20:52 -04:00
Ben Senescu
f8d5ffd285
Scheduled rank checks run through DataForSEO's task queue (#268) 2026-06-13 11:20:42 -04:00
Ben Senescu
b809f87b5e
Hybrid keyword data: Google Ads coverage for non-Labs countries + opt-in clickstream volumes (#262) 2026-06-12 14:08:11 -04:00
Ben Senescu
16b92ccea3
Backlinks: server-side pagination, sorting, filters, and a one-per-domain view (#258) 2026-06-10 23:27:04 -04:00
Ben Senescu
8a85b7b794
AI Visibility: competitor Share of Voice + clarity fixes for AI citations (#248) 2026-06-09 22:33:11 -04:00
Ben Senescu
1495660631
fix errors and onboarding problems (#242) 2026-06-05 15:22:32 -04:00
Ben Senescu
7a26be46cc
Self-hosted Google Search Console (#236) 2026-06-04 18:06:16 -04:00
Ben Senescu
23c38ca4e5
GSC MCP part 1 (#233) 2026-06-04 13:50:26 -04:00
Ben Senescu
b64ef56b4d
refactor: unify dataforseo sdk usage (#231) 2026-06-01 09:48:11 -04:00
Ben Senescu
06e15dc70f
Add per-feature usage breakdown to billing page (#232) 2026-05-31 18:42:41 -04:00
Ben Senescu
03fd3588ef
refactor: clean up complex code (#229) 2026-05-28 16:37:33 -04:00
Ben Senescu
506dde84d1
Add additional MCP tools for Local SEO (#222) 2026-05-26 18:38:14 -04:00
Ben Senescu
3456af12fe
bump to reddit v3 api (#209) 2026-05-21 00:08:50 -04:00
Ben Senescu
4f638db702
Add Reddit conversion tracking (#208) 2026-05-20 23:36:30 -04:00
Ben Senescu
ae2031bfad
Harden domain and backlinks validation before external calls (#183)
* Validate backlinks targets before calling DataForSEO

* Validate backlinks targets before provider calls
2026-05-12 19:15:51 -04:00
Ben Senescu
def9390cea
feat: Implement server-side pagination and filtering for domain keywords (#143) 2026-05-04 15:26:36 -04:00
Ben Senescu
17298da7ee
fix: capture DataForSEO errors with context (#138) 2026-04-24 00:16:20 -04:00
Ben Senescu
ebc9e0b4fa
fix: prompt explorer bugs (#136) 2026-04-23 18:25:11 -04:00
Ben Senescu
1108cdd9a7
feat: gate llm mentions in self hosted product (#134) 2026-04-23 18:02:24 -04:00
Ben Senescu
faf7a294a3
feat: AI Visibility — Brand Lookup and Prompt Explorer (#128) 2026-04-23 10:48:38 -04:00
Ben Senescu
9a9a9a2a6c
feat: add keyword metrics (volume, difficulty, CPC) to rank tracking (#118) 2026-04-16 10:41:31 -04:00
Ben Senescu
aeafa5649d
feat: polish rank tracking UI (#115) 2026-04-16 00:42:10 -04:00
Ben Senescu
53a83996ef feat: Rank Tracking MVP (#91) 2026-04-14 20:29:38 -04:00
Ben Senescu
fb1291537c
fix: treat empty DataForSEO backlinks results as valid empty data (#109) 2026-04-09 16:44:34 -04:00
Ben Senescu
1d9a503335
hosted: add free trial plan and onboarding (#104) 2026-04-08 19:06:35 -04:00
Ben Senescu
a10f4d82a0 feat: hide spam backlinks by default (#89)
* feat: hide spam backlinks by default

* feat: add configurable backlinks spam filtering

* refactor: simplify backlinks spam filtering flow

Reduce duplicated spam-filter state and normalization so the backlinks page has one source of truth and client/server caching stays aligned.

* refactor: simplify backlinks spam filter semantics

* fix: keep spam filtering scoped to backlink rows

* feat: show backlinks spam score
2026-04-08 14:09:02 -04:00
Ben Senescu
ad3b732f60 hosted: add product analytics (#83)
* track core product analytics flows

Track auth, search, export, audit, and credit-consumption events with canonical route IDs so PostHog funnels and usage dashboards stay low-noise and privacy-safe.

* fix: keep auth actions usable after session loss

* refactor: simplify analytics and auth helpers

- Replace isRecord/getActiveOrganizationId type guards with simple cast
- Refactor getAnalyticsRouteContext from if/return chain to route tables
- Replace toVerificationIssueType switch with zod enum
- Merge duplicate credits_consume events into single event per API call
- Merge two PostHogBootstrap useEffects into one

* refactor: add projectId to middleware context to reduce boilerplate

The requireProjectContext middleware now includes projectId directly,
eliminating repeated manual construction of BillingCustomerContext
objects across all server function handlers.

* remove unused BILLING_* env var fallbacks from cost profile script

* remove before_send event enrichment to preserve native PostHog URL tracking

The before_send hook was stripping $pathname, $current_url, $referrer and
other URL properties, which breaks PostHog web analytics dashboards, paths
analysis, session replay, and attribution. The route_id/route_group injection
it provided is unnecessary since PostHog already captures $pathname natively.

* remove route mapping layer, pass raw redirect paths to analytics events

The route ID registry (STATIC_ROUTES, PROJECT_ROUTES, getAnalyticsRouteContext,
getRedirectRouteId) duplicated what PostHog already captures via $pathname.
Replace redirect_route_id with redirect_to containing the raw path, and remove
~80 lines of route mapping infrastructure.

* clean up analytics events: drop redundant submit events and derived properties

- Remove search_submit events for keywords, domain overview, and backlinks
  (the search_complete events capture the meaningful outcome data)
- Remove target_type from backlinks events (derived 1:1 from search_scope)
- Remove result_limit from keyword research (requested limit, not useful
  alongside actual result_count)
- Remove export_format from data:export events (always "csv")

* refactor: inline wrappers, colocate helpers, deduplicate getActiveOrganizationId

- Inline toVerificationIssueType into verify-email.tsx (single-use wrapper)
- Move mapDataforseoPathToCreditFeature into dataforseoClient.ts (only consumer)
- Extract shared getActiveOrganizationId into lib/auth-session.ts (was
  duplicated in __root.tsx and middleware/ensure-user/hosted.ts)
- Rename shared/analytics.ts → shared/internal-user.ts (only email helpers
  remain after removing route mapping, verification, and dataforseo helpers)

* remove internal user tracking and email domain properties

Drop is_internal_user super property, email_domain person property, and all
supporting code (shared/internal-user.ts, getEmailDomain, isInternalUserEmail).
Simplifies initPostHog and identifyAnalyticsUser signatures.

* remove backlinks:search_complete effect-based tracking

The reactive useEffect + useRef dedup pattern added ~30 lines of plumbing
inside a data hook for a single analytics event. Not worth the complexity.

* simplify: replace manual type guards with zod, deduplicate posthog and sign-out helpers

- Replace hand-rolled typeof checks in getActiveOrganizationId and
  isAuthenticatedServerFunctionContext with zod safeParse
- Extract withPostHogClient helper to deduplicate client posthog wrapper
- Move apiKey guard into getServerPostHogClient factory
- Extract signOutAndRedirect to avoid duplicated sign-out logic
- Drop derivable has_results from analytics events
- Remove unnecessary path normalization in mapDataforseoPathToCreditFeature

* fix: strip email from pageview URLs, restore sign-out guard, harden server posthog, fix path mapper

- Sanitize $current_url on pageviews to remove email query param (PII)
- Restore onSuccess for sign-out redirect to avoid bounce-back on failure
- Swallow shutdown() errors so PostHog outages can't fail billed work
- Rewrite mapDataforseoPathToCreditFeature to match real API path structure
  (path[1] = module, path[3] = endpoint) instead of scanning all segments

* simplify: remove redundant refs in verify-email, infer middleware context type

- Remove unnecessary useRef guards in verify-email effects (deps already prevent re-firing)
- Use z.ZodType<EnsuredUserContext> annotation to infer return type instead of casting
- Add comment explaining one-shot PostHog client on Workers

* fix: reset PostHog identity on sign-out before redirect

* fix: require POSTHOG_HOST env var instead of defaulting to us.i.posthog.com

* fix: annotate url as unknown to satisfy no-unsafe-assignment

* format
2026-04-08 14:09:02 -04:00
Ben Senescu
f3ef909d3e fix: handle empty DataForSEO task results (#84)
* fix: accept empty DataForSEO task results

Treat successful tasks with null items as empty payloads so empty ranked keyword responses do not fail billing validation.

* refactor: simplify DataForSEO null result handling

Add .nullable() to the existing structured result schema instead of
loosening the type to unknown[] and re-parsing in parseTaskItems.

* refactor: simplify backlinks zod parsing with structured result schema

Same pattern as the dataforseoSchemas fix: give taskSchema.result a
structured type with .items instead of z.unknown(), removing the
intermediate resultItemsSchema and two-step parsing in parseItems.

* refactor: replace manual type guards with zod schemas

- progress-kv.ts: replace isCrawledUrlEntry type guard with a zod
  schema and use jsonCodec(z.array(...)) instead of parsing unknown
  then filtering
- helpers.ts: tighten normalizeIntent param from unknown to
  string | null | undefined to match actual call sites
- dataforseoBacklinksSupport.ts: allow null result elements to match
  API responses where result contains [null]

* refactor: filter null result elements at the source

Filter out null elements from task.result in postBacklinks so
downstream functions receive clean BacklinksTaskResult[] instead
of (BacklinksTaskResult | null)[].

* save
2026-04-08 14:09:02 -04:00
Ben Senescu
c1190550b0 fix: handle empty keyword research results (#81)
Treat successful DataForSEO keyword responses with null items as empty results so auto mode can fall back cleanly. Simplify the no-results state by removing dead-end actions and top-aligning the empty card.
2026-04-08 14:09:02 -04:00
Ben Senescu
d773105e8d fix: use backlinks history for default trends (#69)
* fix: use backlinks history for default trends

* simplify backlinks: remove filters, always use history endpoint

Remove the filter UI (status, subdomains, indirect links, exclude internal)
and hardcode defaults across the stack. Replace the conditional
timeseries_summary + timeseries_new_lost_summary fallback with a single
backlinks/history/live call for trend data. This reduces the overview from
5 parallel API calls to 3 and removes all conditional branching.

* improve charts

* fix: refresh backlinks cost docs
2026-04-08 14:09:02 -04:00
Ben Senescu
5617c6b9f3 redesign: auth pages (#60)
Remove card/shadow layout in favor of a clean centered design.
Add tree logo, simplify inputs (placeholder-only, no labels),
use soft buttons, narrow form width, and clean up footer links.

* fix: remove unused AUTUMN_SEO_DATA_USAGE_FEATURE_ID export

This constant became unused after the billing buckets separation
in #57 which replaced it with separate balance/topup feature IDs.

* fix: resolve oxlint errors from billing buckets PR

Add eslint-disable for max-lines in HostedBillingContent.tsx.
Type trackMock properly in dataforseoClient.test.ts to eliminate
unsafe type assertions without needing eslint-disable.
2026-04-08 14:09:01 -04:00
Ben Senescu
d4aacb13be Separate billing buckets (#57)
* refactor: move DataForSEO response cache from KV to R2 (#46)

* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move DataForSEO response cache from KV to R2

KV TTL-based expiry is imprecise for cache freshness. Switch to R2 with
soft TTL via custom metadata (expiresAt) and a 7-day lifecycle rule for
cleanup. Cache objects live under the `dataforseo-cache/` prefix,
separate from durable audit payloads in `site-audit/`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: use Workers crypto for R2 cache keys

Keep the cache helper aligned with the Cloudflare runtime and avoid pulling Worker bindings into Vitest. Also clarify that R2 lifecycle cleanup is optional but recommended to control storage growth.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move hosted billing to Autumn hooks (#48)

* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move hosted billing to Autumn hooks

* fix: satisfy ci checks for hosted billing

* refactor: simplify hosted Autumn billing integration

Limit Autumn wiring to the billing route and remove billing-specific indirection so the hosted billing flow is easier to reason about. Stop sending organization IDs as fake customer names to keep customer identity data honest.

* fix: satisfy ci line-limit check for hosted billing

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: split SEO credits into monthly included and top-up pools

Replaces the single usage_credits balance with two separate pools:
- usage_credits: monthly grant from base-plan, resets each cycle
- topup_credits: one-off purchases from credit-top-up, never expire

Usage now deducts from monthly credits first, then falls back to
top-ups. The billing UI shows both balances separately with a reset
date on the monthly card. Requires Autumn dashboard changes to create
the topup_credits credit system and update the credit-top-up plan.

* formt

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 14:09:01 -04:00
Ben Senescu
ae7712238e refactor: use TanStack Form everywhere (#49)
* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.


* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------


* refactor: unify app forms with TanStack Form

Replace bespoke field and submit error state so validation, async submit handling, and router-synced inputs behave consistently across the app.

* fix: remove dead code from ci check

* fix: delay required form errors until input interaction

* fix: preserve detailed DataForSEO error messages

* fix: sanitize internal errors and delay domain validation

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 13:38:08 -04:00
Ben Senescu
cdd729ec0d feat: add hosted-only PostHog error tracking (#51)
* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.


* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------


* feat: add PostHog error tracking for client and server

Integrate posthog-js and posthog-node to capture unhandled errors in
both the client error boundary and server function middleware, skipping
expected errors like auth and 404s.

* fix: improve PostHog error tracking setup

- Fix captureExceptionImmediate args (pass undefined as distinctId, not request path)
- Rename VITE_POSTHOG_KEY/POSTHOG_KEY to POSTHOG_PUBLIC_KEY (single env var for both client and server)
- Expose POSTHOG_PUBLIC_KEY and POSTHOG_HOST via Vite envPrefix instead of VITE_ prefix
- Enable capture_exceptions in client posthog-js init for error tracking
- Use waitUntil() instead of await for server error capture (non-blocking)
- Add try/catch around client captureException to prevent unhandled rejections

* fix: align PostHog setup with lint rules

* fix: gate PostHog to hosted mode

* docs: remove hosted PostHog README note

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 00:44:14 -04:00
Ben Senescu
4f33a52e0e refactor: move DataForSEO response cache from KV to R2 (#46)
* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.


* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------


* refactor: move DataForSEO response cache from KV to R2

KV TTL-based expiry is imprecise for cache freshness. Switch to R2 with
soft TTL via custom metadata (expiresAt) and a 7-day lifecycle rule for
cleanup. Cache objects live under the `dataforseo-cache/` prefix,
separate from durable audit payloads in `site-audit/`.


* refactor: use Workers crypto for R2 cache keys

Keep the cache helper aligned with the Cloudflare runtime and avoid pulling Worker bindings into Vitest. Also clarify that R2 lifecycle cleanup is optional but recommended to control storage growth.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:27:30 -04:00
Ben Senescu
638f5a6602 refactor: move lighthouse audits to dataforseo (#43)
* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.

* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client
2026-03-26 20:25:27 -04:00
Ben Senescu
739b3f0b6a
Add autumn metered pricing for hosted mode (#44)
* feat: add hosted billing gate and backlinks metering

* refine hosted billing onboarding flow

* fix: align Autumn credit billing flows

* refactor: simplify Autumn billing flow

* refactor: narrow backlinks billing context

* refactor: require billing identity for backlinks profiles

* refactor: colocate backlinks billing flow

* clean

* refactor: centralize DataForSEO billing client

* refactor: simplify dataforseo billing flow

* chore: fix ci check lint issues

* fix: stabilize backlinks chart sizing

Measure chart container width before rendering the backlinks charts so Recharts does not mount at 0x0 inside responsive layouts.

* docs: document hosted DataForSEO metering

Capture the Autumn credit model and require hosted code to go through the metered client path so provider usage is harder to bypass.

* fix: enforce hosted billing access checks

* fix: preserve hosted billing subscription access

* fix: handle hosted billing UI failures
2026-03-25 13:02:12 -04:00
Ben Senescu
4040a854a7
feat: Add better auth (#24)
* refactor: rename delegated auth user table

* feat: scaffold hosted better auth setup

* feat: add hosted auth flows

* refactor: scope project access to organizations

* fix: harden hosted auth entry points

* fix: stabilize org backfills and auth state

* refactor: simplify hosted organization setup

* fix: restore hosted auth signup flow

* fix: preserve hosted workspace access

* fix: preserve hosted auth redirects

* Improve hosted auth UX: auto-redirect to sign-up, hide header on auth pages, add form placeholders, and trust portless dev origins

- Auto-redirect unauthenticated users to /sign-up in hosted mode
- Hide top nav on /sign-in and /sign-up for a cleaner auth experience
- Add input placeholders across sign-in and sign-up forms
- Make name field optional on sign-up (falls back to email username)
- Update copy: remove 'hosted' from user-facing text, rename link to 'Create account'
- Trust *.open-seo.localhost:1355 in dev mode to fix Better Auth origin rejection with portless worktrees

* Simplify hosted auth flow and remove standalone PSI

Use TanStack Form for sign-in and sign-up, make hosted unauthenticated handling redirect-focused, and inline auth route errors. Remove the leftover standalone PSI route, services, and table so PSI only exists within site audits.

* Align project auth with Better Auth organizations

* Make server function auth middleware global

* Reduce auth server function boilerplate

* delete migrations

* fix regenerated migration data backfills

* Simplify hosted auth flow and project audit scoping

* Use active project context for audit actions

* Allow hosted session project updates

* Let agent dev server inherit auth mode

* Match hosted header to gateway account menu

* Scope project session updates to active project

* Inline authenticated server function setup

* Polish header project and account controls

* restore auth generate script

* Use explicit project access in server functions

Make project-scoped server functions take projectId input and enforce ownership through shared middleware instead of session-backed current project state. Document the tradeoffs in an ADR so future changes can follow the same boundary.

* fix ci dependency detection for auth tooling

* Harden project auth in server middleware

Authorize projectId automatically in authenticated server middleware and add a requireProject guard for project-scoped handlers. This makes the auth boundary harder to bypass and removes ad hoc non-null assertions from server functions.

* Inline project id input schemas

Remove tiny shared projectId schema helpers where they were adding indirection without reducing real complexity. Keep project-scoped validation explicit at each server function boundary.

* Skip hosted backlinks access checks

* Simplify auth mode helpers

* Avoid rerunning auth server middleware

* Simplify server function scoping ADR

* Fix backlinks project scoping in hosted auth

* Refine auth route foundations

* Simplify ensure user auth resolution

Split auth-mode context resolvers into focused modules so the middleware reads as request orchestration instead of implementation details. Reuse a shared ensured-user context type across server middleware.

* Simplify hosted organization bootstrap

Use Better Auth to own hosted organization creation and membership so hosted auth only needs to resolve a default active organization. Keep delegated-mode compatibility records isolated in a separate helper.

* Clarify hosted auth and backlinks behavior

Document the hosted AUTH_MODE deploy contract and explain why hosted deployments skip manual backlinks verification. This makes the platform-managed behavior explicit in the code paths that differ from self-serve mode.

* Document hosted org creation callback

Explain why auth.ts injects createOrganization into the hosted org helper. This makes the dependency direction explicit and avoids future import cycles while keeping the helper reusable.

* Fix CI check failures

* Fix nav link prop forwarding

* save
2026-03-19 19:24:34 -04:00
Ben Senescu
82c2f99ec1
fix: normalize trailing slashes for page backlinks (#42) 2026-03-19 10:29:12 -04:00
Ben Senescu
8405564925
feat: Add Backlinks page (#23)
* feat: add backlinks setup and overview workflow

* save

* fix backlinks timeseries and refine overview layout

* update backlinks table guidance

* refactor backlinks CI cleanup

* refactor backlinks page state colocation

* fix backlinks access error handling

Differentiate DataForSEO subscription access failures from billing issues and preserve exact page hosts for page-level backlink lookups.

* fix backlinks project scoping and error states

* simplify backlinks billing and lazy loading

* harden backlinks profiling and access flow

* refine backlinks access status typing

* update backlinks scope selection and access handling

* fix backlinks scope fallback and cache scoping

* fix backlinks ci checks

* docs: simplify backlinks pricing

* refine backlinks search: move scope toggle below input, remove redundant description

* refine backlinks summaries and help text

* refine backlinks table sorting

* fix backlinks invalid target handling

* delete docs

* fix backlinks scope inference and domain normalization

* docs: mark backlinks as a supported workflow

* fix backlinks links badge wrapping

* test: split backlinks test suites
2026-03-16 17:48:13 -04:00
Ben Senescu
7b4f85dc10
test: add vitest checks and run tests in CI (#21) 2026-03-12 19:34:18 -04:00
Ben Senescu
b9c1061934
chore: clean up knip.jsonc (#20)
* Add website

* chore: clean knip config and remove dead exports
2026-03-11 23:36:54 -04:00