The stateless MCP server returns JSON on POST (enableJsonResponse) and
pushes no server-initiated messages, so the optional standalone GET SSE
stream serves no purpose. Left enabled, each GET holds an SSE stream open
indefinitely (25s keepalive, no eventStore) and pins a fresh per-request
McpServer (~5MB of tools + Zod schemas); a few dozen concurrent connected
clients exceed the 128MB isolate limit. This was 100% of the /mcp
exceededMemory OOMs (GET only; POST never OOMed).
Return 405 (spec-compliant 'no standalone stream') before building the
server, so GET allocates nothing. Also removes the bulk of the elevated
GET canceled / responseStreamDisconnected outcomes.
DataForSEO dropped the Claude Sonnet 4.0 family from its llm_responses
catalog, so model_name=claude-sonnet-4-0 was rejected with 'Invalid
Field: model_name' while still billing the failed task. Point Claude at
claude-sonnet-4-5 and validate every model_name against DataForSEO's
accepted catalog before dispatching the paid call.
The cap was only a soft guard against runaway scheduled DataForSEO
workload, not a hard product constraint. Bump it to 100 so projects
tracking many domain/location combos aren't blocked.
Co-authored-by: Claude <noreply@anthropic.com>
* feat(onboarding): hide agent chat step; subscribe after intro steps
Remove the hosted-only strategy-chat diversion from the onboarding
sequence. After the three intro questions, hosted users now hit the
subscribe paywall directly, then return to the GSC and MCP connect
steps. The chat route and components stay in place but unlinked, to be
revisited later. Preserve the post-payment 'You're in!' interstitial by
carrying checkout=success through validateSearch.
* fix(onboarding): set checkout=success from subscribe route, not speculatively
The previous redirect baked checkout=success into the onboarding return
URL at the point needsSubscription is true — i.e. before the user had
paid. It only worked because the subscribe route gates its redirect on
actual access. Move the marker to the subscribe route's redirect-to-app
path, where checkoutCompleted reflects a real returned-from-Stripe
payment, so the 'You're in!' screen can never show pre-payment.
* fix(mcp): render full row data in tool text output, not just counts
MCP clients that surface only the text content block (not structuredContent)
saw count/truncated summaries instead of the actual rows. Add a shared table
renderer (server/mcp/table.ts) and use it so every row ships in the text block
across research_keywords, get_ranked_keywords, get_keyword_metrics,
search_local_businesses, get_local_serp_results, get_google_business_questions,
find_serp_competitors, get_backlinks_profile, get_backlinks_overview,
get_domain_keyword_suggestions, get_rank_tracker, get_serp_results, and
get_search_console_performance.
* test(mcp): assert tool text output renders full row tables
Cover the shared table renderer (table.test.ts) and add per-tool assertions
that each tool's text content block contains the actual row data (exact
rendered rows, null->em-dash), guarding against a column wired to the wrong
field. Adds tool-text-output.test.ts for the service-backed tools and text
assertions to the DataForSEO and Search Console tool tests.
* release: v0.0.23
* style(mcp): prettier-format tool table columns
* feat: add refresh metrics action to Saved Keywords page
Adds an Actions dropdown to the Saved Keywords header with an
"Update keyword stats" option. Fetches fresh volume, CPC, competition,
difficulty, and intent from DataForSEO for all saved keywords in the
project, grouped by location/language. Mirrors the existing refresh
pattern from Rank Tracking.
Closes#49
* fix(keywords): preserve full metric shape on saved-keyword refresh
Align refreshSavedKeywordMetrics with the research/save persistence shape
so a refresh never degrades stored data:
- Persist real monthly_searches trend instead of writing "[]"
- Derive Google Ads competition from competition_index/100 instead of null
- Normalize intent via normalizeIntent (Labs) / "unknown" (Ads) to match
mapKeywordDataItems / mapAdsKeywordItems
Also extract the per-batch fetch+map into fetchBatchMetrics to resolve the
oxlint max-depth violation, and apply prettier formatting so ci:check passes.
---------
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
* feat: add paginated get_backlinks_profile MCP tool
Exposes detailed per-link backlink rows over MCP, reusing BacklinksService
paginated row fetching. Closes#36.
* fix(test): use type-narrowing text assertion to satisfy oxlint
expect.stringContaining inside toMatchObject tripped
typescript-eslint(no-unsafe-assignment), breaking ci:check. Match the
.toContain() pattern used elsewhere in the MCP tool tests.
* refactor(mcp): simplify get_backlinks_profile handler + fix cost estimate
- Drop the redundant backlinksRowsPageRequestSchema.parse re-validation in
the handler; the MCP SDK already validates args against inputSchema. Build
the service request straight from args, removing the duplicated defaults
that could silently diverge.
- Preserve the target length cap by adding .max(2048) to the input schema
(previously enforced only via the re-parse).
- Correct the credit estimate in the tool description: measured cost is
~30 credits/page, not the ~200-500 copied from get_backlinks_overview.
- Minor: type the statuses accumulator.
* fix(mcp): correct get_backlinks_overview credit estimate
Measured real cost: summary (~26 credits) + history for domain scope
(~29 credits) = ~50 per domain, ~25 for a single page. The previous
~200-500 figure was a ~5-10x overestimate (same one get_backlinks_profile
inherited).
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
* feat: add search and filters to the Tracked Domains list
Client-side search + device/country filtering over the tracked-domains set,
handling multi-config domains. Closes#33.
* design: gate tracked-domains filter bar by volume and flatten its chrome
Hide the domain filter bar until there are enough rows to warrant it
(>= 6, or whenever a filter is active so it can't be orphaned), drop the
redundant 'Refine results' sub-header in favor of an inline clear control,
and remove the gradient band so the filter row shares the card surface
with a single divider under the header.
* review: fix failing test, drop dead exports, clear oxlint/knip errors
- Align getDomainListFilterOptions test fixtures with the real LOCATIONS
map (short labels FR/UK), which the suite asserted as full names — the
list rows already render short labels, so the dropdown matches them.
- Inline the single-use FilterPanelHeader back into FilterPanel (it was
extracted to share with the domain filter bar, which no longer uses it).
- Validate the device <select> value instead of an unsafe type assertion,
matching the config/header select idioms.
- Use toSorted() for location options; memoize allSummaries so dependent
useMemos have a stable dependency; un-export the internal option type.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
* feat(rank-tracking): allow explicit language selection
* fix(rank-tracking): remove duplicate country picker and fix LocationSelect handler
The modal rendered two country pickers: a stale native <select> with a
broken empty onChange and the new LocationSelect combobox. The LocationSelect
handler also treated its numeric argument as a DOM event (Number(e.target.value)
-> NaN). Remove the dead <select> and pass the location code through directly.
* feat(rank-tracking): offer the full DataForSEO SERP language list
The language picker only listed the ~43 languages that happened to be a
country's default, so users could not track e.g. Hindi, Chinese (Simplified),
Tamil, or Urdu. Expand LANGUAGE_OPTIONS to the full set of languages the
DataForSEO SERP (Google) API accepts (sourced live from
/v3/serp/google/languages), dropping the deprecated 'iw' Hebrew duplicate and
the redundant 'no' (Norway uses 'nb', which both SERP and Labs accept). Mismatched
location+language pairs are accepted by SERP and invalid codes fail at zero cost,
so the wider list adds no charged-but-failed risk.
Also drop the now-unused LOCATION_OPTIONS re-export from the client shim.
* docs(rank-tracking): cite DataForSEO source for the language list
Note where LANGUAGE_OPTIONS comes from (/v3/serp/google/languages) and how it
relates to the country list (the Labs locations_and_languages endpoint), plus
the intentional deviations from the raw endpoint.
* chore(knip): treat scripts/** as entry points
The standalone CLI/dev scripts under scripts/ are invoked via package.json
scripts (tsx/node), not imported, so knip flagged all six as unused files.
Add scripts/** to knip entry so ci:check passes. (Pre-existing on main;
bundled here to keep the branch's ci:check green.)
* feat(rank-tracking): filter language picker to the country's supported languages
Showing all 128 languages for every country was noise. Restrict the picker to
the languages DataForSEO actually supports for the selected country via a new
getLanguageOptions() helper, backed by a per-country map from the Labs
locations_and_languages endpoint. Most countries expose only their default
language; the ~20 genuinely multilingual ones (US en/es, Canada en/fr,
Switzerland de/fr/it, India en/hi, etc.) list their real set. googleAdsOnly
countries have no per-country language data, so they show their default only.
The Language select is disabled when a country offers a single language.
LANGUAGE_OPTIONS becomes the internal master list (no longer exported).
---------
Co-authored-by: Ben Senescu <44480372+bensenescu@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
* docs: document DCR redirect-URI step for MCP behind Cloudflare Access
Enabling Managed OAuth alone leaves MCP clients unable to finish Dynamic
Client Registration when their redirect URI isn't allowlisted, so they log
in but expose no tools (issue #47). Document allowing localhost/loopback
clients and HTTPS web-connector redirect URIs, linking Cloudflare's docs.
* docs: fold DCR redirect-URI guidance into the setup steps
Move the Dynamic Client Registration guidance from a standalone paragraph
into step 6 of the numbered list, with sub-bullets for localhost/web
redirect URIs and a footnote explaining why (logs in but no tools).
* docs: replace DCR footnote with a sub-bullet
Bump all customer-facing references to the managed base plan price from
$10/month to $20/month, and scale the bundled "Includes $X of Usage
Credits" copy from $10 to $20 to match the price.
Covers the subscribe/billing pages, onboarding plan card and fact sheet,
backlinks/AI-search access gates, and the marketing pricing/comparison/
GSC pages plus the README.
Claude-Session: https://claude.ai/code/session_017QayWXGubs1JNbLbqRF4CS
Co-authored-by: Claude <noreply@anthropic.com>
* Add onboarding agent v1 product spec
* Add onboarding agent implementation plan (Project Think)
* Update onboarding plan: chat + seed function (drop Think/Workflows)
* feat(onboarding): data + metering foundation, Project Context store, MCP tool
* feat(onboarding): site read + DataForSEO signal + OpenRouter strategy seed
* feat(onboarding): strategy + streaming chat UI with update_project_context tool
* fix(onboarding): address review — bound free runs, cap chat, share auth+error helpers, harden scrape
* fix(onboarding): use canonical keyword-locations list, not a separate country list
* Improve onboarding strategy chat
* feat(onboarding): refine upgrade rail UI + fact-checked copy
- Rebuild upgrade sidebar: drop the nested card so the rail itself is the
container (header / plan / features / CTA / progress footer with dividers)
- Remove the 'Free preview' badge + headline pitch; header now reads
'Previewing OpenSEO' with the site domain beneath
- Tighten copy against the fact sheet: fix monthly-vs-top-up credit wording,
drop 'live' rank tracking, add money-back + open-source trust signals,
unify CTAs to 'Upgrade to continue', cut cross-panel feature redundancy
- Replace off-strategy suggested question; add progress bar counter
- FORCE_FREE_PREVIEW flag to always show the preview/limit UI while testing
* feat(onboarding): add 'What do you recommend' strategy chip; revert suggested questions
- Add a highlighted suggestion chip that prompts Sam for the strategy, shown
only when the user hasn't already used the welcome 'Show my strategy' CTA
- Track strategyRequested so the chip isn't re-offered after use
- Restore the original four suggested questions
* feat(onboarding): add OpenSEO Discord CTA + fact-sheet entry
- Discord link in the upgrade sidebar
- Fact-sheet community entry + system-prompt guidance so Sam can point
users to the Discord for community/second-opinion help
* chore(merge-ready): round 1 fixes
- scrape.ts SSRF: validate the initial domain via audit/url-policy
(normalizeAndValidateStartUrl) and re-validate each redirect hop with
redirect:"manual" (one hop, blocked/private/metadata hosts + DoH rebinding).
Replace the content-length-only guard with a bounded streaming read so
chunked/CDN responses can't buffer past MAX_RESPONSE_BYTES. Remove the
unguarded normalizeDomainToUrl helper. Add scrape.test.ts.
- http-errors.ts: map PAYMENT_REQUIRED AppError to HTTP 402 (was 500), so the
onboarding chat paywall backstop surfaces correctly.
- OnboardingStrategyChat: replace the hardcoded FORCE_FREE_PREVIEW=true debug
flag (which forced paid users into the free-preview/paywall UI) with a
safe-by-default ?preview=1 URL override.
- onboardingStrategy.ts: delete the dead generateOnboardingStrategy export
(knip) and its now-unused imports; the chat tool path uses runOnboardingSeed.
- chat.ts: rename inner runOnboardingSeed result to fix no-shadow.
- Extract presentational chat sub-components into OnboardingStrategyChatParts
to satisfy max-lines; reformat Markdown.tsx for prettier.
* chore(merge-ready): round 2 fixes
- chat.ts: validate message role in schema + count total messages (not just user-role) so the free-question gate can't be bypassed with mislabelled roles
- OnboardingStrategyChat.tsx: surface useChat error state with a paywall-aware notice; branch 'Ask about OpenSEO' message text on isPaid
- OnboardingStrategyChatParts.tsx: guard free-preview welcome copy behind !isPaid (paid variant for subscribers)
- onboardingStrategy.ts: reset onboardingRunStatus/onboardingRunAt when the domain changes so a corrected domain can get a fresh free seed
* chore(merge-ready): round 3 fixes
- onboarding chat: count only user-role messages for free-question paywall to match client gate (was counting all messages, firing ~3 turns early)
- ProjectContextStore: drop unused return value/type from saveProjectContextVersion, inline latest-version query into getCurrentProjectContextMarkdown, remove dead toVersion helper and ProjectContextVersion type
- onboarding chat UI: replace 'Why is OpenSEO better than Claude?' suggested chip with 'How does OpenSEO work with Claude?' (Claude is an MCP client, not a competitor)
* feat(onboarding): route post-upgrade to GSC step; drop isPaid from preview chat
- Checkout successUrl now returns to /onboarding?step=3 (GSC connect) instead
of the strategy chat, with a 'You're in!' success banner introducing the
remaining GSC + MCP setup steps. Fixes the post-Stripe 'stuck on paywall'
race since the user leaves the chat entirely.
- The strategy chat is now purely the pre-upgrade free preview: removed the
managed-access query, the isPaid branching, and the ?preview override. The
7-question cap always applies (kept as a conversion funnel).
* feat(onboarding): show post-upgrade success as its own step screen
Instead of a banner stacked above the GSC step, render a 'You're in!' screen
using the standard step layout (logo, title, card, Continue) in place of the
GSC step when ?checkout=success is present. Continue drops the param to reveal
the actual GSC step.
* refactor(nav): remove Project settings from account dropdown
Project settings is now reachable only via the project switcher's 'Manage
projects' → /projects → per-project settings. Drops the dead
projectSettingsLinkOptions helper and the now-unused AccountMenu projectId prop.
* refactor(onboarding): remove project-context persistence + MCP tool
Defers the Project Context store to a later PR to simplify this one.
- Delete ProjectContextStore, the get_project_context MCP tool (+ registration),
the project_context_versions table (schema + migration 0024 + snapshot), and
the update_project_context chat tool.
- generate_initial_strategy now returns the synthesized strategy to the chat
without persisting it; claimRun still gates paid spend (one free run).
- Chat system prompt no longer injects saved context; it just grounds Sam with
the project's domain.
- getOnboardingStrategyState returns only { projectId, domain }.
- Move the agent fact sheet out of docs/ (human docs) to
src/server/features/onboarding/openseo-fact-sheet.md.
* refactor(routing): move /strategy to /onboarding/chat
Rename the onboarding strategy chat route from /strategy to /onboarding/chat.
_authenticated.onboarding.tsx becomes the index route; the chat is a sibling,
so TanStack auto-creates the shared /onboarding parent (Outlet).
* chore(onboarding): clean up leftovers from the persistence removal
- Drop the chat's onFinish=invalidateStrategyState refetch: now that the
strategy state is just { projectId, domain } (no persisted markdown), the
chat can't mutate it, so the post-turn refetch was dead work.
- Fix a stale 'Project Context' doc comment in synthesis.ts.
- Note in specs 0005/0006 that strategy persistence + the MCP read tool were
deferred, so the docs don't contradict the shipped code.
* feat(onboarding): meter LLM (OpenRouter) spend via Autumn track_tokens
Mirror the DataForSEO metering pattern for LLM cost: a best-effort
trackLlmUsage helper emits a PostHog usage event and records token usage on
Autumn's token-tracking endpoint (REST; not in the autumn-js SDK yet), priced
from the model slug. Wired into the chat stream (onFinish) and the strategy
synthesis call. getOnboardingModel now also returns the resolved model slug.
* feat(onboarding): step-styled site form + account menu on chat page
- Restyle the website/country form to match the onboarding step layout (logo,
title, helper) and explain why we ask (read the site + pick the search market).
- Extract OnboardingAccountMenu to a shared component and render it on the
onboarding chat page so signed-in users can reach account actions there too.
* fix(auth): keep verify-email on 'check your inbox' after email sign-up
The post-sign-up redirect always passes ?email=; key the waiting state off it
so a just-signed-up user sees check-your-inbox + resend instead of the sign-in
CTA the verification gate would immediately block, even while the session is
still resolving.
* fix(onboarding): meter total LLM usage across all stream steps
Review caught that streamText onFinish 'usage' is only the last step; with
stopWhen=4 + the strategy tool, multi-step runs under-metered. Use 'totalUsage'
and await the metering so it fires before the stream closes. Also drop the
in-flux cache/reasoning token fields (negligible here).
* feat(onboarding): adopt the 'chat with tools' architecture from agent-onboarding-2
Replace the deterministic seed + synthesis pipeline (and the
onboarding_run_status/run_at columns) with two on-demand tools Sam calls —
read_website and get_seo_metrics — and have Sam write the strategy itself
in-stream, so a mid-stream refresh re-runs cache-backed tools instead of
dead-ending on a 'complete' status. Rename OnboardingStrategy* -> OnboardingChat*.
Preserved from this branch: LLM metering (now via the chat onFinish totalUsage,
covering the in-stream strategy), the account menu on the chat page, the
verify-email fix, and the step-styled site form. Drop columns via migration 0025.
* docs(onboarding): correct spend-bound + stale synthesis comments
Clarify that get_seo_metrics spend is bounded by the question cap + one project
per un-upgraded account (not solely caching, which doesn't cover no-data sites),
and drop 'synthesis' from comments now that Sam writes the strategy in-stream.
* refactor(onboarding): metered LLM via Autumn AI-SDK adapter; drop skipBalanceAssert
Now that every org gets an onboarding_plan with usage credits, onboarding spend
draws down the normal balance — no bypass needed.
- LLM metering: use Autumn's official @useautumn/gateway adapter (withLlmMetering
wraps the model; correct token-pool pricing for cached/reasoning tokens),
replacing the hand-rolled onFinish/track_tokens REST plumbing. Point it at the
existing 'llm_usage' feature (backed by usage_credits + topup_credits) rather
than a to-be-created 'ai_credits' feature.
- DataForSEO: remove skipBalanceAssert end-to-end (chat metering object, the
meter() plumbing in dataforseo/client.ts, and the DomainService override type);
onboarding now asserts balance like every other caller. Kept the email-verified
+ Labs-location gate on get_seo_metrics as the anti-farming bound.
Co-authored with a parallel agent's LLM-metering refactor.
* docs(onboarding): fix stale metering comment + diverged-architecture specs
- DomainService MeteringOverrides comment no longer claims a balance-gate bypass
(skipBalanceAssert + the onboarding seed are gone).
- specs 0005/0006: correct the update notes — the seed/synthesis pipeline,
claimRun, and skipBalanceAssert were replaced by the chat-with-tools design;
flag the bodies as the superseded plan.
- Document the pinned Autumn track_tokens API version.
* feat(onboarding): gate the chat turn on credit balance (LLM included)
Now that every org gets onboarding_plan trial credits and LLM tokens draw from
the same usage/topup balance, assert that balance before streaming — not just
track it. Extract the DataForSEO balance check into subscription.ts
(getUsageCreditsRemaining / assertUsageCreditsAvailable) and reuse it; the chat
throws a friendly PAYMENT_REQUIRED when credits are gone (client shows the
upgrade copy).
* feat(onboarding): make the strategy chat hosted-only
The chat needs the managed LLM + trial credits, so self-hosted has no business
there. Gate the step-2 navigation on hosted mode and add a beforeLoad redirect
on /onboarding/chat so self-hosted lands back in the wizard.
* feat(onboarding): site-form + welcome copy; drop open-source badge
- Site form: 'Tell us about your website.' title, short input labels, no extra
helper descriptions.
- Welcome message: lead with the upgrade ask + a Discord/email escape hatch.
- Remove the 'Open source — self-host for free anytime' badge from the rail.
* fix(onboarding): show typing indicator during the submitted wait
showTyping gated on the last message lacking assistant text, but right after
send the last message is the user's own (which has text), so nothing showed
until the assistant message appeared. Show it whenever busy and the last
message isn't assistant-text-yet.
* refactor(onboarding): drop redundant email-verified gate on get_seo_metrics
The route guard already requires a verified email to reach the chat in hosted
mode, and the trial-credit balance bounds spend — so the in-tool emailVerified
check was redundant for real users and blocked local/bypass testing. Keep the
Labs-location check (functional).
* feat(billing): meter onboarding LLM spend into the shared credit pool
Both DataForSEO and onboarding-LLM now draw from the same usage_credits/
topup_credits pool via one helper, instead of LLM needing a separate Autumn
ai_credit_system.
- Extract trackUsageCreditSpend (markup -> credits -> monthly/topup split ->
autumn.track + usage:credits_consume) into subscription.ts; DataForSEO's
trackDataforseoCost now delegates to it (behavior unchanged, tests pass).
- Enable OpenRouter usage accounting; the chat onFinish sums the real per-step
cost OpenRouter reports and deducts it through the same helper.
- Drop the @useautumn/gateway adapter, llm-metering.ts, track_tokens, and the
AUTUMN_LLM_USAGE_FEATURE_ID constant — no ai_credit_system feature needed.
* feat(onboarding): persist the strategy chat in a Durable Object (AIChatAgent)
Move the onboarding chat from a stateless streamText route to an Agents SDK
AIChatAgent Durable Object, so the conversation persists (DO SQLite) and
survives reloads — one instance per project.
- OnboardingChatAgent.onChatMessage ports the system prompt, read_website +
get_seo_metrics tools, the credit-balance/free-question gate, and the
OpenRouter cost metering. Billing gates surface as a normal assistant message
(staticAssistantResponse) rather than an HTTP 402.
- The Worker authorizes every /agents/* connection (resolve session + verify the
caller's org owns the projectId) before it reaches the DO; the DO derives org
/domain from the project it is named after. Auth stays on the proven path.
- Client swaps useChat -> useAgent + useAgentChat (WebSocket), keyed by projectId.
- Adds the DO binding + new_sqlite_classes migration; pins @cloudflare/ai-chat
0.6.1 to match agents 0.12.3.
* chore(onboarding): bump agents+ai-chat to latest; fix review findings
- Bump agents 0.12.3 -> 0.15.0 and @cloudflare/ai-chat -> 0.8.4 (the supported
pairing; verified MCP, the DO, and the build still compile).
- Thread the per-turn abortSignal into streamText so a user aborting mid-stream
cancels the billable LLM call (was leaking sub-cent cost on abort).
- Ensure the org's Autumn customer exists in the Worker authorize step before
the DO checks the credit balance, avoiding a false 'out of credits' gate on a
brand-new org's first message.
* chore: remove stray reservation-booker-seo-report.html
* chore: drop stale @useautumn/gateway minimumReleaseAge exclusion
The package was removed when LLM metering moved to the shared credit pool.
* perf(onboarding): fetch get_seo_metrics signals in parallel; clarify question-cap
- get_seo_metrics now fetches the domain overview and ranked keywords
concurrently instead of in series (faster tool turn). Trade-off: it always
issues the metered ranked-keywords call now, including for no-ranking sites.
- Correct the FREE_ONBOARDING_QUESTION_LIMIT comment: the server re-check counts
client-supplied history, so the cap is a conversion nudge, not a security
boundary — the credit balance is the real spend bound.
