530 Commits

Author SHA1 Message Date
Ben Senescu
e39de22f96
Fix: don't mask DataForSEO 40501 validation errors as empty results (#346)
* Fix: don't mask DataForSEO 40501 validation errors as empty results

isNoResultsTask classified any task with status_code 40501 as a
successful empty result, but 40501 is not unique to "No Search
Results" — DataForSEO also returns it for validation rejections
like "Invalid Field: 'target'.".

After the Business Listings and Q&A endpoints opted into
treatNoResultsAsEmpty, invalid-field 40501 responses (reachable via
the MCP search_local_businesses categories input, which accepts
arbitrary strings) were silently masked as empty successes — billed
and tracked, but hiding the real provider validation error.

Match on the status message ("no search results") instead of the
ambiguous code so charged validation failures flow through to
DataforseoChargedTaskError with the diagnostic message, while genuine
no-results responses still return as empty successes. Fixes all call
sites uniformly (Business, SERP live, SERP task_get polling).

* Format isNoResultsTask per prettier
2026-07-05 17:54:14 -04:00
Ben Senescu
9b6c5640cd
Sanitize Copy keywords clipboard against spreadsheet formula injection (#348)
The Striking Distance 'Copy keywords' action wrote raw selected GSC query
strings to the clipboard, bypassing the formula-injection sanitizer used
by the CSV and Sheets export paths. GSC query strings are untrusted and can
begin with =, +, -, @, tab, CR, or LF; pasting them into Sheets/Excel could
execute them as formulas.

Route the copy path through normalizeExportValue (same OWASP-recommended
guard as CSV/Sheets exports) so dangerous leading characters are prefixed
with a single quote.
2026-07-05 17:50:01 -04:00
Ben Senescu
7c64cd3c5c
Rank tracking: enforce config cap on unarchive, raise cap to 500 (#347)
The archived-config reactivation path in createConfig returned before the
MAX_CONFIGS_PER_PROJECT check, so re-adding previously archived domains
could push a project past the active-config cap (Codex security finding
ba873417). The cap check now runs before both the reactivation and the
new-row insert. Also raises MAX_CONFIGS_PER_PROJECT from 100 to 500.
2026-07-05 17:39:53 -04:00
Ben Senescu
c993ef4931
Bound concurrent D1 upserts in saved-keyword metric refresh (#338)
refreshSavedKeywordMetrics fanned out one Promise per keyword across an
entire location/language group, so a project with thousands of saved
keywords could trigger thousands of simultaneous D1 upserts, pressuring
Worker memory/CPU and D1 concurrency (potential OOM / availability DoS).

Restore bounded DB-write batching: upsert in chunks of 100 so concurrent
writes stay capped regardless of group size.
2026-07-05 17:35:58 -04:00
Ben Senescu
1a74904b67
Cut worker isolate baseline memory: stub just-bash, lazy-load cheerio (#340)
Production OOM triage: every 'Worker exceeded memory limit' burst hits
unrelated cheap routes right after a deploy — the 128MB limit is per
isolate, and the main worker's eagerly-evaluated module graph is what
crowds it, not any single request.

- Alias just-bash to a throwing stub (worker never uses it): removes
  just-bash + turndown + @mixmark-io/domino from the bundle. The chain
  was pulled in eagerly by @cloudflare/think via the SamChatAgent
  re-export in src/server.ts; SAM only uses its own MCP tools.
- Disable Think's workspace bash tool on SamChatAgent so the stub is
  unreachable at runtime.
- Dynamic-import page-analyzer (cheerio) in the site-audit crawl step
  so it evaluates only when an audit runs, not in every isolate.
- Drop the turndown CJS alias workaround (#339): turndown is no longer
  in the graph at all.

Main eager server chunk: 14,434 kB -> 11,712 kB (-19%); cheerio's
503 kB now a lazy chunk.
2026-07-03 22:23:35 -04:00
Ben Senescu
93cf32081a
Fix turndown ESM build compatibility with Cloudflare Workers (#339) 2026-07-03 21:53:55 -04:00
Ben Senescu
41d37792e7
feat: SAM — in-app SEO agent with full MCP toolset (#322) 2026-07-03 18:48:18 -04:00
Ben Senescu
7ed574c408
Sidebar: promote AI & MCP to nav, consolidate footer into account menu (#337) 2026-07-03 13:23:18 -04:00
Ben Senescu
615ecce033
Redesign app shell: Sidebar with cutout content panel (#336) 2026-07-02 19:06:13 -04:00
Ben Senescu
4f17fe4942 Triage production log errors: audit crash, Autumn webhook FK, PostHog capture, auth rate-limit IP, log noise (#327) 2026-07-02 18:15:46 -04:00
Ben Senescu
fffdbc9329 Add Cloudflare Turnstile captcha on email signup (#326) 2026-07-02 18:15:46 -04:00
Ben Senescu
97f026562e Pin production to Postgres via committed Hyperdrive binding (#329) 2026-07-02 18:15:46 -04:00
Ben Senescu
b5bda159ea Re-add free plan as the floor; remove subscribe gate (#321) 2026-07-02 18:15:46 -04:00
Ben Senescu
819e33be07 fix(mcp): 405 the standalone GET SSE stream to stop /mcp OOM (#325)
The stateless MCP server returns JSON on POST (enableJsonResponse) and
pushes no server-initiated messages, so the optional standalone GET SSE
stream serves no purpose. Left enabled, each GET holds an SSE stream open
indefinitely (25s keepalive, no eventStore) and pins a fresh per-request
McpServer (~5MB of tools + Zod schemas); a few dozen concurrent connected
clients exceed the 128MB isolate limit. This was 100% of the /mcp
exceededMemory OOMs (GET only; POST never OOMed).

Return 405 (spec-compliant 'no standalone stream') before building the
server, so GET allocates nothing. Also removes the bulk of the elevated
GET canceled / responseStreamDisconnected outcomes.
2026-07-02 18:15:46 -04:00
Ben Senescu
a49ec51e95 fix(ai-search): use valid Claude model_name and fail fast on unknown ones (#323)
DataForSEO dropped the Claude Sonnet 4.0 family from its llm_responses
catalog, so model_name=claude-sonnet-4-0 was rejected with 'Invalid
Field: model_name' while still billing the failed task. Point Claude at
claude-sonnet-4-5 and validate every model_name against DataForSEO's
accepted catalog before dispatching the paid call.
2026-07-02 18:15:46 -04:00
Ben Senescu
4e48f8344f Fix production errors: onboarding crash hardening + DataForSEO spend/noise cleanup (#282) 2026-07-02 18:15:46 -04:00
Ben Senescu
868924e1fe refactor(keywords): unify keyword-metric fetching behind one helper (#320) 2026-07-02 18:15:46 -04:00
Ben Senescu
116719a019 fix(db): add missing indexes and drop redundant ones (#319)
Postgres advisor flagged seq-scans and redundant indexes across both
backends (D1 + Postgres):

- add projects(organization_id) — org-scoped project listings seq-scanned
- add account(account_id, provider_id) — better-auth sign-in lookup
- add verification(expires_at) — expired-token cleanup range scan
- drop saved_keyword_tag_assignments_keyword_idx — covered by unique
  (saved_keyword_id, tag_id) prefix
- drop rank_snapshots_run_idx — covered by unique
  (run_id, tracking_keyword_id, device) prefix

Mirrored in both schema dialects + parity-test required-index guard.
2026-07-02 18:15:46 -04:00
Ben Senescu
e5e5029e6b rank-tracking: raise per-project config limit from 20 to 100 (#318)
The cap was only a soft guard against runaway scheduled DataForSEO
workload, not a hard product constraint. Bump it to 100 so projects
tracking many domain/location combos aren't blocked.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-02 18:15:46 -04:00
Ben Senescu
fa0651b96d Fix Postgres-only rank-tracking & site-audit workflow failures (#317) 2026-07-02 18:15:46 -04:00
Ben Senescu
19a4bc6585 D1 → Postgres data migration (ETL + runbook) (#274) 2026-07-02 18:15:46 -04:00
Ben Senescu
c5cbe84ce6 Unify dual-backend DB layer (D1 default + Postgres opt-in) (#238) 2026-07-02 18:15:46 -04:00
Ben Senescu
3a2dad3a96 fix(rank-tracking): unarchive config when re-adding an archived domain (#313) 2026-07-02 18:15:46 -04:00
Ben Senescu
9942c5a590 website: change link 2026-07-02 18:15:46 -04:00
Ben Senescu
1164a3467a feat(onboarding): hide agent chat step; subscribe after intro steps (#312)
* feat(onboarding): hide agent chat step; subscribe after intro steps

Remove the hosted-only strategy-chat diversion from the onboarding
sequence. After the three intro questions, hosted users now hit the
subscribe paywall directly, then return to the GSC and MCP connect
steps. The chat route and components stay in place but unlinked, to be
revisited later. Preserve the post-payment 'You're in!' interstitial by
carrying checkout=success through validateSearch.

* fix(onboarding): set checkout=success from subscribe route, not speculatively

The previous redirect baked checkout=success into the onboarding return
URL at the point needsSubscription is true — i.e. before the user had
paid. It only worked because the subscribe route gates its redirect on
actual access. Move the marker to the subscribe route's redirect-to-app
path, where checkoutCompleted reflects a real returned-from-Stripe
payment, so the 'You're in!' screen can never show pre-payment.
2026-07-02 18:15:46 -04:00
mattmacrocket
c8a1e17359
fix: surface DataForSEO auth failures as an actionable error instead of "unexpected error" (#53) 2026-07-02 17:38:13 -04:00
mattmacrocket
b8f47a51cc
feat: GSC UI + Striking Distance 2026-07-02 17:37:36 -04:00
Ben Senescu
b733be4a5e
release: v0.0.23 - fix mcp for pi + refresh saved keyword metrics (#56)
* fix(mcp): render full row data in tool text output, not just counts

MCP clients that surface only the text content block (not structuredContent)
saw count/truncated summaries instead of the actual rows. Add a shared table
renderer (server/mcp/table.ts) and use it so every row ships in the text block
across research_keywords, get_ranked_keywords, get_keyword_metrics,
search_local_businesses, get_local_serp_results, get_google_business_questions,
find_serp_competitors, get_backlinks_profile, get_backlinks_overview,
get_domain_keyword_suggestions, get_rank_tracker, get_serp_results, and
get_search_console_performance.

* test(mcp): assert tool text output renders full row tables

Cover the shared table renderer (table.test.ts) and add per-tool assertions
that each tool's text content block contains the actual row data (exact
rendered rows, null->em-dash), guarding against a column wired to the wrong
field. Adds tool-text-output.test.ts for the service-backed tools and text
assertions to the DataForSEO and Search Console tool tests.

* release: v0.0.23

* style(mcp): prettier-format tool table columns
2026-07-02 13:01:44 -04:00
Eugene
752561b0ac
feat: add refresh metrics action to Saved Keywords page (#51)
* feat: add refresh metrics action to Saved Keywords page

Adds an Actions dropdown to the Saved Keywords header with an
"Update keyword stats" option. Fetches fresh volume, CPC, competition,
difficulty, and intent from DataForSEO for all saved keywords in the
project, grouped by location/language. Mirrors the existing refresh
pattern from Rank Tracking.

Closes #49

* fix(keywords): preserve full metric shape on saved-keyword refresh

Align refreshSavedKeywordMetrics with the research/save persistence shape
so a refresh never degrades stored data:

- Persist real monthly_searches trend instead of writing "[]"
- Derive Google Ads competition from competition_index/100 instead of null
- Normalize intent via normalizeIntent (Labs) / "unknown" (Ads) to match
  mapKeywordDataItems / mapAdsKeywordItems

Also extract the per-batch fetch+map into fetchBatchMetrics to resolve the
oxlint max-depth violation, and apply prettier formatting so ci:check passes.

---------

Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-30 15:52:12 -04:00
Ben Senescu
e9871bde8d
release: v0.0.22 (#310) 2026-06-28 20:12:29 -04:00
Matt Van Horn
1dabbc88a6
feat: add paginated get_backlinks_profile MCP tool (#38)
* feat: add paginated get_backlinks_profile MCP tool

Exposes detailed per-link backlink rows over MCP, reusing BacklinksService
paginated row fetching. Closes #36.

* fix(test): use type-narrowing text assertion to satisfy oxlint

expect.stringContaining inside toMatchObject tripped
typescript-eslint(no-unsafe-assignment), breaking ci:check. Match the
.toContain() pattern used elsewhere in the MCP tool tests.

* refactor(mcp): simplify get_backlinks_profile handler + fix cost estimate

- Drop the redundant backlinksRowsPageRequestSchema.parse re-validation in
  the handler; the MCP SDK already validates args against inputSchema. Build
  the service request straight from args, removing the duplicated defaults
  that could silently diverge.
- Preserve the target length cap by adding .max(2048) to the input schema
  (previously enforced only via the re-parse).
- Correct the credit estimate in the tool description: measured cost is
  ~30 credits/page, not the ~200-500 copied from get_backlinks_overview.
- Minor: type the statuses accumulator.

* fix(mcp): correct get_backlinks_overview credit estimate

Measured real cost: summary (~26 credits) + history for domain scope
(~29 credits) = ~50 per domain, ~25 for a single page. The previous
~200-500 figure was a ~5-10x overestimate (same one get_backlinks_profile
inherited).

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-28 18:47:24 -04:00
Matt Van Horn
200a6e8b61
feat: add search and filters to the Tracked Domains list (#39)
* feat: add search and filters to the Tracked Domains list

Client-side search + device/country filtering over the tracked-domains set,
handling multi-config domains. Closes #33.

* design: gate tracked-domains filter bar by volume and flatten its chrome

Hide the domain filter bar until there are enough rows to warrant it
(>= 6, or whenever a filter is active so it can't be orphaned), drop the
redundant 'Refine results' sub-header in favor of an inline clear control,
and remove the gradient band so the filter row shares the card surface
with a single divider under the header.

* review: fix failing test, drop dead exports, clear oxlint/knip errors

- Align getDomainListFilterOptions test fixtures with the real LOCATIONS
  map (short labels FR/UK), which the suite asserted as full names — the
  list rows already render short labels, so the dropdown matches them.
- Inline the single-use FilterPanelHeader back into FilterPanel (it was
  extracted to share with the domain filter bar, which no longer uses it).
- Validate the device <select> value instead of an unsafe type assertion,
  matching the config/header select idioms.
- Use toSorted() for location options; memoize allSummaries so dependent
  useMemos have a stable dependency; un-export the internal option type.

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-28 18:45:57 -04:00
Amol Budhwant
a19baed678
feat(rank-tracking): allow explicit language selection (#46)
* feat(rank-tracking): allow explicit language selection

* fix(rank-tracking): remove duplicate country picker and fix LocationSelect handler

The modal rendered two country pickers: a stale native <select> with a
broken empty onChange and the new LocationSelect combobox. The LocationSelect
handler also treated its numeric argument as a DOM event (Number(e.target.value)
-> NaN). Remove the dead <select> and pass the location code through directly.

* feat(rank-tracking): offer the full DataForSEO SERP language list

The language picker only listed the ~43 languages that happened to be a
country's default, so users could not track e.g. Hindi, Chinese (Simplified),
Tamil, or Urdu. Expand LANGUAGE_OPTIONS to the full set of languages the
DataForSEO SERP (Google) API accepts (sourced live from
/v3/serp/google/languages), dropping the deprecated 'iw' Hebrew duplicate and
the redundant 'no' (Norway uses 'nb', which both SERP and Labs accept). Mismatched
location+language pairs are accepted by SERP and invalid codes fail at zero cost,
so the wider list adds no charged-but-failed risk.

Also drop the now-unused LOCATION_OPTIONS re-export from the client shim.

* docs(rank-tracking): cite DataForSEO source for the language list

Note where LANGUAGE_OPTIONS comes from (/v3/serp/google/languages) and how it
relates to the country list (the Labs locations_and_languages endpoint), plus
the intentional deviations from the raw endpoint.

* chore(knip): treat scripts/** as entry points

The standalone CLI/dev scripts under scripts/ are invoked via package.json
scripts (tsx/node), not imported, so knip flagged all six as unused files.
Add scripts/** to knip entry so ci:check passes. (Pre-existing on main;
bundled here to keep the branch's ci:check green.)

* feat(rank-tracking): filter language picker to the country's supported languages

Showing all 128 languages for every country was noise. Restrict the picker to
the languages DataForSEO actually supports for the selected country via a new
getLanguageOptions() helper, backed by a per-country map from the Labs
locations_and_languages endpoint. Most countries expose only their default
language; the ~20 genuinely multilingual ones (US en/es, Canada en/fr,
Switzerland de/fr/it, India en/hi, etc.) list their real set. googleAdsOnly
countries have no per-country language data, so they show their default only.
The Language select is disabled when a country offers a single language.

LANGUAGE_OPTIONS becomes the internal master list (no longer exported).

---------

Co-authored-by: Ben Senescu <44480372+bensenescu@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-28 18:04:49 -04:00
Ben Senescu
da06190e5f
docs: document DCR redirect-URI step for MCP behind Cloudflare Access (#48)
* docs: document DCR redirect-URI step for MCP behind Cloudflare Access

Enabling Managed OAuth alone leaves MCP clients unable to finish Dynamic
Client Registration when their redirect URI isn't allowlisted, so they log
in but expose no tools (issue #47). Document allowing localhost/loopback
clients and HTTPS web-connector redirect URIs, linking Cloudflare's docs.

* docs: fold DCR redirect-URI guidance into the setup steps

Move the Dynamic Client Registration guidance from a standalone paragraph
into step 6 of the numbered list, with sub-bullets for localhost/web
redirect URIs and a footnote explaining why (logs in but no tools).

* docs: replace DCR footnote with a sub-bullet
2026-06-28 17:58:51 -04:00
jalendarreddy97
e4b0c53b3b
Add monthly rank tracking schedule (#45)
* feat: add monthly rank tracking schedule

* fix(ci): un-export ScheduledRankTrackingInterval to satisfy knip

---------

Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-28 17:32:05 -04:00
Ben Senescu
93c2ecf48d fix knip + reorganize self hosting docs 2026-06-28 16:50:16 -04:00
Ben Senescu
6313812f3f Dynamic star count on website (#306) 2026-06-28 16:38:09 -04:00
Ben Senescu
cd8ce2f752 fix(ci): raise Node heap limit for sourcemaps build (#304) 2026-06-28 16:38:09 -04:00
Matt Van Horn
be60440ca0
docs: document manual Wrangler deploy path for Cloudflare self-hosting (#44)
Fixes #27

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-06-28 16:30:04 -04:00
Fishlex0
e4a90d6767
feat: add country combobox (#43)
* feat: add country combobox

* refactor: change to a switch statement
2026-06-28 16:29:53 -04:00
Ben Senescu
e9955b46cb
Simplify readme (#303) 2026-06-26 15:20:17 -04:00
Ben Senescu
7cc4fa9aa5
Revert "Update base plan price from $10/month to $20/month (#283)" (#301)
This reverts commit 0adf77824ac9be3ab01e06b6f697482793823488.
2026-06-26 09:29:52 -04:00
Ben Senescu
616b6727fa
Onboarding chat: let read_website fetch specific user-named URLs (#300) 2026-06-25 19:57:50 -04:00
Ben Senescu
90a04d1845
Onboarding chat: MiniMax M3 + market-research tools + UX polish (#296) 2026-06-25 12:31:07 -04:00
Ben Senescu
770005e2b9
release: v0.0.21 (#293) 2026-06-24 15:22:17 -04:00
Ben Senescu
2087d0d614
Onboarding agent: clarify OpenSEO analyzes any domain; keep recs in-product (#295) 2026-06-24 15:19:17 -04:00
Ben Senescu
327d741d4c Fix onboarding overflow issue 2026-06-24 13:45:47 -04:00
Ben Senescu
23e7eea4c2 bump node max-old-space 2026-06-24 13:24:11 -04:00
Ben Senescu
31eb147bb5
Revise 'Why use OpenSEO?' section in README 2026-06-24 13:23:18 -04:00
Ben Senescu
0adf77824a
Update base plan price from $10/month to $20/month (#283)
Bump all customer-facing references to the managed base plan price from
$10/month to $20/month, and scale the bundled "Includes $X of Usage
Credits" copy from $10 to $20 to match the price.

Covers the subscribe/billing pages, onboarding plan card and fact sheet,
backlinks/AI-search access gates, and the marketing pricing/comparison/
GSC pages plus the README.


Claude-Session: https://claude.ai/code/session_017QayWXGubs1JNbLbqRF4CS

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-21 22:01:25 -04:00