33 Commits

Author SHA1 Message Date
Ben Senescu
f2c41b1db1 MCP activation + usage PostHog events, North Star dashboard prompt (#369)
* Add MCP activation + usage PostHog events and North Star dashboard prompt

- mcp:authorize_success (server) on OAuth consent completion
- mcp:tool_call (server) on every MCP tool invocation, with tool/success/
  error_code/client_id and mcp_client vs in_app_agent source
- mcp:consent_viewed / mcp:consent_denied on the OAuth consent page
- mcp:setup_url_copy / mcp:setup_command_copy intent events on /ai
- docs/posthog-north-star-dashboard.md: prompt to configure the PostHog
  core-actions dashboard (activation rate, MCP setup funnel, daily MCP users)

* Fix codex review findings: count schema-rejected MCP calls as failures, wrap audit tools

- mcp:tool_call now fires after output validation; schema mismatches report
  success:false with MCP_OUTPUT_VALIDATION (the SDK surfaces them to the
  client as JSON-RPC errors)
- run_site_audit/get_audit_status/get_audit_issues/get_audit_pages were
  registered without instrumentMcpToolHandler, so their usage and failures
  were invisible

* Remove dashboard prompt doc
2026-07-07 22:08:14 -04:00
Ben Senescu
1c74fded7b Site audit P0 (1/3): issue engine, incremental persistence, block detection (#362)
* Site audit P0 (1/3): issue engine, incremental persistence, block detection

Server-side foundation of the P0 feature set from docs/site-audit-pm-research.md:

- Issue engine: shared registry of issue types (severity, explanation,
  how-to-fix). Per-page reporters run inside crawl steps; cross-page checks
  (duplicate titles/descriptions/content, broken internal links, redirect
  chains/loops, orphan pages) run at finalize as SQL over the persisted crawl.
- New audit_links + audit_issues tables, audit_pages columns (depth, content
  hash, header signals, fetch class, sitemap flag); audit tables moved to
  src/db/{,pg/}audit.schema.ts; migrations 0029 (D1) / 0006 (PG).
- Incremental persistence: pages/links/issues written inside each crawl-batch
  step with deterministic row ids + upserts (retry idempotent); slim step
  state; robots.txt checkpointed as step state; merged progress steps keep a
  10k-page crawl within the Workflows step budget.
- Crawler: manual redirect handling with inline follow of normalization-
  equivalent redirects, response header capture (X-Robots-Tag, Link
  rel=canonical), BFS depth, sitemap-last seeding, SSRF check on discovered
  links, honest 'we were blocked' classification (403/429/cf-mitigated/
  challenge).
- MCP: run_site_audit, get_audit_status, get_audit_issues, get_audit_pages;
  limitTier resolved via shared AuditService.resolveAuditLimitTier.
- Lighthouse strategies reduced to auto/none (legacy all/manual map on read).
- Self-healing: getStatus reconciles audits whose workflow instance errored/
  terminated without reaching mark-failed.

The Issues UI and the badseo.dev e2e fixture site stack on top of this PR.

Deploy notes: run db:migrate:prod (additive); terminate running audits before
deploying — the workflow step structure changed and in-flight instances cannot
replay under the new code (a finalize guard fails them loudly instead of
completing empty).

* Store only internal link edges in audit_links

Both consumers (broken-internal-link and orphan checks) filter on
isInternal; per-page external counts already live on audit_pages.
Dropping external rows cuts stored edges on outbound-heavy sites.
Column stays so P1 external-link checks can re-add rows without a
migration.

* Review fixes: failAudit CAS guard, dedupe hash helpers, cheaper checks

- failAudit only transitions running audits, so the getStatus reconciler
  can't flip a just-completed audit to failed when it races finalize
- collapse the duplicate SHA-256 helper into audit/ids.ts
- finalize integrity guard uses a limit-1 existence probe instead of
  fetching every page row
- get_audit_status MCP tool no longer reads the audit row twice when an
  explicit auditId is given
2026-07-07 22:08:14 -04:00
Ben Senescu
c9bcc444b4
Triage unresolved PostHog errors: DataForSEO validation, DO retry, Autumn 429, exception noise (#333) 2026-07-05 22:38:53 -04:00
Ben Senescu
41d37792e7
feat: SAM — in-app SEO agent with full MCP toolset (#322) 2026-07-03 18:48:18 -04:00
Ben Senescu
4f17fe4942 Triage production log errors: audit crash, Autumn webhook FK, PostHog capture, auth rate-limit IP, log noise (#327) 2026-07-02 18:15:46 -04:00
Ben Senescu
819e33be07 fix(mcp): 405 the standalone GET SSE stream to stop /mcp OOM (#325)
The stateless MCP server returns JSON on POST (enableJsonResponse) and
pushes no server-initiated messages, so the optional standalone GET SSE
stream serves no purpose. Left enabled, each GET holds an SSE stream open
indefinitely (25s keepalive, no eventStore) and pins a fresh per-request
McpServer (~5MB of tools + Zod schemas); a few dozen concurrent connected
clients exceed the 128MB isolate limit. This was 100% of the /mcp
exceededMemory OOMs (GET only; POST never OOMed).

Return 405 (spec-compliant 'no standalone stream') before building the
server, so GET allocates nothing. Also removes the bulk of the elevated
GET canceled / responseStreamDisconnected outcomes.
2026-07-02 18:15:46 -04:00
Ben Senescu
4e48f8344f Fix production errors: onboarding crash hardening + DataForSEO spend/noise cleanup (#282) 2026-07-02 18:15:46 -04:00
Ben Senescu
868924e1fe refactor(keywords): unify keyword-metric fetching behind one helper (#320) 2026-07-02 18:15:46 -04:00
Ben Senescu
b733be4a5e
release: v0.0.23 - fix mcp for pi + refresh saved keyword metrics (#56)
* fix(mcp): render full row data in tool text output, not just counts

MCP clients that surface only the text content block (not structuredContent)
saw count/truncated summaries instead of the actual rows. Add a shared table
renderer (server/mcp/table.ts) and use it so every row ships in the text block
across research_keywords, get_ranked_keywords, get_keyword_metrics,
search_local_businesses, get_local_serp_results, get_google_business_questions,
find_serp_competitors, get_backlinks_profile, get_backlinks_overview,
get_domain_keyword_suggestions, get_rank_tracker, get_serp_results, and
get_search_console_performance.

* test(mcp): assert tool text output renders full row tables

Cover the shared table renderer (table.test.ts) and add per-tool assertions
that each tool's text content block contains the actual row data (exact
rendered rows, null->em-dash), guarding against a column wired to the wrong
field. Adds tool-text-output.test.ts for the service-backed tools and text
assertions to the DataForSEO and Search Console tool tests.

* release: v0.0.23

* style(mcp): prettier-format tool table columns
2026-07-02 13:01:44 -04:00
Matt Van Horn
1dabbc88a6
feat: add paginated get_backlinks_profile MCP tool (#38)
* feat: add paginated get_backlinks_profile MCP tool

Exposes detailed per-link backlink rows over MCP, reusing BacklinksService
paginated row fetching. Closes #36.

* fix(test): use type-narrowing text assertion to satisfy oxlint

expect.stringContaining inside toMatchObject tripped
typescript-eslint(no-unsafe-assignment), breaking ci:check. Match the
.toContain() pattern used elsewhere in the MCP tool tests.

* refactor(mcp): simplify get_backlinks_profile handler + fix cost estimate

- Drop the redundant backlinksRowsPageRequestSchema.parse re-validation in
  the handler; the MCP SDK already validates args against inputSchema. Build
  the service request straight from args, removing the duplicated defaults
  that could silently diverge.
- Preserve the target length cap by adding .max(2048) to the input schema
  (previously enforced only via the re-parse).
- Correct the credit estimate in the tool description: measured cost is
  ~30 credits/page, not the ~200-500 copied from get_backlinks_overview.
- Minor: type the statuses accumulator.

* fix(mcp): correct get_backlinks_overview credit estimate

Measured real cost: summary (~26 credits) + history for domain scope
(~29 credits) = ~50 per domain, ~25 for a single page. The previous
~200-500 figure was a ~5-10x overestimate (same one get_backlinks_profile
inherited).

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Ben Senescu <bensenescu@gmail.com>
2026-06-28 18:47:24 -04:00
Ben Senescu
d339f0c3d9
Fix MCP output validation and error reporting 2026-06-16 13:52:59 -04:00
Ben Senescu
9abed9278f
Remove free trial: subscription paywall + 30-day money-back guarantee (#251) 2026-06-14 21:21:04 -04:00
Ben Senescu
b809f87b5e
Hybrid keyword data: Google Ads coverage for non-Labs countries + opt-in clickstream volumes (#262) 2026-06-12 14:08:11 -04:00
Ben Senescu
e41b52ca56
MCP directory metadata, resources/prompts handlers, full param descriptions (#260) 2026-06-11 16:11:24 -04:00
Ben Senescu
16b92ccea3
Backlinks: server-side pagination, sorting, filters, and a one-per-domain view (#258) 2026-06-10 23:27:04 -04:00
Ben Senescu
076a7fb6d7
Multi-project support per organization (#246) 2026-06-09 20:12:09 -04:00
Ben Senescu
7a26be46cc
Self-hosted Google Search Console (#236) 2026-06-04 18:06:16 -04:00
Ben Senescu
23c38ca4e5
GSC MCP part 1 (#233) 2026-06-04 13:50:26 -04:00
Ben Senescu
b64ef56b4d
refactor: unify dataforseo sdk usage (#231) 2026-06-01 09:48:11 -04:00
Ben Senescu
03fd3588ef
refactor: clean up complex code (#229) 2026-05-28 16:37:33 -04:00
Ben Senescu
506dde84d1
Add additional MCP tools for Local SEO (#222) 2026-05-26 18:38:14 -04:00
Ben Senescu
a3721a0f53
Normalize dynamic MCP client registration for Perplexity (#217) 2026-05-22 22:22:42 -04:00
Ben Senescu
5cef024c68
Add skills for agents (#211)
* Add OpenSEO skills for keyword and link workflows

* Remove credit scare copy from MCP and skills

* Remove local skills test sandbox

* Format OpenSEO skill docs

* Disclose OpenSEO credit threshold for MCP research

* Tighten OpenSEO skill workflows
2026-05-22 08:46:06 -04:00
Ben Senescu
7a0ab0de2c
Add more mcp tool metadara + ChatGPT domain verification route (#197) 2026-05-14 13:02:08 -04:00
Ben Senescu
49a2d7f732
Set MCP access token TTL to 24 hours (#191) 2026-05-13 15:01:32 -04:00
Ben Senescu
570fe2aa62
Add keyword tags, server-side saved filtering, and filtered exports (#179) 2026-05-12 00:02:43 -04:00
Ben Senescu
b9b85a6d48
Support Cloudflare Access and local no-auth MCP modes (#166) 2026-05-10 17:31:30 -04:00
Ben Senescu
0dd818e114
Replace better-auth with Cloudflare for oauth (#174) 2026-05-08 22:19:55 -04:00
Ben Senescu
db10ffa250
Fix Claude Desktop MCP OAuth registration and resume flows (#173) 2026-05-08 18:07:05 -04:00
Ben Senescu
f58efea164
Improve MCP structured result visibility (#167) 2026-05-08 16:17:28 -04:00
Ben Senescu
103b6fe91c
fix: add additional well known for claude.ai + Claude Desktop (#170)
* Simplify MCP discovery metadata and lazy auth flow

* Fix ci check for MCP discovery branch
2026-05-08 15:59:43 -04:00
Ben Senescu
57790b930f
feat: add OpenSEO MCP tools (#161) 2026-05-08 00:34:29 -04:00
Ben Senescu
0ff7bc96b2
Add stateless MCP server (#162) 2026-05-07 23:38:00 -04:00