35 Commits

Author SHA1 Message Date
Ben Senescu
8ea55a3e77
fix gsc onboarding nudge state (#240) 2026-06-04 23:25:01 -04:00
Ben Senescu
7a26be46cc
Self-hosted Google Search Console (#236) 2026-06-04 18:06:16 -04:00
Ben Senescu
23c38ca4e5
GSC MCP part 1 (#233) 2026-06-04 13:50:26 -04:00
Ben Senescu
06e15dc70f
Add per-feature usage breakdown to billing page (#232) 2026-05-31 18:42:41 -04:00
Ben Senescu
8b74c7da61
Improve onboarding questions and sign-up routing (#225) 2026-05-26 23:05:20 -04:00
Ben Senescu
3456af12fe
bump to reddit v3 api (#209) 2026-05-21 00:08:50 -04:00
Ben Senescu
4f638db702
Add Reddit conversion tracking (#208) 2026-05-20 23:36:30 -04:00
Ben Senescu
2484f93ce6
Refactor pages to improve performance and consistency (#205) 2026-05-20 17:23:09 -04:00
Ben Senescu
570fe2aa62
Add keyword tags, server-side saved filtering, and filtered exports (#179) 2026-05-12 00:02:43 -04:00
Ben Senescu
0dd818e114
Replace better-auth with Cloudflare for oauth (#174) 2026-05-08 22:19:55 -04:00
Ben Senescu
0ff7bc96b2
Add stateless MCP server (#162) 2026-05-07 23:38:00 -04:00
Ben Senescu
def9390cea
feat: Implement server-side pagination and filtering for domain keywords (#143) 2026-05-04 15:26:36 -04:00
Ben Senescu
1108cdd9a7
feat: gate llm mentions in self hosted product (#134) 2026-04-23 18:02:24 -04:00
Ben Senescu
faf7a294a3
feat: AI Visibility — Brand Lookup and Prompt Explorer (#128) 2026-04-23 10:48:38 -04:00
Ben Senescu
5c2b71d340
feat: bulk delete for saved keywords (#122) 2026-04-16 19:14:50 -04:00
Ben Senescu
9a9a9a2a6c
feat: add keyword metrics (volume, difficulty, CPC) to rank tracking (#118) 2026-04-16 10:41:31 -04:00
Ben Senescu
86d3714846
feat: allow free plan users to set up rank tracking (#116) 2026-04-16 02:19:23 -04:00
Ben Senescu
aeafa5649d
feat: polish rank tracking UI (#115) 2026-04-16 00:42:10 -04:00
Ben Senescu
53a83996ef feat: Rank Tracking MVP (#91) 2026-04-14 20:29:38 -04:00
Ben Senescu
c9d212b8b8 fix: explicitly pick billingCustomer fields for workflow serialization (#97)
The full middleware context object passed as billingCustomer contained
non-serializable properties (Drizzle project result, TanStack internals)
that caused a DataCloneError in Miniflare's workflow binding during
Docker self-hosting.
2026-04-08 14:09:02 -04:00
Ben Senescu
93c95dc8f7 feat: add structured filters to backlinks tables (#94)
* feat: add structured filters to backlinks tables

Add per-tab filter panels (Backlinks, Referring Domains, Top Pages) with
include/exclude text search, numeric range filters, link type selector,
and visibility toggles. Filter values persist in localStorage across
searches and page reloads.

* fix: improve backlinks filter layout and hide domain overview filters on pages tab

- Add w-full to text filter inputs so they fill their grid columns
- Restructure backlinks results card to use border-separated sections
  matching the domain overview pattern
- Hide filter button and panel on domain overview's Top Pages tab since
  keyword filters don't apply there

* fix: move backlinks filter button to its own toolbar row below tabs

Match the domain overview layout: tabs in their own header row, then a
separate toolbar row with the Filters button underneath.

* style: fix prettier formatting in backlinks filter files

* fix: resolve all ci:check issues

- Remove unused BacklinksResultsHeader.tsx and useBacklinksSpamPreferences.ts
- Un-export DEFAULT_BACKLINKS_SPAM_THRESHOLD and normalizeBacklinksSpamThreshold
  (only used internally)
- Remove unused BacklinksAllFilterValues type
- Fix oxlint unsafe type assertions in useBacklinksFilters.ts

* fix: wire spam filter options through to server and remove spam filters from non-backlinks tabs

The client was not passing hideSpam/spamThreshold to the server, so the
server always defaulted to hideSpam:true with threshold 40. This made
the new client-side spam score filters on the backlinks tab silently
filter an already-truncated dataset. Now the client sends hideSpam:false
so the server returns all rows and client-side filtering works correctly.

Removed spam score filters from the Referring Domains tab since spam
filtering is not needed there.

* fix: restore backlinks spam filtering and safe filter hydration

* simplify: remove server-side spam filter wiring, keep client-side only

All backlinks filtering (including spam score) is now done client-side.
Server-side spam filtering adds complexity without meaningful benefit
at current data volumes. This removes the bridging code added in the
last two commits and simplifies filter hydration.
2026-04-08 14:09:02 -04:00
Ben Senescu
ad3b732f60 hosted: add product analytics (#83)
* track core product analytics flows

Track auth, search, export, audit, and credit-consumption events with canonical route IDs so PostHog funnels and usage dashboards stay low-noise and privacy-safe.

* fix: keep auth actions usable after session loss

* refactor: simplify analytics and auth helpers

- Replace isRecord/getActiveOrganizationId type guards with simple cast
- Refactor getAnalyticsRouteContext from if/return chain to route tables
- Replace toVerificationIssueType switch with zod enum
- Merge duplicate credits_consume events into single event per API call
- Merge two PostHogBootstrap useEffects into one

* refactor: add projectId to middleware context to reduce boilerplate

The requireProjectContext middleware now includes projectId directly,
eliminating repeated manual construction of BillingCustomerContext
objects across all server function handlers.

* remove unused BILLING_* env var fallbacks from cost profile script

* remove before_send event enrichment to preserve native PostHog URL tracking

The before_send hook was stripping $pathname, $current_url, $referrer and
other URL properties, which breaks PostHog web analytics dashboards, paths
analysis, session replay, and attribution. The route_id/route_group injection
it provided is unnecessary since PostHog already captures $pathname natively.

* remove route mapping layer, pass raw redirect paths to analytics events

The route ID registry (STATIC_ROUTES, PROJECT_ROUTES, getAnalyticsRouteContext,
getRedirectRouteId) duplicated what PostHog already captures via $pathname.
Replace redirect_route_id with redirect_to containing the raw path, and remove
~80 lines of route mapping infrastructure.

* clean up analytics events: drop redundant submit events and derived properties

- Remove search_submit events for keywords, domain overview, and backlinks
  (the search_complete events capture the meaningful outcome data)
- Remove target_type from backlinks events (derived 1:1 from search_scope)
- Remove result_limit from keyword research (requested limit, not useful
  alongside actual result_count)
- Remove export_format from data:export events (always "csv")

* refactor: inline wrappers, colocate helpers, deduplicate getActiveOrganizationId

- Inline toVerificationIssueType into verify-email.tsx (single-use wrapper)
- Move mapDataforseoPathToCreditFeature into dataforseoClient.ts (only consumer)
- Extract shared getActiveOrganizationId into lib/auth-session.ts (was
  duplicated in __root.tsx and middleware/ensure-user/hosted.ts)
- Rename shared/analytics.ts → shared/internal-user.ts (only email helpers
  remain after removing route mapping, verification, and dataforseo helpers)

* remove internal user tracking and email domain properties

Drop is_internal_user super property, email_domain person property, and all
supporting code (shared/internal-user.ts, getEmailDomain, isInternalUserEmail).
Simplifies initPostHog and identifyAnalyticsUser signatures.

* remove backlinks:search_complete effect-based tracking

The reactive useEffect + useRef dedup pattern added ~30 lines of plumbing
inside a data hook for a single analytics event. Not worth the complexity.

* simplify: replace manual type guards with zod, deduplicate posthog and sign-out helpers

- Replace hand-rolled typeof checks in getActiveOrganizationId and
  isAuthenticatedServerFunctionContext with zod safeParse
- Extract withPostHogClient helper to deduplicate client posthog wrapper
- Move apiKey guard into getServerPostHogClient factory
- Extract signOutAndRedirect to avoid duplicated sign-out logic
- Drop derivable has_results from analytics events
- Remove unnecessary path normalization in mapDataforseoPathToCreditFeature

* fix: strip email from pageview URLs, restore sign-out guard, harden server posthog, fix path mapper

- Sanitize $current_url on pageviews to remove email query param (PII)
- Restore onSuccess for sign-out redirect to avoid bounce-back on failure
- Swallow shutdown() errors so PostHog outages can't fail billed work
- Rewrite mapDataforseoPathToCreditFeature to match real API path structure
  (path[1] = module, path[3] = endpoint) instead of scanning all segments

* simplify: remove redundant refs in verify-email, infer middleware context type

- Remove unnecessary useRef guards in verify-email effects (deps already prevent re-firing)
- Use z.ZodType<EnsuredUserContext> annotation to infer return type instead of casting
- Add comment explaining one-shot PostHog client on Workers

* fix: reset PostHog identity on sign-out before redirect

* fix: require POSTHOG_HOST env var instead of defaulting to us.i.posthog.com

* fix: annotate url as unknown to satisfy no-unsafe-assignment

* format
2026-04-08 14:09:02 -04:00
Ben Senescu
d773105e8d fix: use backlinks history for default trends (#69)
* fix: use backlinks history for default trends

* simplify backlinks: remove filters, always use history endpoint

Remove the filter UI (status, subdomains, indirect links, exclude internal)
and hardcode defaults across the stack. Replace the conditional
timeseries_summary + timeseries_new_lost_summary fallback with a single
backlinks/history/live call for trend data. This reduces the overview from
5 parallel API calls to 3 and removes all conditional branching.

* improve charts

* fix: refresh backlinks cost docs
2026-04-08 14:09:02 -04:00
Ben Senescu
b15b62cae6 Fix project-route auth gaps for domain and SERP lookups (#65) 2026-04-08 14:09:01 -04:00
Ben Senescu
724a92db0c refactor: move hosted billing to Autumn hooks (#48)
* refactor: move lighthouse audits to dataforseo (#43)

* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.


* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client

---------


* refactor: move hosted billing to Autumn hooks

* fix: satisfy ci checks for hosted billing

* refactor: simplify hosted Autumn billing integration

Limit Autumn wiring to the billing route and remove billing-specific indirection so the hosted billing flow is easier to reason about. Stop sending organization IDs as fake customer names to keep customer identity data honest.

* fix: satisfy ci line-limit check for hosted billing

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:28:28 -04:00
Ben Senescu
638f5a6602 refactor: move lighthouse audits to dataforseo (#43)
* refactor: move lighthouse audits to dataforseo

* chore: remove obsolete audit settings modal

* refactor: rename psi flows to lighthouse

* save

* refactor: simplify audit lighthouse storage flow

* fix: separate lighthouse metrics from actionable audits

* refactor: remove redundant audit project inputs

* feat: redesign lighthouse issues screen with score gauges and table layout

Replace flat score cards with circular SVG gauges, condense metrics into
a compact grid, and switch issue list from cards to an expandable table
with fixed column widths.

* test: harden lighthouse regression coverage

* fix: restore project-scoped audit inputs

* refactor: simplify lighthouse payload handling

* refactor: inline lighthouse server handlers

* refactor: share audit workflow types

* refactor: simplify lighthouse payload flows

* save

* refactor: drop project pagespeed api key

* fix: restore lighthouse issues loading with resilient project context

* fix: restore audit issues back navigation

* refactor: simplify project context and lighthouse error handling

* fix: tolerate DataForSEO lighthouse payload drift

* refactor: route audit lighthouse through dataforseo client
2026-03-26 20:25:27 -04:00
Ben Senescu
739b3f0b6a
Add autumn metered pricing for hosted mode (#44)
* feat: add hosted billing gate and backlinks metering

* refine hosted billing onboarding flow

* fix: align Autumn credit billing flows

* refactor: simplify Autumn billing flow

* refactor: narrow backlinks billing context

* refactor: require billing identity for backlinks profiles

* refactor: colocate backlinks billing flow

* clean

* refactor: centralize DataForSEO billing client

* refactor: simplify dataforseo billing flow

* chore: fix ci check lint issues

* fix: stabilize backlinks chart sizing

Measure chart container width before rendering the backlinks charts so Recharts does not mount at 0x0 inside responsive layouts.

* docs: document hosted DataForSEO metering

Capture the Autumn credit model and require hosted code to go through the metered client path so provider usage is harder to bypass.

* fix: enforce hosted billing access checks

* fix: preserve hosted billing subscription access

* fix: handle hosted billing UI failures
2026-03-25 13:02:12 -04:00
Ben Senescu
4040a854a7
feat: Add better auth (#24)
* refactor: rename delegated auth user table

* feat: scaffold hosted better auth setup

* feat: add hosted auth flows

* refactor: scope project access to organizations

* fix: harden hosted auth entry points

* fix: stabilize org backfills and auth state

* refactor: simplify hosted organization setup

* fix: restore hosted auth signup flow

* fix: preserve hosted workspace access

* fix: preserve hosted auth redirects

* Improve hosted auth UX: auto-redirect to sign-up, hide header on auth pages, add form placeholders, and trust portless dev origins

- Auto-redirect unauthenticated users to /sign-up in hosted mode
- Hide top nav on /sign-in and /sign-up for a cleaner auth experience
- Add input placeholders across sign-in and sign-up forms
- Make name field optional on sign-up (falls back to email username)
- Update copy: remove 'hosted' from user-facing text, rename link to 'Create account'
- Trust *.open-seo.localhost:1355 in dev mode to fix Better Auth origin rejection with portless worktrees

* Simplify hosted auth flow and remove standalone PSI

Use TanStack Form for sign-in and sign-up, make hosted unauthenticated handling redirect-focused, and inline auth route errors. Remove the leftover standalone PSI route, services, and table so PSI only exists within site audits.

* Align project auth with Better Auth organizations

* Make server function auth middleware global

* Reduce auth server function boilerplate

* delete migrations

* fix regenerated migration data backfills

* Simplify hosted auth flow and project audit scoping

* Use active project context for audit actions

* Allow hosted session project updates

* Let agent dev server inherit auth mode

* Match hosted header to gateway account menu

* Scope project session updates to active project

* Inline authenticated server function setup

* Polish header project and account controls

* restore auth generate script

* Use explicit project access in server functions

Make project-scoped server functions take projectId input and enforce ownership through shared middleware instead of session-backed current project state. Document the tradeoffs in an ADR so future changes can follow the same boundary.

* fix ci dependency detection for auth tooling

* Harden project auth in server middleware

Authorize projectId automatically in authenticated server middleware and add a requireProject guard for project-scoped handlers. This makes the auth boundary harder to bypass and removes ad hoc non-null assertions from server functions.

* Inline project id input schemas

Remove tiny shared projectId schema helpers where they were adding indirection without reducing real complexity. Keep project-scoped validation explicit at each server function boundary.

* Skip hosted backlinks access checks

* Simplify auth mode helpers

* Avoid rerunning auth server middleware

* Simplify server function scoping ADR

* Fix backlinks project scoping in hosted auth

* Refine auth route foundations

* Simplify ensure user auth resolution

Split auth-mode context resolvers into focused modules so the middleware reads as request orchestration instead of implementation details. Reuse a shared ensured-user context type across server middleware.

* Simplify hosted organization bootstrap

Use Better Auth to own hosted organization creation and membership so hosted auth only needs to resolve a default active organization. Keep delegated-mode compatibility records isolated in a separate helper.

* Clarify hosted auth and backlinks behavior

Document the hosted AUTH_MODE deploy contract and explain why hosted deployments skip manual backlinks verification. This makes the platform-managed behavior explicit in the code paths that differ from self-serve mode.

* Document hosted org creation callback

Explain why auth.ts injects createOrganization into the hosted org helper. This makes the dependency direction explicit and avoids future import cycles while keeping the helper reusable.

* Fix CI check failures

* Fix nav link prop forwarding

* save
2026-03-19 19:24:34 -04:00
Ben Senescu
8405564925
feat: Add Backlinks page (#23)
* feat: add backlinks setup and overview workflow

* save

* fix backlinks timeseries and refine overview layout

* update backlinks table guidance

* refactor backlinks CI cleanup

* refactor backlinks page state colocation

* fix backlinks access error handling

Differentiate DataForSEO subscription access failures from billing issues and preserve exact page hosts for page-level backlink lookups.

* fix backlinks project scoping and error states

* simplify backlinks billing and lazy loading

* harden backlinks profiling and access flow

* refine backlinks access status typing

* update backlinks scope selection and access handling

* fix backlinks scope fallback and cache scoping

* fix backlinks ci checks

* docs: simplify backlinks pricing

* refine backlinks search: move scope toggle below input, remove redundant description

* refine backlinks summaries and help text

* refine backlinks table sorting

* fix backlinks invalid target handling

* delete docs

* fix backlinks scope inference and domain normalization

* docs: mark backlinks as a supported workflow

* fix backlinks links badge wrapping

* test: split backlinks test suites
2026-03-16 17:48:13 -04:00
Ben Senescu
b9c1061934
chore: clean up knip.jsonc (#20)
* Add website

* chore: clean knip config and remove dead exports
2026-03-11 23:36:54 -04:00
Ben Senescu
75f9868500
fix: Refactor folder structure & services (#17)
* refactor: simplify keyword source selection flow

* fix: improve stacked keyword page layout

* fix: keep SERP tied to searched keyword

Stop silently falling back SERP lookups to related keywords and show a keyword-specific empty message so missing coverage is explicit to users.

* refactor: extract keyword controller and switch SERP to live

Move keyword research orchestration out of the route, call DataForSEO organic live SERP, and harden response normalization with runtime validation for safer typed handling.

* refactor: split keyword research page state into focused hooks

* fix: aggregate auto keyword fallback and reuse cached results

* scope keyword metrics to projectId

* refactor: modularize SEO routes and harden parsing for CI checks

* refactor: adopt zod json codecs at parse boundaries

* refactor: adopt papaparse, tldts, and zod error enums

* refactor: reorganize client and server feature layout

Group keyword UI/controller files and server repositories/services by feature while keeping serverFunctions as the stable API surface. This improves navigation and maintainability without changing runtime behavior.

* refactor: move PSI business logic into service layer

Keep serverFunctions/psi as thin transport handlers by delegating orchestration, export shaping, and source resolution to PsiAuditService. This aligns PSI with the existing service-first backend pattern without changing endpoint behavior.

* refactor: enforce 350/120 size limits across modules

* move self host

* fix: show dynamic range for search trends

Replace the static 'Past 12 months' subtitle with a computed range from the actual plotted trend points so the UI reflects data lag correctly.

* fix: bind audit workflow writes to workflow instance

* chore: make ci checks pass
2026-03-11 23:01:52 -04:00
Ben Senescu
6017fbc4ef
feat: Add DataForSEO key setup guard and onboarding help (#12)
* add DataForSEO key setup guard and onboarding help

Detect missing DataForSEO credentials at app load so users get a clear setup path before running SEO workflows.

* fix api key setup warning visibility and help docs

Make the setup warning fail-safe when status checks error, switch the top notice to DaisyUI alert styling, and add Cloudflare dashboard secret instructions to the help page.

* fix README formatting for CI

* fix seo api key status fallback behavior

* harden DataForSEO setup status checks and modal UX

Gate API key status behind authenticated middleware and improve the setup modal’s keyboard/accessibility behavior, with a clearer fallback notice when status checks fail.

* refine setup UX and local env docs

Hide the API key setup modal on the setup guide route to avoid a self-blocking flow, and clarify README local configuration for DATAFORSEO_API_KEY format.
2026-03-11 15:12:29 -04:00
Ben Senescu
3e489d0771
feat: remove Every App dependency (#8)
* remove Every App SDK and add auth modes for Cloudflare Access and local_noauth

* align local dev auth defaults and normalize Access team domain

* Apply suggestions from code review

* restore local drizzle D1 URL helper

* save

* fix auth error mapping and document self-hosting setup

* Tweak readme

* improve auth config error UI and remove manifest link

* fix team domain config validation and docs anchor
2026-03-10 14:36:16 -04:00
Ben Senescu
35700ca80a
Improve error handling + local setup (#4)
* Add unified Docker self-host stack for OpenSEO and Gateway

* Fix self-host migration flow and env wiring

* Track latest gateway release by default and harden migrations

* Fix gateway static asset serving in self-host runtime

* Run gateway self-host from source with .env-based Vite runtime

* Harden self-host env mode and remove apt dependency from gateway build

* Avoid gateway build ENOSPC by installing deps at runtime

* Revert temporary ENOSPC workaround for gateway Docker setup

* Use Cloudflare selfhost env precedence for Docker runtime

* Add selfhost wrangler environment and tighten Docker docs

* Stabilize selfhost Docker runtime and persistence

Use full Node base images for reliable TLS trust, simplify selfhost startup/env handling, and persist local gateway wrangler state to avoid resets between restarts.

* Simplify Docker selfhost: rename scripts to docker:selfhost:*, remove GATEWAY_RELEASE_REPO, default to latest gateway

* Remove selfhost wrangler env, use .env.local everywhere, simplify docker script names

* Simplify selfhost compose: remove D1 preflight, conditional migration, and healthcheck

* Extract inline Node.js from gateway Dockerfile into standalone scripts

* Save

* Move OpenSEO dependency install from container startup to image build step

* Move self-host docker scripts and compose into dedicated folder

* save

* disable TanStack devtools in Docker self-host

* Make edits

* clarify self-hosting env setup docs

* save

* save

* centralize server-function error middleware and simplify client-safe messaging

* remove local migration-specific error remapping

Keep auth middleware simple by letting DB/schema failures flow through the centralized error handler and remove the unused LOCAL_DB_MIGRATION_REQUIRED client/server code path.

* remove custom error logging in favor of plain console.error

* standardize PSI error codes and sanitize non-Error throws
2026-03-04 16:27:41 -05:00
Ben Senescu
e36abbf19b Initial public release 2026-02-27 14:09:33 -05:00