Ben Senescu 1c74fded7b Site audit P0 (1/3): issue engine, incremental persistence, block detection (#362)
* Site audit P0 (1/3): issue engine, incremental persistence, block detection

Server-side foundation of the P0 feature set from docs/site-audit-pm-research.md:

- Issue engine: shared registry of issue types (severity, explanation,
  how-to-fix). Per-page reporters run inside crawl steps; cross-page checks
  (duplicate titles/descriptions/content, broken internal links, redirect
  chains/loops, orphan pages) run at finalize as SQL over the persisted crawl.
- New audit_links + audit_issues tables, audit_pages columns (depth, content
  hash, header signals, fetch class, sitemap flag); audit tables moved to
  src/db/{,pg/}audit.schema.ts; migrations 0029 (D1) / 0006 (PG).
- Incremental persistence: pages/links/issues written inside each crawl-batch
  step with deterministic row ids + upserts (retry idempotent); slim step
  state; robots.txt checkpointed as step state; merged progress steps keep a
  10k-page crawl within the Workflows step budget.
- Crawler: manual redirect handling with inline follow of normalization-
  equivalent redirects, response header capture (X-Robots-Tag, Link
  rel=canonical), BFS depth, sitemap-last seeding, SSRF check on discovered
  links, honest 'we were blocked' classification (403/429/cf-mitigated/
  challenge).
- MCP: run_site_audit, get_audit_status, get_audit_issues, get_audit_pages;
  limitTier resolved via shared AuditService.resolveAuditLimitTier.
- Lighthouse strategies reduced to auto/none (legacy all/manual map on read).
- Self-healing: getStatus reconciles audits whose workflow instance errored/
  terminated without reaching mark-failed.

The Issues UI and the badseo.dev e2e fixture site stack on top of this PR.

Deploy notes: run db:migrate:prod (additive); terminate running audits before
deploying — the workflow step structure changed and in-flight instances cannot
replay under the new code (a finalize guard fails them loudly instead of
completing empty).

* Store only internal link edges in audit_links

Both consumers (broken-internal-link and orphan checks) filter on
isInternal; per-page external counts already live on audit_pages.
Dropping external rows cuts stored edges on outbound-heavy sites.
Column stays so P1 external-link checks can re-add rows without a
migration.

* Review fixes: failAudit CAS guard, dedupe hash helpers, cheaper checks

- failAudit only transitions running audits, so the getStatus reconciler
  can't flip a just-completed audit to failed when it races finalize
- collapse the duplicate SHA-256 helper into audit/ids.ts
- finalize integrity guard uses a limit-1 existence probe instead of
  fetching every page row
- get_audit_status MCP tool no longer reads the audit row twice when an
  explicit auditId is given
2026-07-07 22:08:14 -04:00
2026-06-04 13:50:26 -04:00
2026-07-05 21:05:01 -04:00
2026-02-27 14:09:33 -05:00
2026-04-08 14:09:01 -04:00
2026-07-02 18:15:46 -04:00
2026-07-05 21:05:01 -04:00
2026-02-27 14:09:33 -05:00
2026-02-27 14:14:23 -05:00
2026-07-05 21:05:01 -04:00
2026-03-11 23:23:47 -04:00

OpenSEO

Open source alternative to Semrush and Ahrefs

OpenSEO is an SEO tool for the people. If tools like Semrush or Ahrefs are too expensive or bloated, OpenSEO is a pay-as-you-go alternative that you actually control.

All-in-one SEO tool for you and your AI agent.

Connect with any agent like Claude Code, OpenClaw or Hermes. We have pre-built skills, but you can build your own to tailor OpenSEO to your needs.

Image

Table of Contents

Hosted Version

If you're not interested in self hosting, or just want to support the project, we also have a hosted version:

openseo.so

Why use OpenSEO?

  • Best in class MCP and AI Skills.
  • Modern, simple UI.
    • Focused workflows instead of a bloated, complex SEO suite.
  • No subscriptions.
    • Bring your own DataForSEO API key and pay only for what you use.
  • Fork and vibe code your own custom tool.

Main SEO Workflows

  • Keyword research
  • Rank tracking
  • Competitor Insights
  • Backlinks
  • Site Audits
  • AI Visibility

Community

Join Discord to chat: Discord

Follow along for updates:

OpenSEO MCP

OpenSEO exposes an MCP server so AI agents can use your SEO data directly.

Connect Claude Code, OpenClaw, Hermes or any other agent.

Setup

  • Set up the app
  • Click "AI & Agents" in the header
  • Follow the instructions to connect to your agent

OpenSEO Agent Skills

OpenSEO Agent Skills are reusable workflows for your agent

They guide your agent through SEO tasks and use the OpenSEO MCP so your agent makes better recommendations.

Available Skills

  • seo-project-setup
  • seo-coach
  • keyword-research
  • keyword-clustering
  • competitive-landscape
  • competitor-analysis
  • link-prospecting

Installation Guide

Read our docs for how to install the skills:

https://openseo.so/docs/skills/setup

Roadmap

Top priorities:

  • Improved and Scheduled Site Audits
  • Custom Reports for Clients
  • Local SEO
  • In App AI Agent

Our top priority is always refining the current product and making existing features better based on user feedback.

If something important is missing, please join the Discord or email me at ben@openseo.so and request it.

Pricing / Costs

OpenSEO is totally free to use. It works by using DataForSEO's APIs, which is a paid third-party service unaffiliated with OpenSEO.

There are two separate things:

  1. OpenSEO app cost: $0, you host it yourself.
  2. DataForSEO API: pay-as-you-go based on usage.

For cost estimates, see DataForSEO API Cost Reference.

DataForSEO API Key Setup

OpenSEO uses DataForSEO to fetch SEO data. You need an API key to connect OpenSEO to the service.

  1. Go to DataForSEO API Access.
  2. Click "Send by email" to get set your credentials.
  3. Copy the longer crendentials labelled "Base64" credentials.
  4. Set this as DATAFORSEO_API_KEY in your environment file:
  • Docker self-hosting: .env
  • Cloudflare: Set it in the workers UI
  • Local development: .env.local

Google Search Console

Search Console is optional and works in self-hosted deployments using your own Google OAuth client. It takes ~10 minutes of one-time setup — see docs/SELF_HOSTING_GOOGLE_SEARCH_CONSOLE.md.

AI Features (SAM)

AI features like SAM, the in-app SEO agent, are optional — set the OPENROUTER_API_KEY environment variable to enable them (create a key at openrouter.ai/settings/keys).

Self-hosting

OpenSEO supports two self-hosting paths:

  • Docker for personal use and testing (Recommended for local use).
  • Cloudflare for internet-facing self-hosting across multiple devices or for your team.

Docker

Docker is recommended for getting started. It's super easy to get up and running once you install Docker.

Cloudflare

If you love OpenSEO and want to use it across multiple devices or with your team, you can host it on Cloudflare which we'll be a SaaS-like experience. Also, this will have automatic database backups and other nice convenience features. It's just a bit more effort to get started if you're unfamiliar with Cloudflare.

Docker Self Hosting

Warning

By default, the Docker version is intended for local use only. It runs in single-user mode with no authentication. For internet-facing self-hosting, use Cloudflare (free plan compatible). Or read docs/SELF_HOSTING_DOCKER.md before exposing to the internet.

Prerequisites:

Quickstart:

  1. cp .env.example .env
  2. Set DATAFORSEO_API_KEY in .env
  3. docker compose up -d
  4. Open http://localhost:<PORT> (default 3001)

To update to the newest published image, pull first and then restart:

docker compose pull
docker compose up -d

For more info, see docs/SELF_HOSTING_DOCKER.md.

Cloudflare Self-Hosting

Deploy the Worker

Clicking this button opens a page to deploy OpenSEO in your Cloudflare account. If you do not have an account yet, it will take you to account creation first (OpenSEO works great on the free plan).

Reference these docs while deploying since the Cloudflare UI doesn't indicate what steps you need to take: docs/SELF_HOSTING_CLOUDFLARE.md.

Deploy to Cloudflare

Local Development

See docs/LOCAL_DEVELOPMENT.md.

Contributing

Contributions are very welcome.

  • Open an issue for bugs, UX friction, or feature requests.
  • Open a PR if you want to implement a feature directly.
  • Community-driven improvements are prioritized, and high-quality PRs are encouraged.

If you want to contribute but are unsure where to start, open an issue and describe what you want to build.

SEO API Cost Reference

Use this section to estimate DataForSEO spend per request type. OpenSEO itself remains free; these are API usage costs only.

As of February 26, 2026, DataForSEOs public docs/pricing pages say:

  • New accounts include $1 free credit to test the API.
  • The minimum top-up/payment is $50.

That means you can try OpenSEO for free with the starter credit, then decide if/when to top up.

Planning examples

  • Track 100 keywords weekly at depth 50: ~$1.20/month
  • 100 keyword research requests at the default 150 results: $3.50
  • 100 keyword research requests at 500 results each: $7.00
  • 100 domain overviews (200 ranked keywords each): $4.01
  • 100 backlinks domain searches at current defaults before opening extra tabs: about $6.34
  • 100 backlinks page searches at current defaults before opening extra tabs: about $4.30
  • 100 fully explored backlinks domain searches: about $10.94
  • 100 fully explored backlinks page searches: about $8.61

Pricing sources

Description
No description provided
Readme MIT 24 MiB
Languages
TypeScript 95.9%
MDX 2.4%
CSS 0.8%
JavaScript 0.5%
HTML 0.2%
Other 0.2%