admin/metatron-open-seo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Site audit P0: Issues tab UI, badseo.dev e2e harness, new checks + pages-table polish (#367)

Browse Source 
* Site audit P0: issue engine, incremental persistence, block detection

Implements the P0 feature set from docs/site-audit-pm-research.md:

- Issue engine: 24 issue types (shared registry with severity,
  explanation, how-to-fix). Per-page reporters run inside crawl steps;
  cross-page checks (duplicate titles/descriptions/content, broken
  internal links, redirect chains/loops, orphan pages) run at finalize
  as SQL over the persisted crawl.
- New audit_links + audit_issues tables, audit_pages columns (depth,
  content hash, header signals, fetch class, sitemap flag); audit
  tables moved to src/db/audit.schema.ts.
- Incremental persistence: pages/links/issues written to D1 inside
  each crawl-batch step with deterministic row ids + upserts (retry
  idempotent); slim step state; robots.txt checkpointed as step state
  for deterministic replay; merged progress steps keep a 10k-page
  crawl within the Workflows step budget.
- Crawler: manual redirect handling with inline follow of
  normalization-equivalent redirects (slash-canonical sites), response
  header capture (X-Robots-Tag, Link rel=canonical), BFS depth,
  sitemap-last seeding, SSRF check on discovered links, honest
  "we were blocked" classification (403/429/cf-mitigated/challenge).
- UI: Issues tab (default) with severity grouping, per-type
  explanations, drill-down, CSV/JSON/Sheets export, blocked banner.
- MCP: run_site_audit, get_audit_status, get_audit_issues (severity-
  sorted, how_to_fix per issue), get_audit_pages.
- Lighthouse strategies reduced to auto/none (legacy all/manual map on
  read); auto stays 10 URLs x 2 = 20 checks.
- Self-healing: getStatus reconciles audits whose workflow instance
  errored/terminated without reaching mark-failed.

Deploy notes: run db:migrate:prod (additive migration 0022); terminate
running audits before deploying - the workflow step structure changed
and in-flight instances cannot replay under the new code (a finalize
guard fails them loudly instead of completing empty).

* feat(onboarding): hide agent chat step; subscribe after intro steps (#312)

* feat(onboarding): hide agent chat step; subscribe after intro steps

Remove the hosted-only strategy-chat diversion from the onboarding
sequence. After the three intro questions, hosted users now hit the
subscribe paywall directly, then return to the GSC and MCP connect
steps. The chat route and components stay in place but unlinked, to be
revisited later. Preserve the post-payment 'You're in!' interstitial by
carrying checkout=success through validateSearch.

* fix(onboarding): set checkout=success from subscribe route, not speculatively

The previous redirect baked checkout=success into the onboarding return
URL at the point needsSubscription is true — i.e. before the user had
paid. It only worked because the subscribe route gates its redirect on
actual access. Move the marker to the subscribe route's redirect-to-app
path, where checkoutCompleted reflects a real returned-from-Stripe
payment, so the 'You're in!' screen can never show pre-payment.

* website: change link

* fix(rank-tracking): unarchive config when re-adding an archived domain (#313)

* Unify dual-backend DB layer (D1 default + Postgres opt-in) (#238)

* D1 → Postgres data migration (ETL + runbook) (#274)

* Fix Postgres-only rank-tracking & site-audit workflow failures (#317)

* rank-tracking: raise per-project config limit from 20 to 100 (#318)

The cap was only a soft guard against runaway scheduled DataForSEO
workload, not a hard product constraint. Bump it to 100 so projects
tracking many domain/location combos aren't blocked.

Co-authored-by: Claude <noreply@anthropic.com>

* fix(db): add missing indexes and drop redundant ones (#319)

Postgres advisor flagged seq-scans and redundant indexes across both
backends (D1 + Postgres):

- add projects(organization_id) — org-scoped project listings seq-scanned
- add account(account_id, provider_id) — better-auth sign-in lookup
- add verification(expires_at) — expired-token cleanup range scan
- drop saved_keyword_tag_assignments_keyword_idx — covered by unique
  (saved_keyword_id, tag_id) prefix
- drop rank_snapshots_run_idx — covered by unique
  (run_id, tracking_keyword_id, device) prefix

Mirrored in both schema dialects + parity-test required-index guard.

* refactor(keywords): unify keyword-metric fetching behind one helper (#320)

* Fix production errors: onboarding crash hardening + DataForSEO spend/noise cleanup (#282)

* fix(ai-search): use valid Claude model_name and fail fast on unknown ones (#323)

DataForSEO dropped the Claude Sonnet 4.0 family from its llm_responses
catalog, so model_name=claude-sonnet-4-0 was rejected with 'Invalid
Field: model_name' while still billing the failed task. Point Claude at
claude-sonnet-4-5 and validate every model_name against DataForSEO's
accepted catalog before dispatching the paid call.

* fix(mcp): 405 the standalone GET SSE stream to stop /mcp OOM (#325)

The stateless MCP server returns JSON on POST (enableJsonResponse) and
pushes no server-initiated messages, so the optional standalone GET SSE
stream serves no purpose. Left enabled, each GET holds an SSE stream open
indefinitely (25s keepalive, no eventStore) and pins a fresh per-request
McpServer (~5MB of tools + Zod schemas); a few dozen concurrent connected
clients exceed the 128MB isolate limit. This was 100% of the /mcp
exceededMemory OOMs (GET only; POST never OOMed).

Return 405 (spec-compliant 'no standalone stream') before building the
server, so GET allocates nothing. Also removes the bulk of the elevated
GET canceled / responseStreamDisconnected outcomes.

* Re-add free plan as the floor; remove subscribe gate (#321)

* Pin production to Postgres via committed Hyperdrive binding (#329)

* Add Cloudflare Turnstile captcha on email signup (#326)

* Triage production log errors: audit crash, Autumn webhook FK, PostHog capture, auth rate-limit IP, log noise (#327)

* Add badseo.dev: a test site of deliberate SEO mistakes

An open-source Cloudflare Worker that serves ~27 pages, each breaking one
common technical-SEO rule (missing title, redirect loop, orphan page, thin
content, and so on). It doubles as the end-to-end fixture for the OpenSEO
site audit: every page declares the audit issues it should trigger, and
scripts/run-audit.ts drives the real audit engine against a running copy to
check that it does (36/36 checks, 25/25 issue types).

Styled to match the OpenSEO marketing site (web/). Maintained-by-OpenSEO
badge links back to openseo.so.

* badseo.dev: logo in pill, footer/hover polish, SEO-optimized titles

- Use the OpenSEO pine-tree logo (downscaled, base64-embedded, served at
  /openseo-logo.png) in a light chip inside the badge, replacing the ◎ glyph.
- Footer band now fills to the bottom of the page (dropped the mismatched
  body padding strip) with room for the floating badge.
- Index rows: remove the stray full-row underline and the stark white hover
  box; hover is now a soft cream tint with the name underlined.
- Drop the "Maintained by OpenSEO" hero eyebrow; new H1 "A website
  demonstrating common technical SEO problems" and a cleaner subtitle.
- Optimize homepage + catalog <title>/meta around real keywords from OpenSEO
  keyword research (technical seo issues KD25/vol170; technical seo checklist
  KD16/vol390), keeping meta lengths within limits.

* Site audit P0 (1/3): issue engine, incremental persistence, block detection

Server-side foundation of the P0 feature set from docs/site-audit-pm-research.md:

- Issue engine: shared registry of issue types (severity, explanation,
  how-to-fix). Per-page reporters run inside crawl steps; cross-page checks
  (duplicate titles/descriptions/content, broken internal links, redirect
  chains/loops, orphan pages) run at finalize as SQL over the persisted crawl.
- New audit_links + audit_issues tables, audit_pages columns (depth, content
  hash, header signals, fetch class, sitemap flag); audit tables moved to
  src/db/{,pg/}audit.schema.ts; migrations 0029 (D1) / 0006 (PG).
- Incremental persistence: pages/links/issues written inside each crawl-batch
  step with deterministic row ids + upserts (retry idempotent); slim step
  state; robots.txt checkpointed as step state; merged progress steps keep a
  10k-page crawl within the Workflows step budget.
- Crawler: manual redirect handling with inline follow of normalization-
  equivalent redirects, response header capture (X-Robots-Tag, Link
  rel=canonical), BFS depth, sitemap-last seeding, SSRF check on discovered
  links, honest 'we were blocked' classification (403/429/cf-mitigated/
  challenge).
- MCP: run_site_audit, get_audit_status, get_audit_issues, get_audit_pages;
  limitTier resolved via shared AuditService.resolveAuditLimitTier.
- Lighthouse strategies reduced to auto/none (legacy all/manual map on read).
- Self-healing: getStatus reconciles audits whose workflow instance errored/
  terminated without reaching mark-failed.

The Issues UI and the badseo.dev e2e fixture site stack on top of this PR.

Deploy notes: run db:migrate:prod (additive); terminate running audits before
deploying — the workflow step structure changed and in-flight instances cannot
replay under the new code (a finalize guard fails them loudly instead of
completing empty).

* Site audit P0 (2/3): Issues tab UI

- Issues tab (new default) with severity grouping, per-type explanations and
  how-to-fix, drill-down to affected pages, CSV/JSON/Sheets export, and the
  'we were blocked' banner when the crawl was challenged.
- Tabs always render (Issues/Pages, Performance when Lighthouse ran);
  audit route search schema gains the issues tab and defaults to it.

Stacks on claude/audit-p0-server (issue engine + persistence).

* badseo.dev: render the badge logo as a white tree, no chip

The silver source logo was invisible on the dark pill, so it sat in a white
chip. Render it white via a CSS filter instead, so the tree fills the pill
with no backing background.

* badseo.dev: add build (typecheck) step before deploy

- Add 'build'/'typecheck' scripts (tsc --noEmit); 'deploy' now runs the build
  before wrangler deploy.
- Scope the tsconfig typecheck to the Worker source (src/); the e2e harness in
  scripts/ imports the main app and is run with tsx from the repo root.
- Document the deploy flow and first-time custom-domain setup in the README.

* badseo.dev: add trailing-slash redirect-cycle fixture + regression test

Reproduces the 508 "Loop Detected" class of bug from every-app/open-seo#61: a
CMS-style page whose canonical URL ends in a trailing slash, with the non-slash
form 301-redirecting to it. A crawler that strips trailing slashes turns the
canonical /foo/ back into /foo, follows the 301 to /foo/, strips it again, and
loops.

- New fixture at /redirect/trailing-slash: the non-slash form (intercepted in
  index.ts on the raw path) 301s to the slash form, which is served as the
  canonical 200.
- Harness asserts the page is crawled exactly once as a 200 with NO redirect
  loop, plus a dedicated "Trailing-slash cycle -> 200, no loop" guard.

Verified the guard bites: temporarily disabling crawlPage's slash-canonical
inline-follow makes both checks fail (redirect-loop, status 301); with it in
place the harness is 38/38, 25/25 issue types.

* Add webapp-testing skill (installed via /reload-skills)

Vendors the anthropics/skills webapp-testing toolkit: real files under
.agents/skills/webapp-testing, a symlink from .claude/skills/, and skills-lock.json
pinning the source + hash. Matches how the other project skills are tracked.

* Site audit: redesign issues tab as grouped table + calmer page header

- Issues: single bordered table with severity sections (Critical/Warning/Info
  headers carry the counts), dot indicators instead of filled pills, plain
  right-aligned page counts, all rows collapsed by default; expanded rows get
  a severity-colored left rule
- Removed the dead severity-count chips (they looked like filters but were
  inert spans)
- Header: audited hostname is now the H1 with the status badge inline
- Blocked banner: compact tinted panel instead of a full-size alert
- Stats: hairline strip instead of four separate cards; issues stat shows a
  severity breakdown, Lighthouse tile hidden when no tests ran, dropped the
  orange issues-count coloring

* audit: fix trailing-slash redirect cycle at the root (preserve slashes)

Replaces the crawlPage inline-follow workaround with the root-cause fix, so we
don't carry two fixes for the same bug (every-app/open-seo#61).

- normalizeUrl: stop stripping trailing slashes. A trailing slash is the
  canonical form on most CMSes, which 301 the non-slash version to it. Stripping
  rewrote the canonical URL into its own redirect source and looped (508). Now
  /path and /path/ are distinct and the redirect resolves normally.
- crawlPage: remove the isSelfAfterNormalization inline-follow (+ now-unused
  resolveRawUrl). With slashes preserved it's dead code; a trailing-slash
  redirect is recorded as an ordinary hop.
- add canonicalUrlKey (www/http/https-tolerant) and use it for the Lighthouse
  homepage match, which had the same redirect-mismatch vulnerability.
- tests: preserve-trailing-slash + canonicalUrlKey unit tests; badseo harness
  guard is now fix-agnostic (canonical resolves to 200, no loop/error).

Verified: 36 audit unit tests pass, tsc clean, badseo e2e 38/38. Reintroducing
stripping makes the trailing-slash guard fail (redirect-loop), confirming the
regression guard bites.

* Audit: add no-outgoing-links + meta-description-too-short checks, catch empty H1s

Two checks Ahrefs covers that we didn't, plus a fix: <h1></h1> now counts
as missing. badseo.dev gains fixtures for all three (41 checks, 27/27
issue types covered).

* Audit pages table: honest redirect/non-HTML rows, wrapped titles

- 3xx rows show their redirect target (dim →) instead of a red 'missing'
  title, and dash out H1/Words/Images since nothing was analyzed
- red 'missing' only when the engine actually flagged missing-title, so
  200 non-HTML files (security.txt) read as blank, not broken
- URL cells include the host when it differs from the audited site's, so
  apex→www redirect sources no longer render identically to their target
- titles wrap to two lines (line-clamp) in a wider column instead of
  truncating at 220px; PagesTable moved to its own file (lint max-lines)

* Audit pages table: canonical-host display, URL default sort, full title wrap

- host prefix now compares against the site's predominant 2xx host, not
  the typed start URL — auditing apex 12port.com no longer prefixes every
  www row with the host
- default sort by URL so the table opens as a site inventory instead of
  leading with redirects on error-free sites
- titles wrap fully instead of clamping at two lines; long titles are the
  thing being audited, so their tails shouldn't be hidden

* ci: exclude vendored skills from prettier; format test file

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Ben Senescu 2026-07-07 12:47:10 -04:00 committed by Ben Senescu
parent 1c74fded7b
commit 67265a0046
48 changed files with 4170 additions and 300 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

202
.agents/skills/webapp-testing/LICENSE.txt Normal file
View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Anthropic, PBC.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

96
.agents/skills/webapp-testing/SKILL.md Normal file
View File

@ -0,0 +1,96 @@
---
name: webapp-testing
description: Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
license: Complete terms in LICENSE.txt
---
# Web Application Testing
To test local web applications, write native Python Playwright scripts.
**Helper Scripts Available**:
- `scripts/with_server.py` - Manages server lifecycle (supports multiple servers)
**Always run scripts with `--help` first** to see usage. DO NOT read the source until you try running the script first and find that a customized solution is abslutely necessary. These scripts can be very large and thus pollute your context window. They exist to be called directly as black-box scripts rather than ingested into your context window.
## Decision Tree: Choosing Your Approach
```
User task → Is it static HTML?
├─ Yes → Read HTML file directly to identify selectors
│ ├─ Success → Write Playwright script using selectors
│ └─ Fails/Incomplete → Treat as dynamic (below)
│
└─ No (dynamic webapp) → Is the server already running?
├─ No → Run: python scripts/with_server.py --help
│ Then use the helper + write simplified Playwright script
│
└─ Yes → Reconnaissance-then-action:
1. Navigate and wait for networkidle
2. Take screenshot or inspect DOM
3. Identify selectors from rendered state
4. Execute actions with discovered selectors
```
## Example: Using with_server.py
To start a server, run `--help` first, then use the helper:
**Single server:**
```bash
python scripts/with_server.py --server "npm run dev" --port 5173 -- python your_automation.py
```
**Multiple servers (e.g., backend + frontend):**
```bash
python scripts/with_server.py \
--server "cd backend && python server.py" --port 3000 \
--server "cd frontend && npm run dev" --port 5173 \
-- python your_automation.py
```
To create an automation script, include only Playwright logic (servers are managed automatically):
```python
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(headless=True) # Always launch chromium in headless mode
page = browser.new_page()
page.goto('http://localhost:5173') # Server already running and ready
page.wait_for_load_state('networkidle') # CRITICAL: Wait for JS to execute
# ... your automation logic
browser.close()
```
## Reconnaissance-Then-Action Pattern
1. **Inspect rendered DOM**:
```python
page.screenshot(path='/tmp/inspect.png', full_page=True)
content = page.content()
page.locator('button').all()
```
2. **Identify selectors** from inspection results
3. **Execute actions** using discovered selectors
## Common Pitfall
❌ **Don't** inspect the DOM before waiting for `networkidle` on dynamic apps
✅ **Do** wait for `page.wait_for_load_state('networkidle')` before inspection
## Best Practices
- **Use bundled scripts as black boxes** - To accomplish a task, consider whether one of the scripts available in `scripts/` can help. These scripts handle common, complex workflows reliably without cluttering the context window. Use `--help` to see usage, then invoke directly.
- Use `sync_playwright()` for synchronous scripts
- Always close the browser when done
- Use descriptive selectors: `text=`, `role=`, CSS selectors, or IDs
- Add appropriate waits: `page.wait_for_selector()` or `page.wait_for_timeout()`
## Reference Files
- **examples/** - Examples showing common patterns:
- `element_discovery.py` - Discovering buttons, links, and inputs on a page
- `static_html_automation.py` - Using file:// URLs for local HTML
- `console_logging.py` - Capturing console logs during automation

35
.agents/skills/webapp-testing/examples/console_logging.py Normal file
View File

@ -0,0 +1,35 @@
from playwright.sync_api import sync_playwright
# Example: Capturing console logs during browser automation
url = 'http://localhost:5173' # Replace with your URL
console_logs = []
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page(viewport={'width': 1920, 'height': 1080})
# Set up console log capture
def handle_console_message(msg):
console_logs.append(f"[{msg.type}] {msg.text}")
print(f"Console: [{msg.type}] {msg.text}")
page.on("console", handle_console_message)
# Navigate to page
page.goto(url)
page.wait_for_load_state('networkidle')
# Interact with the page (triggers console logs)
page.click('text=Dashboard')
page.wait_for_timeout(1000)
browser.close()
# Save console logs to file
with open('/mnt/user-data/outputs/console.log', 'w') as f:
f.write('\n'.join(console_logs))
print(f"\nCaptured {len(console_logs)} console messages")
print(f"Logs saved to: /mnt/user-data/outputs/console.log")

40
.agents/skills/webapp-testing/examples/element_discovery.py Normal file
View File

@ -0,0 +1,40 @@
from playwright.sync_api import sync_playwright
# Example: Discovering buttons and other elements on a page
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page()
# Navigate to page and wait for it to fully load
page.goto('http://localhost:5173')
page.wait_for_load_state('networkidle')
# Discover all buttons on the page
buttons = page.locator('button').all()
print(f"Found {len(buttons)} buttons:")
for i, button in enumerate(buttons):
text = button.inner_text() if button.is_visible() else "[hidden]"
print(f" [{i}] {text}")
# Discover links
links = page.locator('a[href]').all()
print(f"\nFound {len(links)} links:")
for link in links[:5]: # Show first 5
text = link.inner_text().strip()
href = link.get_attribute('href')
print(f" - {text} -> {href}")
# Discover input fields
inputs = page.locator('input, textarea, select').all()
print(f"\nFound {len(inputs)} input fields:")
for input_elem in inputs:
name = input_elem.get_attribute('name') or input_elem.get_attribute('id') or "[unnamed]"
input_type = input_elem.get_attribute('type') or 'text'
print(f" - {name} ({input_type})")
# Take screenshot for visual reference
page.screenshot(path='/tmp/page_discovery.png', full_page=True)
print("\nScreenshot saved to /tmp/page_discovery.png")
browser.close()

33
.agents/skills/webapp-testing/examples/static_html_automation.py Normal file
View File

@ -0,0 +1,33 @@
from playwright.sync_api import sync_playwright
import os
# Example: Automating interaction with static HTML files using file:// URLs
html_file_path = os.path.abspath('path/to/your/file.html')
file_url = f'file://{html_file_path}'
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page(viewport={'width': 1920, 'height': 1080})
# Navigate to local HTML file
page.goto(file_url)
# Take screenshot
page.screenshot(path='/mnt/user-data/outputs/static_page.png', full_page=True)
# Interact with elements
page.click('text=Click Me')
page.fill('#name', 'John Doe')
page.fill('#email', 'john@example.com')
# Submit form
page.click('button[type="submit"]')
page.wait_for_timeout(500)
# Take final screenshot
page.screenshot(path='/mnt/user-data/outputs/after_submit.png', full_page=True)
browser.close()
print("Static HTML automation completed!")

106
.agents/skills/webapp-testing/scripts/with_server.py Executable file
View File

@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""
Start one or more servers, wait for them to be ready, run a command, then clean up.
Usage:
# Single server
python scripts/with_server.py --server "npm run dev" --port 5173 -- python automation.py
python scripts/with_server.py --server "npm start" --port 3000 -- python test.py
# Multiple servers
python scripts/with_server.py \
--server "cd backend && python server.py" --port 3000 \
--server "cd frontend && npm run dev" --port 5173 \
-- python test.py
"""
import subprocess
import socket
import time
import sys
import argparse
def is_server_ready(port, timeout=30):
"""Wait for server to be ready by polling the port."""
start_time = time.time()
while time.time() - start_time < timeout:
try:
with socket.create_connection(('localhost', port), timeout=1):
return True
except (socket.error, ConnectionRefusedError):
time.sleep(0.5)
return False
def main():
parser = argparse.ArgumentParser(description='Run command with one or more servers')
parser.add_argument('--server', action='append', dest='servers', required=True, help='Server command (can be repeated)')
parser.add_argument('--port', action='append', dest='ports', type=int, required=True, help='Port for each server (must match --server count)')
parser.add_argument('--timeout', type=int, default=30, help='Timeout in seconds per server (default: 30)')
parser.add_argument('command', nargs=argparse.REMAINDER, help='Command to run after server(s) ready')
args = parser.parse_args()
# Remove the '--' separator if present
if args.command and args.command[0] == '--':
args.command = args.command[1:]
if not args.command:
print("Error: No command specified to run")
sys.exit(1)
# Parse server configurations
if len(args.servers) != len(args.ports):
print("Error: Number of --server and --port arguments must match")
sys.exit(1)
servers = []
for cmd, port in zip(args.servers, args.ports):
servers.append({'cmd': cmd, 'port': port})
server_processes = []
try:
# Start all servers
for i, server in enumerate(servers):
print(f"Starting server {i+1}/{len(servers)}: {server['cmd']}")
# Use shell=True to support commands with cd and &&
process = subprocess.Popen(
server['cmd'],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
server_processes.append(process)
# Wait for this server to be ready
print(f"Waiting for server on port {server['port']}...")
if not is_server_ready(server['port'], timeout=args.timeout):
raise RuntimeError(f"Server failed to start on port {server['port']} within {args.timeout}s")
print(f"Server ready on port {server['port']}")
print(f"\nAll {len(servers)} server(s) ready")
# Run the command
print(f"Running: {' '.join(args.command)}\n")
result = subprocess.run(args.command)
sys.exit(result.returncode)
finally:
# Clean up all servers
print(f"\nStopping {len(server_processes)} server(s)...")
for i, process in enumerate(server_processes):
try:
process.terminate()
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
print(f"Server {i+1} stopped")
print("All servers stopped")
if __name__ == '__main__':
main()

1
.claude/skills/webapp-testing Symbolic link
View File

@ -0,0 +1 @@
../../.agents/skills/webapp-testing

2
.prettierignore
View File

@ -4,6 +4,8 @@ pnpm-lock.yaml
routeTree.gen.ts routeTree.gen.ts
.opencode/package.json .opencode/package.json
# vendored skills are hash-pinned in skills-lock.json; don't reformat them
.agents/skills/
dist/ dist/
drizzle/ drizzle/
drizzle-pg/ drizzle-pg/

5
badseo/.gitignore vendored Normal file
View File

@ -0,0 +1,5 @@
node_modules
dist
.wrangler
.dev.vars
*.log

132
badseo/README.md Normal file
View File

@ -0,0 +1,132 @@
# badseo.dev
**A test site full of SEO mistakes.**
badseo.dev is a set of open-source web pages. Each page breaks one common
technical-SEO rule: a missing `<title>`, a redirect loop, a page nothing links
to, thin content. Point an SEO crawler at it and check what the crawler catches.
It is also the end-to-end test fixture for the
[OpenSEO](https://openseo.so) site audit. Every page lists the audit issues it
should trigger, and a harness runs the real audit engine against a running copy
to check that it does.
Maintained by the team behind [OpenSEO](https://openseo.so), an open-source SEO
tool.
---
## What's covered
Every issue type in the OpenSEO audit engine is exercised by at least one page
(the harness enforces this). Pages are grouped by category:
| Category | Pages |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------- |
| **Head tags & headings** | missing title, title too long/short, missing meta, meta too long, missing H1, multiple H1, heading-level skip |
| **Content quality** | thin content, images missing alt, duplicate content, duplicate title, duplicate meta description |
| **Indexability & canonical** | noindex (meta + `X-Robots-Tag` header), canonicalized to another URL, conflicting canonicals |
| **HTTP status & links** | 404, 500, 403 (blocked), broken internal link |
| **Redirects** | redirect chain, redirect loop, trailing-slash canonical (redirect-cycle trap) |
| **Performance** | slow server response (TTFB) |
| **Site structure** | orphan page, deep click-path |
| **Kitchen sink** | one page that breaks six ways at once |
Browse them all at `/catalog`.
## How it's built
A single, dependency-free Cloudflare Worker (`src/index.ts`) that serves
hand-authored HTML with byte-level control over every SEO signal — status codes,
redirects, response headers (`X-Robots-Tag`, `Link: …; rel=canonical`), response
timing, and the raw `<head>`. No framework: SSR machinery tends to _fix_ the very
things we're trying to break (it insists on injecting a `<title>`, etc.).
- `src/index.ts` — router: fixtures → URLs, plus `robots.txt` and `sitemap.xml`.
- `src/lib.ts` — page rendering. The shared chrome (nav, footer, the OpenSEO
badge, the "what this page tests" panel) is deliberately **SEO-neutral**: it
emits no `<h1>`–`<h6>` and no `<img>`, so each fixture fully controls its own
headings and images and the audit measures exactly the defect we injected.
- `src/fixtures/*.ts` — the fixtures, one file per category.
- `src/pages.ts` — the homepage and catalog (both must audit clean).
## Run it locally
```bash
# from the badseo/ directory (uses the repo's wrangler)
npx wrangler dev # serves on http://localhost:8787
```
## Run the end-to-end audit
The harness drives the **real** OpenSEO crawl + issue-detection functions
(imported straight from `../src`) against a running badseo.dev, then asserts every
fixture triggers exactly the issues it declares — and that the homepage, catalog,
and support pages come back clean.
```bash
# with `wrangler dev` running in another terminal:
npx tsx scripts/run-audit.ts http://localhost:8787
```
It prints a per-page pass/fail matrix and an issue-type coverage line, and exits
non-zero on any mismatch — so it works as a CI gate for the audit engine.
## Add a fixture
Contributions are welcome — a new fixture _is_ a new regression test. Each is a
small object:
```ts
const myFixture: Fixture = {
path: "/category/my-mistake",
category: "Content quality",
name: "My SEO mistake",
summary: "One-line description shown in the on-page test panel.",
lesson: "Why it matters / how to fix it.",
expectedIssues: ["thin-content"], // the audit issue ids this page must trigger
handler: () =>
htmlResponse(
renderPage({
fixture: myFixture,
title: "…",
metaDescription: "…",
bodyHtml: "…",
}),
),
};
```
Then add it to its category's exported array. `expectedIssues` is type-checked
against the real audit registry, and the harness will hold you to it.
Guidelines:
- **Isolate one issue per page.** A themed page should be healthy in every way
_except_ the defect it demonstrates, so the audit result is unambiguous. (The
kitchen-sink page is the deliberate exception.)
- **Keep titles and meta descriptions unique** across the site, or you'll create
accidental duplicate-title / duplicate-meta groups. The exceptions are the
intentional duplicate pairs.
- **Keep the copy plain.** Say what the page does and why the mistake matters.
No hype.
## Deploy
There's no bundling build — wrangler/esbuild bundles `src/index.ts` on deploy.
The `build` script is a typecheck (`tsc --noEmit`) that runs before the deploy:
```bash
npm run build # typecheck the Worker source
npm run deploy # build, then wrangler deploy → badseo.dev
```
`npm run deploy` runs `npm run build && wrangler deploy --env production`. To
deploy without the typecheck gate, run `npx wrangler deploy --env production`
directly.
First-time setup: the `production` env in `wrangler.jsonc` binds the custom
domains `badseo.dev` and `www.badseo.dev`, so the zone must be on the Cloudflare
account before the first deploy. (The routes live under `production` so that
plain `wrangler dev` still serves on localhost.) To preview on a `*.workers.dev`
URL without the custom domain, deploy the top-level env: `npx wrangler deploy`.

21
badseo/package.json Normal file
View File

@ -0,0 +1,21 @@
{
"name": "badseo",
"private": true,
"type": "module",
"description": "badseo.dev — a deliberately broken website full of SEO mistakes, used as e2e test fixtures for the OpenSEO site audit.",
"scripts": {
"dev": "wrangler dev",
"start": "wrangler dev",
"typecheck": "tsc --noEmit",
"build": "tsc --noEmit",
"deploy": "npm run build && wrangler deploy --env production",
"audit": "tsx scripts/run-audit.ts",
"test:e2e": "tsx scripts/run-audit.ts"
},
"devDependencies": {
"@cloudflare/workers-types": "^4.20250109.0",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"wrangler": "^4.67.0"
}
}

BIN
badseo/public/openseo-logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.1 KiB

419
badseo/scripts/run-audit.ts Normal file
View File

@ -0,0 +1,419 @@
/**
* End-to-end audit harness for badseo.dev.
*
* This drives the REAL OpenSEO audit engine (the same crawl + issue-detection
* functions the production Worker uses) against a running badseo.dev, then
* checks that every fixture triggers exactly the audit issues it declares.
*
* It reimplements only the crawl *frontier* loop — deliberately, so it can
* crawl localhost (the production frontier's SSRF policy blocks private hosts).
* Every actual detection call below is imported straight from ../src.
*
* pnpm --filter badseo audit # against http://localhost:8787
* tsx scripts/run-audit.ts http://host:port # against any origin
*/
import { crawlPage } from "../../src/server/workflows/site-audit-workflow-helpers";
import {
discoverUrls,
parseRobotsTxt,
} from "../../src/server/lib/audit/discovery";
import {
normalizeUrl,
isSameOrigin,
} from "../../src/server/lib/audit/url-utils";
import { runPageReporters } from "../../src/server/lib/audit/issues/page-reporters";
import {
findDuplicates,
findRedirectChainsAndLoops,
type SlimPage,
} from "../../src/server/lib/audit/issues/multipage-checks";
import type { DetectedIssue } from "../../src/server/lib/audit/issues/page-reporters";
import type { CrawledPageResult } from "../../src/server/lib/audit/types";
import { AUDIT_ISSUE_TYPES } from "../../src/shared/audit-issues";
import { allFixtures, fixturePaths } from "../src/fixtures/registry";
import { TRAILING_SLASH_CANONICAL } from "../src/fixtures/redirects";
import type { Fixture, IssueId } from "../src/fixtures/types";
const BASE = (process.argv[2] ?? "http://localhost:8787").replace(/\/$/, "");
const MAX_PAGES = 200;
const CONCURRENCY = 10;
interface CrawlLink {
sourceId: string;
sourceUrl: string;
targetUrl: string;
}
async function warmup(): Promise<void> {
// Prime the dev server so healthy pages don't read as slow on a cold start.
const paths = new Set<string>(["/", "/catalog"]);
for (const f of allFixtures) for (const p of fixturePaths(f)) paths.add(p);
await Promise.all(
[...paths].map((p) =>
fetch(`${BASE}${p}`, {
redirect: "manual",
signal: AbortSignal.timeout(8000),
}).catch(() => {}),
),
);
}
interface CrawlEntry {
url: string;
depth: number | null;
}
/** BFS crawl frontier: link-discovered URLs first, sitemap-only URLs last. */
async function crawl(origin: string): Promise<{
pages: CrawledPageResult[];
links: CrawlLink[];
completed: boolean;
}> {
const { urls: sitemapUrls, robotsText } = await discoverUrls(
origin,
MAX_PAGES,
);
const robots = parseRobotsTxt(origin, robotsText);
const sitemapSet = new Set(
sitemapUrls.map((u) => normalizeUrl(u)).filter((u): u is string => !!u),
);
const visited = new Set<string>();
const queued = new Set<string>();
const linkQueue: CrawlEntry[] = [];
const sitemapQueue: CrawlEntry[] = [];
const start = normalizeUrl(`${origin}/`) ?? `${origin}/`;
linkQueue.push({ url: start, depth: 0 });
queued.add(start);
for (const u of sitemapSet) {
if (u === start || queued.has(u)) continue;
sitemapQueue.push({ url: u, depth: null });
queued.add(u);
}
const pages: CrawledPageResult[] = [];
const links: CrawlLink[] = [];
const enqueue = (url: string, depth: number | null) => {
const n = normalizeUrl(url);
if (!n) return;
if (!isSameOrigin(n, origin)) return;
if (visited.has(n) || queued.has(n)) return;
if (!robots.isAllowed(n)) return;
linkQueue.push({ url: n, depth });
queued.add(n);
};
while (
(linkQueue.length > 0 || sitemapQueue.length > 0) &&
pages.length < MAX_PAGES
) {
const batch: CrawlEntry[] = [];
while (
(linkQueue.length > 0 || sitemapQueue.length > 0) &&
batch.length < CONCURRENCY &&
pages.length + batch.length < MAX_PAGES
) {
const entry = (linkQueue.length > 0 ? linkQueue : sitemapQueue).shift()!;
queued.delete(entry.url);
if (visited.has(entry.url)) continue;
visited.add(entry.url);
batch.push(entry);
}
const crawled = await Promise.all(
batch.map((e) => crawlPage(e.url, e.depth, sitemapSet.has(e.url))),
);
for (let i = 0; i < crawled.length; i++) {
const page = crawled[i];
const depth = batch[i].depth;
pages.push(page);
for (const link of page.links) {
if (link.isInternal) {
links.push({
sourceId: page.id,
sourceUrl: page.url,
targetUrl: link.targetUrl,
});
enqueue(link.targetUrl, depth === null ? null : depth + 1);
}
}
if (page.redirectUrl) enqueue(page.redirectUrl, depth);
}
}
const completed = linkQueue.length === 0 && sitemapQueue.length === 0;
return { pages, links, completed };
}
/** In-memory equivalents of the two D1-backed multipage checks. */
function findBrokenInternalLinks(
pages: CrawledPageResult[],
links: CrawlLink[],
): DetectedIssue[] {
const byUrl = new Map(pages.map((p) => [p.url, p]));
const issues: DetectedIssue[] = [];
const seen = new Set<string>();
for (const link of links) {
const target = byUrl.get(link.targetUrl);
if (!target) continue;
if (target.fetchClass !== "ok" || target.statusCode < 400) continue;
const key = `${link.sourceUrl}::${link.targetUrl}`;
if (seen.has(key)) continue;
seen.add(key);
issues.push({
issueType: "broken-internal-link",
pageId: link.sourceId,
pageUrl: link.sourceUrl,
dedupeKey: link.targetUrl,
details: { targetUrl: link.targetUrl, targetStatus: target.statusCode },
});
}
return issues;
}
function findOrphanPages(
pages: CrawledPageResult[],
links: CrawlLink[],
startUrl: string,
): DetectedIssue[] {
const hasInlink = new Set<string>();
for (const link of links) {
const source = pages.find((p) => p.id === link.sourceId);
// Self-links don't make a page reachable.
if (source && source.url === link.targetUrl) continue;
hasInlink.add(link.targetUrl);
}
const redirectTargets = new Set(
pages.map((p) => p.redirectUrl).filter((u): u is string => !!u),
);
const issues: DetectedIssue[] = [];
for (const page of pages) {
if (page.fetchClass !== "ok") continue;
if (page.statusCode < 200 || page.statusCode >= 300) continue;
if (page.url === startUrl) continue;
if (hasInlink.has(page.url) || redirectTargets.has(page.url)) continue;
issues.push({
issueType: "orphan-page",
pageId: page.id,
pageUrl: page.url,
});
}
return issues;
}
function toSlim(page: CrawledPageResult): SlimPage {
return {
id: page.id,
url: page.url,
statusCode: page.statusCode,
fetchClass: page.fetchClass,
title: page.title || null,
metaDescription: page.metaDescription || null,
contentHash: page.contentHash,
redirectUrl: page.redirectUrl,
wordCount: page.wordCount,
isIndexable: page.isIndexable,
canonicalUrl: page.canonicalUrl,
headerCanonicalUrl: page.headerCanonicalUrl,
};
}
// ---- ANSI helpers -------------------------------------------------------
const c = {
red: (s: string) => `\x1b[31m${s}\x1b[0m`,
green: (s: string) => `\x1b[32m${s}\x1b[0m`,
yellow: (s: string) => `\x1b[33m${s}\x1b[0m`,
dim: (s: string) => `\x1b[2m${s}\x1b[0m`,
bold: (s: string) => `\x1b[1m${s}\x1b[0m`,
};
const INFO_ISSUES = new Set<IssueId>(
(Object.keys(AUDIT_ISSUE_TYPES) as IssueId[]).filter(
(id) => AUDIT_ISSUE_TYPES[id].severity === "info",
),
);
async function main() {
console.log(c.bold(`\n🧪 badseo.dev audit harness → ${BASE}\n`));
await warmup();
const origin = new URL(BASE).origin;
const startUrl = normalizeUrl(`${origin}/`) ?? `${origin}/`;
const { pages, links, completed } = await crawl(origin);
if (process.env.DEBUG_DEPTH) {
for (const p of pages) {
if (p.url.includes("/structure/deep/") || p.url.endsWith("/catalog")) {
console.log(` depth ${p.crawlDepth} ${p.url}`);
}
}
}
// Assign deterministic-ish ids already handled by crawlPage; run detection.
const detected: DetectedIssue[] = [];
for (const page of pages) detected.push(...runPageReporters(page));
const slim = pages.map(toSlim);
detected.push(...findDuplicates(slim));
detected.push(...findRedirectChainsAndLoops(slim));
detected.push(...findBrokenInternalLinks(pages, links));
if (completed) detected.push(...findOrphanPages(pages, links, startUrl));
const byUrl = new Map<string, Set<IssueId>>();
for (const issue of detected) {
const set = byUrl.get(issue.pageUrl) ?? new Set<IssueId>();
set.add(issue.issueType);
byUrl.set(issue.pageUrl, set);
}
const crawledUrls = new Set(pages.map((p) => p.url));
console.log(
c.dim(
`crawled ${pages.length} pages, ${links.length} internal links, ${detected.length} issues, crawl ${
completed ? "completed" : "TRUNCATED"
}\n`,
),
);
interface Row {
name: string;
url: string;
ok: boolean;
detail: string;
}
const rows: Row[] = [];
let failures = 0;
const check = (
name: string,
path: string,
expected: IssueId[],
support: boolean,
) => {
const url = normalizeUrl(`${origin}${path}`) ?? `${origin}${path}`;
const got = byUrl.get(url) ?? new Set<IssueId>();
if (!crawledUrls.has(url)) {
failures++;
rows.push({ name, url, ok: false, detail: c.red("NOT CRAWLED") });
return;
}
const expectedSet = new Set(expected);
if (support) {
// Support pages may carry info-level noise (e.g. a deep waypoint that is
// itself deep) but must not have critical/warning issues.
const bad = [...got].filter((id) => !INFO_ISSUES.has(id));
const ok = bad.length === 0;
if (!ok) failures++;
rows.push({
name,
url,
ok,
detail: ok
? c.dim("clean (support)")
: c.red(`unexpected: ${bad.join(", ")}`),
});
return;
}
const missing = [...expectedSet].filter((id) => !got.has(id));
const extra = [...got].filter((id) => !expectedSet.has(id));
const ok = missing.length === 0 && extra.length === 0;
if (!ok) failures++;
const parts: string[] = [];
if (missing.length) parts.push(c.red(`missing: ${missing.join(", ")}`));
if (extra.length) parts.push(c.yellow(`extra: ${extra.join(", ")}`));
rows.push({
name,
url,
ok,
detail: ok ? c.green(expected.join(", ") || "clean") : parts.join(" "),
});
};
// Homepage + catalog must be clean.
check("Homepage", "/", [], false);
check("Catalog", "/catalog", [], false);
const byCategory = new Map<string, Fixture[]>();
for (const f of allFixtures) {
const list = byCategory.get(f.category) ?? [];
list.push(f);
byCategory.set(f.category, list);
}
for (const f of allFixtures) {
check(f.name, f.path, f.expectedIssues, f.support === true);
}
// Explicit regression guard for the trailing-slash redirect cycle
// (every-app/open-seo#61): the canonical page must resolve to a 200 with no
// redirect loop and no error. Fix-agnostic on purpose — the 200 lands on the
// slash form under the root-cause fix (slashes preserved) or on the non-slash
// form under older slash-stripping code that inline-follows. A crawler that
// still strips and doesn't follow would 508 or record a self-redirect loop.
{
const base = `${origin}${TRAILING_SLASH_CANONICAL}`;
const urls = [base, `${base}/`];
const related = pages.filter((p) => urls.includes(p.url));
const has200 = related.some(
(p) => p.fetchClass === "ok" && p.statusCode >= 200 && p.statusCode < 300,
);
const looped = related.some((p) =>
(byUrl.get(p.url) ?? new Set<IssueId>()).has("redirect-loop"),
);
const errored = related.some(
(p) => p.fetchClass === "error" || p.statusCode >= 500,
);
const ok = has200 && !looped && !errored;
if (!ok) failures++;
rows.push({
name: "Trailing-slash cycle → 200, no loop",
url: base,
ok,
detail: ok
? c.green("canonical crawled as 200, no redirect loop")
: c.red(
related.length === 0
? "NOT CRAWLED (looped or dropped)"
: `200=${has200} loop=${looped} error=${errored}`,
),
});
}
// ---- report ----
const pad = Math.max(...rows.map((r) => r.name.length));
for (const r of rows) {
const icon = r.ok ? c.green("✓") : c.red("✗");
console.log(`${icon} ${r.name.padEnd(pad)} ${r.detail}`);
}
const total = rows.length;
console.log(
"\n" +
(failures === 0
? c.green(c.bold(`ALL ${total} CHECKS PASSED`))
: c.red(c.bold(`${failures}/${total} CHECKS FAILED`))),
);
// Coverage: every audit issue type should be exercised by at least one page.
const exercised = new Set(allFixtures.flatMap((f) => f.expectedIssues));
const allTypes = Object.keys(AUDIT_ISSUE_TYPES) as IssueId[];
const uncovered = allTypes.filter((id) => !exercised.has(id));
console.log(
c.dim(
`\nissue-type coverage: ${allTypes.length - uncovered.length}/${allTypes.length}` +
(uncovered.length
? ` (missing: ${uncovered.join(", ")})`
: " ✓ all covered"),
),
);
process.exit(failures === 0 ? 0 : 1);
}
main().catch((err) => {
console.error(err);
process.exit(1);
});

240
badseo/src/fixtures/content.ts Normal file
View File

@ -0,0 +1,240 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
import { article } from "./helpers";
const CAT = "Content quality";
// 9 — thin content (< 150 words). The whole document (chrome + panel + body)
// stays under the threshold, so only thin-content fires.
const thinContent: Fixture = {
path: "/content/thin-content",
category: CAT,
name: "Thin content",
summary: "The page has almost no text on it.",
lesson:
"A page with very little text rarely ranks, and a lot of thin pages can pull down how a site is judged overall.",
expectedIssues: ["thin-content"],
handler: () =>
htmlResponse(
renderPage({
fixture: thinContent,
title: "Thin content example",
metaDescription:
"A sparse page with almost no visible text, built to show how thin content reads to a crawler.",
bodyHtml: `<h1>Nothing here yet</h1>
<p>This is the whole page. There is not enough here for a search engine to work with.</p>`,
}),
),
};
// 10 — an <img> with no alt attribute -------------------------------------
const imagesMissingAlt: Fixture = {
path: "/content/images-missing-alt",
category: CAT,
name: "Images missing alt text",
summary: "One image on the page has no alt attribute.",
lesson:
'Alt text is how a screen reader describes an image and how a search engine reads it. Decorative images should use alt=""; the rest need a real description.',
expectedIssues: ["images-missing-alt"],
handler: () =>
htmlResponse(
renderPage({
fixture: imagesMissingAlt,
title: "A page with an image that has no alt text",
metaDescription:
"This page has an image with no alt attribute, so a screen reader and a search engine cannot tell what it shows.",
bodyHtml: `<h1>An image with no alt text</h1>
<p class="lede">The image below has no alt text, so a screen reader skips over it as if it were not there.</p>
<img src="/img/placeholder.svg" width="720" height="360">
<h2>Why alt text matters</h2>
<p>An image with no alt attribute is a blank spot for anyone using a screen reader and a mystery to a crawler trying to read the page. Alt text is the caption the software reads. An image that carries meaning needs a short, specific description of what it shows. An image that is only decoration should carry an empty alt so the software knows to skip it on purpose.</p>
<h2>Getting it right</h2>
<p>Describe what the image shows and does, not that it is an image. Keep it short and do not stuff it with keywords. Never leave the attribute off entirely. A missing alt and an empty alt look almost the same in the markup but mean the opposite thing to the software that reads them.</p>`,
}),
),
};
// 11 — the same page served at two URLs (duplicate content + title + meta) --
const DUP_TITLE = "Pumpkin Spice Latte Recipe";
const DUP_META =
"The same latte recipe, published word for word at two different URLs, so a search engine has to pick one and drop the other.";
const DUP_BODY = article({
h1: "The same latte recipe, twice",
lede: "This page is identical, byte for byte, to another URL on this site.",
sections: [
{
h2: "Two URLs, one page",
body: "Duplicate content is the same text living at more than one address: a trailing-slash variant, an http and an https copy, a print version, or, as here, the same article pasted at two paths. A search engine does not want to show the same thing twice, so it picks one URL to keep and drops the rest, and the ranking signals get split across copies that now compete with each other.",
},
{
h2: "How to fix it",
body: "Pick one URL to be the real one and point the duplicates at it with a rel=canonical tag, or send a 301 redirect so people and crawlers both land on the single page. On templated sites, watch for URL parameters and slashes creating identical pages you did not mean to make, each one taking a little strength from the page that should rank.",
},
],
});
const duplicateContent: Fixture = {
path: "/content/duplicate-a",
extraPaths: ["/content/duplicate-b"],
category: CAT,
name: "Duplicate content (same page, two URLs)",
summary: "Identical to /content/duplicate-b: same title, meta, and body.",
lesson:
"Identical pages at different URLs split the ranking signals. Pick one URL and redirect the rest.",
expectedIssues: [
"duplicate-content",
"duplicate-title",
"duplicate-meta-description",
],
handler: () =>
htmlResponse(
renderPage({
fixture: duplicateContent,
title: DUP_TITLE,
metaDescription: DUP_META,
bodyHtml: DUP_BODY,
}),
),
};
// 12 — two different pages that share only a <title> ----------------------
const SHARED_TITLE = "Blue Running Shoes | Sole Mates";
const dupTitleA: Fixture = {
path: "/content/duplicate-title-a",
category: CAT,
name: "Duplicate title (product A)",
summary: 'Shares the title "Blue Running Shoes | Sole Mates" with product B.',
lesson:
"Templated product pages often inherit one generic title. Put the thing that makes each page different (model, colour, size) in the title.",
expectedIssues: ["duplicate-title"],
handler: () =>
htmlResponse(
renderPage({
fixture: dupTitleA,
title: SHARED_TITLE,
metaDescription:
"The Marathon model in cobalt blue, a cushioned daily trainer built for long, flat road miles.",
bodyHtml: article({
h1: "Marathon Trainer, Cobalt Blue",
lede: "A cushioned road shoe for runners who put in serious weekly mileage.",
sections: [
{
h2: "Built for distance",
body: "The Marathon is a long-run shoe: a soft, high-stack midsole that stays comfortable deep into a twenty-miler, a breathable knit upper, and a durable outsole rated for hundreds of road miles. It is a different product from the Sprint, but both pages ship with the same title above, so a search engine cannot tell them apart at a glance.",
},
{
h2: "Why the title matters here",
body: "When every product in a catalogue inherits the same title, the pages compete for the same searches and none of them stands out. The fix is to put the differences, the model name and the colour and the use case, into the title template so Marathon and Sprint read as the separate products they are.",
},
],
}),
}),
),
};
const dupTitleB: Fixture = {
path: "/content/duplicate-title-b",
category: CAT,
name: "Duplicate title (product B)",
summary: 'Shares the title "Blue Running Shoes | Sole Mates" with product A.',
lesson:
"The other half of the duplicate-title pair. A different product with a different write-up and the same title.",
expectedIssues: ["duplicate-title"],
handler: () =>
htmlResponse(
renderPage({
fixture: dupTitleB,
title: SHARED_TITLE,
metaDescription:
"The Sprint model in electric blue, a firm, low-drop racing flat for 5K and 10K races.",
bodyHtml: article({
h1: "Sprint Racer, Electric Blue",
lede: "A light racing flat for short, fast efforts on the track and road.",
sections: [
{
h2: "Built for speed",
body: "The Sprint is the opposite of a long-run shoe: a firm, responsive plate, a low drop that pushes a snappy forefoot strike, and almost no weight to carry around a fast 5K. No one would confuse it with the Marathon in person, but on a results page the two look the same because they carry the same title.",
},
{
h2: "One title, two products",
body: "This is the common store version of a duplicate title: a catalogue template that outputs a generic colour-and-category title for every item. A search engine groups the pages, the click rate drops, and the store competes with itself. Making the titles different is a small template change that helps across the whole catalogue.",
},
],
}),
}),
),
};
// 13 — two different pages that share only a meta description -------------
const SHARED_META =
"Fresh, seasonal, and made locally every morning, then delivered to your door. Order today.";
const dupMetaA: Fixture = {
path: "/content/duplicate-meta-a",
category: CAT,
name: "Duplicate meta description (bakery)",
summary: "Shares its meta description word for word with the florist page.",
lesson:
"One boilerplate description reused across pages produces the same snippet in search. Write one per page.",
expectedIssues: ["duplicate-meta-description"],
handler: () =>
htmlResponse(
renderPage({
fixture: dupMetaA,
title: "Early Riser Bakery, sourdough and pastries",
metaDescription: SHARED_META,
bodyHtml: article({
h1: "Early Riser Bakery",
lede: "Slow-fermented sourdough and pastries, baked before dawn.",
sections: [
{
h2: "What we bake",
body: "Every loaf takes two days: a long, cold fermentation for flavour, a hot oven for the crust, and an open crumb inside. Alongside the bread we make croissants, morning buns, and a rotating set of seasonal tarts. This is clearly a bakery, but its meta description is the same generic sentence used on our florist site.",
},
{
h2: "The snippet problem",
body: "Because the description is shared, someone searching finds two of our pages with the same snippet and no way to tell the bakery from the flower shop. Reusing one sentence across every page feels efficient, but it gives up the chance to sell each page in the one place people actually read before clicking.",
},
],
}),
}),
),
};
const dupMetaB: Fixture = {
path: "/content/duplicate-meta-b",
category: CAT,
name: "Duplicate meta description (florist)",
summary: "Shares its meta description word for word with the bakery page.",
lesson:
"The other half of the duplicate-meta pair. A different business, a different page, the same snippet.",
expectedIssues: ["duplicate-meta-description"],
handler: () =>
htmlResponse(
renderPage({
fixture: dupMetaB,
title: "Petal & Stem, seasonal flower delivery",
metaDescription: SHARED_META,
bodyHtml: article({
h1: "Petal & Stem Florist",
lede: "Hand-tied seasonal bouquets, cut fresh and delivered the same day.",
sections: [
{
h2: "What we arrange",
body: "Our bouquets follow the seasons: tulips and ranunculus in spring, dahlias in late summer, amaryllis and evergreens through winter. Each is hand-tied to order and delivered in water so it lasts. This is very clearly a flower shop, not a bakery, but the two sites were built from one template and share the same meta description.",
},
{
h2: "One sentence, two businesses",
body: "When the same description is used on unrelated pages, the search results show the same snippet twice with nothing to tell them apart, and the click rate drops on both. The fix is to write a specific, honest description for each page that says what that page offers.",
},
],
}),
}),
),
};
export const contentFixtures: Fixture[] = [
thinContent,
imagesMissingAlt,
duplicateContent,
dupTitleA,
dupTitleB,
dupMetaA,
dupMetaB,
];

331
badseo/src/fixtures/head-tags.ts Normal file
View File

@ -0,0 +1,331 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
import { article, lorem } from "./helpers";
const CAT = "Head tags & headings";
// 1 — no <title> element at all -------------------------------------------
const missingTitle: Fixture = {
path: "/head/missing-title",
category: CAT,
name: "Missing <title> tag",
summary: "This page has no <title> element at all.",
lesson:
"The title is the strongest signal of what a page is about, and it is the headline shown in search results. If there is none, the search engine writes one for you.",
expectedIssues: ["missing-title"],
handler: () =>
htmlResponse(
renderPage({
fixture: missingTitle,
// title intentionally omitted
metaDescription:
"This page is fine except for one thing: it has no title tag for search results or browser tabs.",
bodyHtml: article({
h1: "A page with no title",
lede: "The content and headings here are fine. There is just no title tag in the head.",
sections: [
{
h2: "What the title does",
body: "The title tag is how a page introduces itself to a search engine and to anyone scanning a list of results. It carries the main topic and it is the first thing people read. When it is missing, the search engine builds its own title from text on the page, and that is usually worse than one you would write yourself.",
},
{
h2: "Why it goes missing",
body: "A missing title almost always comes from a template. A content type never got a title field, or a component only renders the title when a value is passed in, or a migration dropped the head tags. The page looks fine to a person, so the problem is easy to miss until you check the head.",
},
],
}),
}),
),
};
// 2 — title longer than ~60 chars -----------------------------------------
const longTitle =
"The Complete And Fully Unabridged Guide To Writing Page Titles That Are Much Too Long To Fit In Search Results";
const titleTooLong: Fixture = {
path: "/head/title-too-long",
category: CAT,
name: "Title too long",
summary: `The title is ${longTitle.length} characters. Search results cut off around 60.`,
lesson:
"Past about 60 characters, the title gets cut off with an ellipsis, often mid-word. Put the important words first.",
expectedIssues: ["title-too-long"],
handler: () =>
htmlResponse(
renderPage({
fixture: titleTooLong,
title: longTitle,
metaDescription:
"The title on this page runs well past the length a search result will show.",
bodyHtml: article({
h1: "A title that runs too long",
lede: "There is a length past which a title stops helping and just gets cut off.",
sections: [
{
h2: "Where the title gets cut",
body: "Search engines show the title in a fixed width, which works out to roughly 60 characters for most English text. Anything past that is replaced with an ellipsis. If the useful, specific words are at the end, people never see them. They see a sentence that trails off and they move on to the next result.",
},
{
h2: "How to shorten it",
body: "Lead with the main topic. Drop filler like complete guide to and ultimate. Move the brand name to the end, where a cut hurts least. A tight title of about 50 characters usually does better than a long one, both for clicks and for how clearly a search engine can tell what the page is about.",
},
],
}),
}),
),
};
// 3 — title shorter than ~10 chars ----------------------------------------
const titleTooShort: Fixture = {
path: "/head/title-too-short",
category: CAT,
name: "Title too short",
summary: 'The title is just "Hi", which describes nothing.',
lesson:
"A one-word title wastes the most useful text you control. It cannot hold a topic or give anyone a reason to click.",
expectedIssues: ["title-too-short"],
handler: () =>
htmlResponse(
renderPage({
fixture: titleTooShort,
title: "Hi",
metaDescription:
"The title on this page is a single short word, which tells a search engine almost nothing.",
bodyHtml: article({
h1: "A title that is too short",
lede: "Short is good. This is too short to say anything.",
sections: [
{
h2: "Titles need room to work",
body: "The title is the most useful string of text you control on a page. A two-character title throws that away. It cannot hold the term people search for, it does not describe the page, and it gives nobody a reason to pick this result over the others next to it.",
},
{
h2: "A good length",
body: "Aim for about 30 to 60 characters. That is enough to name the topic, include the words people actually search for, and add a short hook. In practice that is one clear phrase, not a single word and not a full sentence that runs off the end.",
},
],
}),
}),
),
};
// 4 — no meta description --------------------------------------------------
const missingMeta: Fixture = {
path: "/head/missing-meta-description",
category: CAT,
name: "Missing meta description",
summary: 'There is no <meta name="description"> on the page.',
lesson:
"With no description, the search engine pulls some text from the page for the snippet. Sometimes that is fine, and sometimes it is a nav label.",
expectedIssues: ["missing-meta-description"],
handler: () =>
htmlResponse(
renderPage({
fixture: missingMeta,
title: "A page with no meta description",
// metaDescription intentionally omitted
bodyHtml: article({
h1: "No meta description",
lede: "This page has a title and clean headings. It has no meta description.",
sections: [
{
h2: "What the description does",
body: "The meta description is the line of text under the title in search results. It does not affect ranking directly, but a good one lifts the click rate. Leave it out and the search engine writes one for you from text on the page, which may or may not read well.",
},
{
h2: "How to write one",
body: "Treat it as a short ad. In about 70 to 160 characters, say what the page gives you and give a reason to click. Write a different one for each page so results never show the same snippet twice, and do not stuff it with keywords, which search engines tend to ignore.",
},
],
}),
}),
),
};
// 5 — meta description longer than ~160 chars -----------------------------
const longMeta =
"This meta description keeps going well past the point where a search engine would ever show the whole thing, adding clause after clause after clause, so most of it ends up cut off with an ellipsis that no one reads.";
const metaTooLong: Fixture = {
path: "/head/meta-description-too-long",
category: CAT,
name: "Meta description too long",
summary: `The meta description is ${longMeta.length} characters. It gets cut off around 160.`,
lesson:
"Search engines cut the description off near 160 characters. Anything after that is text no one will read.",
expectedIssues: ["meta-description-too-long"],
handler: () =>
htmlResponse(
renderPage({
fixture: metaTooLong,
title: "A page with a very long meta description",
metaDescription: longMeta,
bodyHtml: article({
h1: "A meta description that runs too long",
lede: "You can write too much here, and this page proves it.",
sections: [
{
h2: "Where the snippet gets cut",
body: "The description is shown in a fixed width and cut off with an ellipsis, usually somewhere around 150 to 160 characters. Everything past that point does not appear in the result. If your reason to click is at the end of a long description, it will not be seen.",
},
{
h2: "Keep it to one line",
body: "Put the most important point first, keep the whole thing to a single sentence, and stop before the cutoff. If you find yourself joining three ideas together with semicolons, that is two descriptions competing for one slot.",
},
],
}),
}),
),
};
// 6 — meta description shorter than ~70 chars -----------------------------
const shortMeta = "A short, unique snippet for this one fixture.";
const metaTooShort: Fixture = {
path: "/head/meta-description-too-short",
category: CAT,
name: "Meta description too short",
summary: `The meta description is only ${shortMeta.length} characters.`,
lesson:
"A very short description wastes the space search results give you. Give the page a real summary and a reason to click.",
expectedIssues: ["meta-description-too-short"],
handler: () =>
htmlResponse(
renderPage({
fixture: metaTooShort,
title: "A page with a very short meta description",
metaDescription: shortMeta,
bodyHtml: article({
h1: "A meta description that is too short",
lede: "The page has a description, but it is too short to do much work.",
sections: [
{
h2: "Why short snippets underperform",
body: "The meta description is the short sales pitch under the title in a search result. A tiny description leaves most of that space blank, so the page says less than the results around it. Search engines may also decide the tag is not useful and pull text from the page instead, which can produce a choppy or irrelevant snippet.",
},
{
h2: "What enough detail looks like",
body: "A useful description names the page, explains what someone will get, and gives a reason to click. It does not need to be long, but it needs enough room to be specific. Around 70 to 160 characters is usually enough to summarize the page without drifting into a paragraph.",
},
],
}),
}),
),
};
// 7 — no <h1> --------------------------------------------------------------
const missingH1: Fixture = {
path: "/head/missing-h1",
category: CAT,
name: "Missing H1",
summary: "The page starts at <h2>. There is no H1 anywhere.",
lesson:
"The H1 is the on-page headline that states the main topic. A page without one has no clear anchor for the subject.",
expectedIssues: ["missing-h1"],
handler: () =>
htmlResponse(
renderPage({
fixture: missingH1,
title: "A page with no H1",
metaDescription:
"Every heading on this page is an H2 or lower. The top-level H1 that should state the topic is missing.",
// Hand-authored body: starts at H2 on purpose, no H1.
bodyHtml: `<h2>Where the headline should be</h2>
<p class="lede">This article has plenty to say, but it never states its topic in an H1.</p>
<p>A well-structured page opens with a single H1 that names what the page is about, then uses H2s and H3s for the sections under it. When the H1 is missing, both a screen reader and a search crawler lose the anchor that tells them the main subject, and the page reads like a chapter with no chapter title.</p>
<h2>Why it happens</h2>
<p>Usually a design system styles the real headline as a plain div for exact control, or a template marks the site logo as the only H1 and leaves the article headline as an H2. Either way the fix is the same: make the real headline a proper H1 so the markup matches what the eye already sees.</p>
<h3>A quick check</h3>
<p>Open the page and confirm there is exactly one H1, and that a stranger could read it and know what the page is about before scrolling.</p>`,
}),
),
};
// 8 — empty <h1> -----------------------------------------------------------
const emptyH1: Fixture = {
path: "/head/empty-h1",
category: CAT,
name: "Empty H1",
summary: "The only H1 exists in the markup, but it has no text.",
lesson:
"An empty H1 is the same as no H1 for users and crawlers. The tag exists, but the page still never states its main topic.",
expectedIssues: ["missing-h1"],
handler: () =>
htmlResponse(
renderPage({
fixture: emptyH1,
title: "A page with an empty H1",
metaDescription:
"The only H1 on this page is empty, so the markup has a top-level heading tag but no actual heading text.",
bodyHtml: `<h1></h1>
<p class="lede">The headline tag is present, but it does not contain a headline.</p>
<p>${lorem(90)}</p>
<h2>Why an empty heading fails</h2>
<p>An H1 is useful because it says what the page is about. A blank tag gives screen readers a heading stop with no label and gives search engines no phrase to connect to the topic. It can happen when a template renders the heading wrapper even when the CMS field is blank, or when styling hides all text from the tag.</p>
<h3>What to fix</h3>
<p>Put the real page headline inside the H1, then use H2 and H3 for the sections below it. If the page does not have a clear headline yet, write one before publishing instead of leaving an empty element in the document outline.</p>`,
}),
),
};
// 9 — multiple <h1> --------------------------------------------------------
const multipleH1: Fixture = {
path: "/head/multiple-h1",
category: CAT,
name: "Multiple H1s",
summary: "The page marks three separate lines as an H1.",
lesson:
"More than one H1 splits the main-topic signal, and it usually means a template is applying H1 to things that are not headlines.",
expectedIssues: ["multiple-h1"],
handler: () =>
htmlResponse(
renderPage({
fixture: multipleH1,
title: "A page with three H1s",
metaDescription:
"This page marks three separate lines as an H1, so a crawler gets three competing claims about the topic.",
bodyHtml: `<h1>This line is an H1</h1>
<h1>So is this one</h1>
<p class="lede">Three H1s on one page. A crawler cannot tell which one is the actual topic.</p>
<p>An H1 is meant to be the single most important label on a page, the headline. When a template makes the logo, the headline, and a sidebar title all H1s, a crawler gets three claims about what the page is about and has to guess. The topic signal you meant to send gets split three ways.</p>
<h1>And this is the third H1</h1>
<p>The usual cause is a shared heading component with a fixed level, dropped into places that should have been H2 or H3. Pick the one real headline, keep it as the H1, and lower the rest. Something like a logo should not be a heading at all; a styled span or div is the right tag for that.</p>`,
}),
),
};
// 10 — heading levels skip (h1 -> h4) -------------------------------------
const headingSkip: Fixture = {
path: "/head/heading-order-skip",
category: CAT,
name: "Heading levels skip",
summary: "The headings jump from <h1> to <h4>, skipping H2 and H3.",
lesson:
"Skipping heading levels breaks the outline that screen readers and parsers rely on. Go down one level at a time.",
expectedIssues: ["heading-order-skip"],
handler: () =>
htmlResponse(
renderPage({
fixture: headingSkip,
title: "A page that skips heading levels",
metaDescription:
"The headings on this page jump from H1 to H4, leaving a gap that confuses screen readers and parsers.",
bodyHtml: `<h1>Headings that skip a level</h1>
<p class="lede">This page goes from an H1 straight to an H4, as if H2 and H3 did not exist.</p>
<h4>The first section, which should have been an H2</h4>
<p>Headings are not just larger and smaller text. They form a nested outline that screen readers announce and that search engines use to understand structure. When the page jumps from H1 to H4, a screen reader user hears that they have gone three levels deep with nothing in between, which is confusing and makes the page harder to move through by headings.</p>
<h4>Another skipped-level section</h4>
<p>The fix is about the tag, not the size. If you want smaller text, use CSS, but keep the heading one level below its parent. A clean outline goes H1, then H2, then H3, then H4, so anyone reading the structure can follow it without a missing rung.</p>`,
}),
),
};
export const headTagFixtures: Fixture[] = [
missingTitle,
titleTooLong,
titleTooShort,
missingMeta,
metaTooLong,
metaTooShort,
missingH1,
emptyH1,
multipleH1,
headingSkip,
];

42
badseo/src/fixtures/helpers.ts Normal file
View File

@ -0,0 +1,42 @@
import { escapeHtml, lorem } from "../lib";
/**
* A healthy article body: exactly one <h1>, headings that descend one level at
* a time, comfortably more than 150 words, and an image WITH alt text. Head-tag
* fixtures reuse this unchanged and break only the <head>, so the audit sees a
* single injected defect and nothing else.
*/
export function article(opts: {
h1: string;
lede: string;
sections: Array<{
h2: string;
body: string;
h3?: { h3: string; body: string };
}>;
/** Include a properly-described image. Default true. */
withImage?: boolean;
}): string {
const parts: string[] = [];
parts.push(`<h1>${escapeHtml(opts.h1)}</h1>`);
parts.push(`<p class="lede">${escapeHtml(opts.lede)}</p>`);
if (opts.withImage !== false) {
parts.push(
`<img src="/img/placeholder.svg" alt="${escapeHtml(
opts.h1,
)}, an illustrative diagram" width="720" height="360">`,
);
}
for (const s of opts.sections) {
parts.push(`<h2>${escapeHtml(s.h2)}</h2>`);
parts.push(`<p>${escapeHtml(s.body)}</p>`);
if (s.h3) {
parts.push(`<h3>${escapeHtml(s.h3.h3)}</h3>`);
parts.push(`<p>${escapeHtml(s.h3.body)}</p>`);
}
}
return parts.join("\n");
}
/** Generate ~n words of on-topic-ish filler to hit word thresholds. */
export { lorem };

148
badseo/src/fixtures/http-status.ts Normal file
View File

@ -0,0 +1,148 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
import { article } from "./helpers";
const CAT = "HTTP status & links";
// 18 — a URL that returns 404 (discovered via sitemap) --------------------
const notFound: Fixture = {
path: "/status/not-found",
category: CAT,
name: "Page returns 404",
summary: "Listed in the sitemap, but responds 404 Not Found.",
lesson:
"A dead URL in your sitemap wastes crawl budget on every visit. Remove it, restore the page, or redirect it to a real one.",
expectedIssues: ["broken-page"],
// The sitemap lists it, so a crawler finds a dead URL. Kept off the catalog
// so the catalog itself does not earn a broken-internal-link.
linkedFromCatalog: false,
inSitemap: true,
handler: () =>
htmlResponse(
renderPage({
fixture: notFound,
title: "404, this page does not exist",
metaDescription: "A URL that is listed in the sitemap but returns 404.",
bodyHtml: article({
h1: "404, but the sitemap still lists it",
lede: "The sitemap says this page exists. The server returns 404.",
sections: [
{
h2: "Why a 404 in the sitemap is a problem",
body: "A sitemap is a list of pages you are telling search engines to go crawl. When one of those URLs returns 404, you spend crawl budget fetching nothing and keep pointing the crawler at a page that is not there. On a large site, thousands of these add up.",
},
{
h2: "The fix",
body: "If the page should exist, restore it. If it is gone for good, take it out of the sitemap and out of any internal links, and if something replaced it, add a 301 to that page. What you do not want is to leave it in the sitemap, telling crawlers to keep visiting a URL that no longer works.",
},
],
}),
}),
{ status: 404 },
),
};
// 19 — a URL that returns 500 --------------------------------------------
const serverError: Fixture = {
path: "/status/server-error",
category: CAT,
name: "Server error (500)",
summary: "Responds 500 Internal Server Error instead of a page.",
lesson:
"Repeated 5xx errors make search engines crawl a site less and can drop pages from the index. A missing page should return 404, not 500.",
expectedIssues: ["server-error"],
linkedFromCatalog: false,
inSitemap: true,
handler: () =>
htmlResponse(
renderPage({
fixture: serverError,
title: "500, the server errored",
metaDescription: "A URL that returns a 500 error to every crawler.",
bodyHtml: article({
h1: "500, a server error",
lede: "This URL does not return 404. It returns a 500 every time.",
sections: [
{
h2: "5xx is worse than 4xx",
body: "A 404 says the page is not here. A 500 says the server itself failed while trying to answer. Search engines treat repeated 5xx errors as a sign the site is unhealthy and respond by slowing their crawl. Do it often enough and pages start dropping out of the index.",
},
{
h2: "Return the right code",
body: "If content is gone, return a 404 or 410 so the search engine can update its records. Keep 500s for actual, unexpected failures, and then read the logs and fix them. A route that returns 500 every time is not an error page, it is a bug.",
},
],
}),
}),
{ status: 500 },
),
};
// 20 — bot challenge / access denied (403) --------------------------------
const blocked: Fixture = {
path: "/status/blocked",
category: CAT,
name: "Crawler blocked (403)",
summary: 'Returns 403 Forbidden. The honest "we could not read this" case.',
lesson:
"A 403, a 429, or a bot challenge means the crawler was blocked. A good audit says so, instead of reporting the page as broken. Real search bots may hit the same wall.",
expectedIssues: ["blocked-page"],
linkedFromCatalog: false,
inSitemap: true,
handler: () =>
htmlResponse(
renderPage({
fixture: blocked,
title: "403, the crawler was blocked",
metaDescription: "A page that returns 403 Forbidden to crawlers.",
bodyHtml: article({
h1: "403, access denied to the crawler",
lede: "Aggressive bot protection can block the good crawlers along with the bad ones.",
sections: [
{
h2: "Blocked is not the same as broken",
body: "When a page answers a crawler with 403, 429, or a challenge screen, the honest conclusion is not that the page is broken. It is that the crawler was not allowed to see it. A good audit says exactly that. Reporting a blocked page as a content problem would send you looking for a bug that is not there, when the real issue is access.",
},
{
h2: "When your own protection backfires",
body: "Strict WAF rules and bot-fight modes often catch real crawlers in the same net as scrapers. If search engines or your own audit keep getting blocked, allowlist their user agents so they can read the site. A page nobody can crawl is a page that cannot rank, however good the content behind the wall is.",
},
],
}),
}),
{ status: 403 },
),
};
// 21 — a healthy page that links to the 404 above -------------------------
const brokenInternalLink: Fixture = {
path: "/links/broken-internal-link",
category: CAT,
name: "Broken internal link",
summary: "Links to /status/not-found, which returns 404.",
lesson:
"Linking to your own dead URLs frustrates people, wastes crawl budget, and sends link strength nowhere. Fix or remove the link.",
expectedIssues: ["broken-internal-link"],
handler: () =>
htmlResponse(
renderPage({
fixture: brokenInternalLink,
title: "A page with a broken internal link",
metaDescription:
"This page is otherwise fine, but it links to one of its own URLs that returns a 404.",
bodyHtml: `<h1>The link below goes nowhere</h1>
<p class="lede">This page is fine, except that it links to a page that no longer exists.</p>
<p>Broken internal links are one of the most common and most avoidable technical SEO problems. A person clicks, hits a 404, and leaves. A crawler follows the link, wastes a request, and learns nothing. Any link strength that should have gone to a real page goes into a dead end instead. Unlike a broken external link, this one is entirely yours to fix.</p>
<p>Here is the link, pointing at a URL on this site that returns a 404: <a href="/status/not-found">read our full guide</a>. Click it and you land on a Not Found page, which is what the audit reports when it crawls this link and sees the target return 404.</p>
<h2>How to catch these</h2>
<p>Crawl your own site regularly and check the status of every internal link target. The moment a linked page starts returning 4xx or 5xx, repoint the link to the correct URL or remove it. Do not rely on a redirect to cover it forever; link straight to the page that works.</p>`,
}),
),
};
export const httpStatusFixtures: Fixture[] = [
notFound,
serverError,
blocked,
brokenInternalLink,
];

157
badseo/src/fixtures/indexability.ts Normal file
View File

@ -0,0 +1,157 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
import { article } from "./helpers";
const CAT = "Indexability & canonical";
// 14 — noindex via robots meta tag ----------------------------------------
const noindexMeta: Fixture = {
path: "/index/noindex-meta",
category: CAT,
name: "Noindex (robots meta)",
summary: 'Has <meta name="robots" content="noindex"> in the head.',
lesson:
"Noindex is often on purpose, like on a thank-you or filter page. An audit flags it so you can catch pages that were hidden from search by mistake.",
expectedIssues: ["noindex-page"],
handler: () =>
htmlResponse(
renderPage({
fixture: noindexMeta,
title: "A page set to noindex",
metaDescription:
"This page can be crawled, but a robots noindex tag tells search engines to keep it out of results.",
robotsMeta: "noindex, follow",
bodyHtml: article({
h1: "This page asks not to be indexed",
lede: "Nothing is wrong with this page. It just does not want to be in Google.",
sections: [
{
h2: "When noindex is right",
body: "Some pages should not show up in search: internal search results, filter combinations, thank-you pages, staging content. A robots noindex tag is the correct way to keep them out while still letting crawlers follow their links. It is a feature, most of the time.",
},
{
h2: "When it is a problem",
body: "The same tag becomes a problem when it ends up on pages that were meant to rank. A noindex left over from a staging setup, or a template that applies it too widely, can drop a whole section of a site from search. That is why an audit reports every noindex it finds, so a person can confirm each one is on purpose.",
},
],
}),
}),
),
};
// 15 — noindex via X-Robots-Tag response header ---------------------------
const noindexHeader: Fixture = {
path: "/index/noindex-header",
category: CAT,
name: "Noindex (X-Robots-Tag header)",
summary: "No robots meta tag. The noindex comes in an HTTP header instead.",
lesson:
"X-Robots-Tag lives in the response headers, not the HTML. A crawler has to read headers, not just the page, to catch it.",
expectedIssues: ["noindex-page"],
handler: () =>
htmlResponse(
renderPage({
fixture: noindexHeader,
title: "Noindex set in a response header",
metaDescription:
"The HTML on this page looks indexable, but an X-Robots-Tag header tells search engines to skip it.",
bodyHtml: article({
h1: "A noindex you cannot see in the HTML",
lede: "View the source all you want. This page's noindex is in the HTTP headers.",
sections: [
{
h2: "Headers can control indexing too",
body: "The X-Robots-Tag response header does what a robots meta tag does, but from the HTTP layer instead of the page. It is handy for files like PDFs and for setting rules at the server or CDN level. The catch is that it is invisible if you only look at the rendered HTML, which is how it can hide for months.",
},
{
h2: "Why a tool has to check both",
body: "A crawler that reads only the HTML will report this page as indexable, because nothing in the markup says otherwise. Catching a header-level rule means reading the response headers on every request. This page exists to check that an audit does that, instead of trusting the HTML.",
},
],
}),
}),
{ headers: { "x-robots-tag": "noindex" } },
),
};
// 16 — canonical points to a different URL --------------------------------
const canonicalized: Fixture = {
path: "/index/canonicalized",
category: CAT,
name: "Canonicalized to another URL",
summary: "Names the homepage as its canonical, so it defers indexing to it.",
lesson:
"A canonical that points at another URL tells search engines to index that page instead. Fine on purpose, and a quiet way to lose rankings by mistake.",
expectedIssues: ["canonicalized-page"],
handler: (ctx) =>
htmlResponse(
renderPage({
fixture: canonicalized,
title: "A page whose canonical points elsewhere",
metaDescription:
"This page's rel=canonical points at the homepage, telling search engines to credit that URL instead of this one.",
canonical: `${ctx.origin}/`,
bodyHtml: article({
h1: "This is not the canonical page",
lede: "This page exists, but it tells search engines to index a different URL in its place.",
sections: [
{
h2: "What a cross-URL canonical does",
body: "A rel=canonical pointing at a different address is an instruction: treat that other URL as the real one and fold the ranking signals into it. It is the right tool for syndicated copies, parameter variants, and print versions. Used on purpose, it prevents duplicate-content problems before they start.",
},
{
h2: "When it goes wrong",
body: "It becomes a real problem when a template hardcodes the homepage, or a staging domain, as the canonical for every page. Now the whole site tells search engines that none of its pages should rank on their own, and they should all defer to one URL. The pages drop out of results with no error anywhere, which is why an audit reports every canonical that points away from its own page.",
},
],
}),
}),
),
};
// 17 — HTML canonical conflicts with Link-header canonical ----------------
const canonicalConflict: Fixture = {
path: "/index/canonical-conflict",
category: CAT,
name: "Conflicting canonical signals",
summary: "The HTML canonical and the HTTP Link-header canonical disagree.",
lesson:
"When two canonical tags point at different URLs, search engines trust neither and pick their own. Declare the canonical in one place.",
expectedIssues: ["canonical-conflict", "canonicalized-page"],
handler: (ctx) =>
htmlResponse(
renderPage({
fixture: canonicalConflict,
title: "A page with two different canonicals",
metaDescription:
"This page ships two canonical URLs that disagree: one in the HTML head and a different one in the HTTP Link header.",
canonical: `${ctx.origin}/index/canonical-conflict?via=html`,
bodyHtml: article({
h1: "Two canonicals that disagree",
lede: "The head names one canonical URL. The HTTP header names another. Both lose.",
sections: [
{
h2: "Mixed signals",
body: "You can set a canonical URL in the HTML head with a link tag, or in the HTTP response with a Link header. Search engines read both. When the two disagree, as they do here, the search engine cannot trust either one, so it drops them and picks a canonical on its own, which is rarely the one you wanted.",
},
{
h2: "Pick one place",
body: "Set the canonical in one place and keep it consistent. Most sites use the HTML head tag and never touch the header. If a CDN or framework is adding a Link-header canonical you did not ask for, that is usually the cause. Line it up with the head tag, or remove it, so there is one clear answer.",
},
],
}),
}),
{
headers: {
link: `<${ctx.origin}/index/canonical-conflict?via=header>; rel="canonical"`,
},
},
),
};
export const indexabilityFixtures: Fixture[] = [
noindexMeta,
noindexHeader,
canonicalized,
canonicalConflict,
];

47
badseo/src/fixtures/kitchen-sink.ts Normal file
View File

@ -0,0 +1,47 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
const KITCHEN_SINK_TITLE =
"The Kitchen Sink Page That Breaks Six SEO Rules At Once And Has A Title That Is Far Too Long";
// A single page that trips several checks at once. Deliberately NOT thin (it
// has plenty of text) and NOT noindex, so it reads as a page that wants to rank
// and gets in its own way six different ways.
const kitchenSink: Fixture = {
path: "/kitchen-sink",
category: "Kitchen sink",
name: "Kitchen sink (everything at once)",
summary:
"One page, six problems: a title that is too long, no meta description, two H1s, a skipped heading level, an image with no alt, and a slow response.",
lesson:
"Real broken pages usually have more than one problem. An audit should report every one of these, not stop at the first.",
expectedIssues: [
"title-too-long",
"missing-meta-description",
"multiple-h1",
"heading-order-skip",
"images-missing-alt",
"slow-response",
],
handler: () =>
htmlResponse(
renderPage({
fixture: kitchenSink,
title: KITCHEN_SINK_TITLE,
// meta description intentionally omitted
bodyHtml: `<h1>The kitchen sink page</h1>
<h1>Yes, that was two H1s</h1>
<p class="lede">This page is broken on purpose, in more than one way, so an audit has plenty to find.</p>
<img src="/img/placeholder.svg" width="720" height="360">
<h4>This section skips straight to an H4</h4>
<p>Everything on this page is wrong on purpose, and it is wrong in more than one way at the same time. That is how broken pages usually show up. Nobody ships a page with a single, tidy problem. They ship a template that is a little off in a few places, and each small problem chips away at how the page does in search.</p>
<p>Here they are. The title runs well past 60 characters, so search results cut it off. There is no meta description, so the search engine writes a snippet from whatever text it finds. Two lines are marked as H1, so the page makes two claims about its own topic. The outline then jumps from H1 to H4, skipping two levels.</p>
<h4>And a few more</h4>
<p>The image a few paragraphs up has no alt attribute, so it is invisible to a screen reader and to image search. And the whole response is slow, waiting well over a second and a half before the first byte. Any one of these is worth fixing on its own. Together they are a checklist of the most common on-page problems, put on one URL so you can watch an audit find all of them instead of stopping after the first.</p>
<p>If an audit reports all six problems for this page, it is working. If it only finds one or two, it is giving people false confidence, which is worse than no audit at all, because real problems hide behind a clean-looking report.</p>`,
}),
{ delayMs: 1700 },
),
};
export const kitchenSinkFixtures: Fixture[] = [kitchenSink];

42
badseo/src/fixtures/performance.ts Normal file
View File

@ -0,0 +1,42 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage } from "../lib";
import { article } from "./helpers";
const CAT = "Performance";
// 24 — slow server response (> 1.5s TTFB) ---------------------------------
const slowResponse: Fixture = {
path: "/perf/slow-response",
category: CAT,
name: "Slow server response",
summary: "The server waits about 1.7 seconds before sending anything.",
lesson:
"A slow time to first byte holds up everything after it and lowers the crawl rate on large sites. Cache or pre-render the HTML.",
expectedIssues: ["slow-response"],
handler: () =>
htmlResponse(
renderPage({
fixture: slowResponse,
title: "A page with a slow server response",
metaDescription:
"This page waits about 1.7 seconds before it responds, past the point where a slow time to first byte starts to hurt.",
bodyHtml: article({
h1: "A slow response",
lede: "The pause before this page loaded was on purpose, and it was too long.",
sections: [
{
h2: "Why time to first byte matters",
body: "Time to first byte is how long the server takes before it sends anything at all. It sets the floor under every other speed metric. The browser cannot start rendering, and the crawler cannot start reading, until that first byte arrives. A slow time to first byte makes a fast front end feel slow, and at scale it means search engines crawl fewer of your pages per visit.",
},
{
h2: "Common causes and fixes",
body: "Most slow responses come from doing real work on every request: uncached database queries, slow upstream APIs, or rendering that could have been done ahead of time. The usual fix is to stop building the same HTML over and over. Cache it, generate it in advance, or serve it from the edge, so the server can answer in milliseconds.",
},
],
}),
}),
{ delayMs: 1700 },
),
};
export const performanceFixtures: Fixture[] = [slowResponse];

108
badseo/src/fixtures/redirects.ts Normal file
View File

@ -0,0 +1,108 @@
import type { Fixture } from "./types";
import { redirect, htmlResponse, renderPage } from "../lib";
import { article } from "./helpers";
const CAT = "Redirects";
/**
* Canonical path for the trailing-slash fixture. The SLASH form is the
* canonical 200; the non-slash form 301-redirects to it (see the intercept in
* index.ts). Exported so the Worker and the fixture agree on the path.
*/
export const TRAILING_SLASH_CANONICAL = "/redirect/trailing-slash";
// 22 — redirect chain: /chain-1 -> /chain-2 -> / (homepage) ----------------
// Two hops before content is a chain. The crawler records each hop as its own
// row and flags the head of the chain.
const redirectChain: Fixture = {
path: "/redirect/chain-1",
category: CAT,
name: "Redirect chain (3 hops)",
summary:
"/redirect/chain-1 redirects to chain-2, which redirects to the homepage. Two hops to reach content.",
lesson:
"Every extra hop adds a little delay, loses a little link strength, and uses crawl budget. Point the first URL straight at the final page.",
expectedIssues: ["redirect-chain"],
inSitemap: false,
handler: () => redirect("/redirect/chain-2", 301),
};
// Support: the middle hop. Not featured anywhere; found via chain-1.
const redirectChainMid: Fixture = {
path: "/redirect/chain-2",
category: CAT,
name: "Redirect chain (middle hop)",
summary: "The middle of the redirect chain. Redirects on to the homepage.",
lesson: "",
expectedIssues: [],
support: true,
linkedFromCatalog: false,
inSitemap: false,
handler: () => redirect("/", 301),
};
// 23 — redirect loop: a URL that redirects to itself ----------------------
const redirectLoop: Fixture = {
path: "/redirect/loop",
category: CAT,
name: "Redirect loop",
summary: "/redirect/loop redirects to itself, so it never resolves.",
lesson:
"A redirect that points back to itself never reaches a real page. Browsers and crawlers give up with an error.",
expectedIssues: ["redirect-loop"],
inSitemap: false,
// Location resolves to this same URL, a self-redirect the crawler can never
// satisfy, which is a redirect loop.
handler: () => redirect("/redirect/loop", 302),
};
// 24 — trailing-slash canonical (redirect-cycle trap) ---------------------
// The canonical URL ends in a slash (/redirect/trailing-slash/ = 200); the
// non-slash form 301-redirects to it, exactly like WordPress and most CMSes.
// A crawler that normalizes away trailing slashes turns /redirect/trailing-slash/
// back into /redirect/trailing-slash, follows the 301 to the slash form, strips
// it again, and loops — the 508 "Loop Detected" class of bug from
// https://github.com/every-app/open-seo/pull/61. The audit must crawl the
// canonical page once as a 200 and NOT report a redirect loop, so this fixture
// expects zero issues. If it ever comes back with redirect-loop (or an error),
// the trailing-slash handling has regressed.
const trailingSlashCanonical: Fixture = {
path: TRAILING_SLASH_CANONICAL,
category: CAT,
name: "Trailing-slash canonical (redirect-cycle trap)",
summary:
"The canonical URL ends in a slash; the non-slash form 301-redirects to it, like WordPress. This page is correct — it's a trap for crawlers that strip trailing slashes.",
lesson:
"A crawler that normalizes /foo/ to /foo will follow the 301 back to /foo/, strip it again, and loop forever (508 Loop Detected). The audit must crawl the canonical page once as a 200 and not report a false redirect loop.",
expectedIssues: [],
handler: () =>
htmlResponse(
renderPage({
fixture: trailingSlashCanonical,
title: "Trailing-slash canonical page",
metaDescription:
"The canonical version of this page ends in a trailing slash, and the non-slash form redirects to it, which trips up crawlers that strip slashes.",
bodyHtml: article({
h1: "This is the canonical, trailing-slash version",
lede: "You reached this via a 301 from the non-slash URL. That is normal, and a crawler must handle it without looping.",
sections: [
{
h2: "Why the trailing slash matters",
body: "Most content management systems treat the trailing-slash form of a URL as canonical and 301-redirect the non-slash form to it. So /services and /services/ are not two pages; one is a permanent redirect to the other. This is correct behaviour, and every crawler has to deal with it.",
},
{
h2: "Where crawlers go wrong",
body: "A crawler that quietly strips trailing slashes to deduplicate URLs turns the canonical /services/ back into /services, fetches it, gets a 301 to /services/, strips the slash again, and queues /services once more. That cycle repeats until a loop detector gives up with a 508. The fix is to preserve the trailing slash, or to follow the redirect to the canonical form and mark it visited, so the page is crawled once and never re-queued.",
},
],
}),
}),
),
};
export const redirectFixtures: Fixture[] = [
redirectChain,
redirectChainMid,
redirectLoop,
trailingSlashCanonical,
];

51
badseo/src/fixtures/registry.ts Normal file
View File

@ -0,0 +1,51 @@
import type { Fixture } from "./types";
import { headTagFixtures } from "./head-tags";
import { contentFixtures } from "./content";
import { indexabilityFixtures } from "./indexability";
import { httpStatusFixtures } from "./http-status";
import { redirectFixtures } from "./redirects";
import { performanceFixtures } from "./performance";
import { structureFixtures } from "./structure";
import { kitchenSinkFixtures } from "./kitchen-sink";
/** Every fixture on the site, in catalog order. */
export const allFixtures: Fixture[] = [
...headTagFixtures,
...contentFixtures,
...indexabilityFixtures,
...httpStatusFixtures,
...redirectFixtures,
...performanceFixtures,
...structureFixtures,
...kitchenSinkFixtures,
];
/** All URL paths a fixture answers on (canonical + any duplicates). */
export function fixturePaths(fixture: Fixture): string[] {
return [fixture.path, ...(fixture.extraPaths ?? [])];
}
/**
* Fixtures the catalog links to (so the crawler can reach them and they aren't
* orphaned). Each gets a card. Includes the deep-chain entrance even though
* it's technically plumbing.
*/
export const catalogLinkedFixtures = allFixtures.filter(
(f) => f.linkedFromCatalog !== false,
);
/** Fixtures whose paths belong in sitemap.xml. */
export const sitemapFixtures = allFixtures.filter((f) => f.inSitemap !== false);
/**
* Duplicate/alternate URLs (a fixture's extraPaths). The catalog links these so
* the duplicate pages are crawled — and, crucially, aren't mistaken for orphans.
*/
export const duplicateUrlLinks = allFixtures.flatMap((f) =>
(f.extraPaths ?? []).map((path) => ({ path, name: f.name })),
);
/** Distinct categories, in first-seen order. */
export const categories: string[] = [
...new Set(catalogLinkedFixtures.map((f) => f.category)),
];

170
badseo/src/fixtures/structure.ts Normal file
View File

@ -0,0 +1,170 @@
import type { Fixture } from "./types";
import { htmlResponse, renderPage, renderDocument, escapeHtml } from "../lib";
import { article } from "./helpers";
const CAT = "Site structure";
// 25 — orphan page: in the sitemap, but nothing links to it ---------------
const orphan: Fixture = {
path: "/structure/orphan",
category: CAT,
name: "Orphan page",
summary: "Only reachable through the sitemap. No internal link points to it.",
lesson:
"A page with no internal links gets little crawl attention and no internal link strength, and people cannot reach it by browsing. Link to it from somewhere relevant.",
expectedIssues: ["orphan-page"],
// The point of the page: it is in the sitemap, but nothing links to it.
linkedFromCatalog: false,
inSitemap: true,
handler: () =>
htmlResponse(
renderPage({
fixture: orphan,
title: "An orphan page",
metaDescription:
"A normal page that no other page on the site links to. It is only reachable because it is listed in the sitemap.",
bodyHtml: article({
h1: "A page nothing links to",
lede: "This page is fine. No other page on the site links to it.",
sections: [
{
h2: "What makes a page an orphan",
body: "An orphan page has no internal links pointing at it. You can still reach it if you know the URL, and it may be in the sitemap, but there is no path to it by clicking around the site. Search engines lean on internal links to find pages and to pass strength between them, so an orphan gets crawled rarely and ranks weakly, however good it is.",
},
{
h2: "How pages get orphaned",
body: "It usually happens by accident. A page is removed from a menu but the URL stays live, a campaign landing page loses its links when the campaign ends, or a bulk import creates pages that were never added to navigation. The fix is to link to the page from somewhere relevant, like a hub page, a related article, or the main navigation, so both crawlers and people can find it.",
},
],
}),
}),
),
};
// 26 — no outgoing links ---------------------------------------------------
const noOutgoingLinks: Fixture = {
path: "/structure/no-outgoing-links",
category: CAT,
name: "No outgoing links",
summary: "The page is otherwise healthy, but contains zero anchor links.",
lesson:
"A page with no outgoing links is a dead end for crawlers and people. Link onward to related content, a parent category, or the homepage.",
expectedIssues: ["no-outgoing-links"],
handler: () =>
htmlResponse(
renderDocument({
title: "Page with no outgoing links",
metaDescription:
"This healthy page deliberately renders no anchor tags, so crawlers and users reach a dead end.",
bodyHtml: `<main class="main">
${article({
h1: "A page with no outgoing links",
lede: "Everything on this page is healthy except the complete lack of links.",
sections: [
{
h2: "Why dead ends are a problem",
body: "Search crawlers move through a site by following links. When a page has no outgoing links, any link strength that reaches it stops there, and a crawler has no next page to discover from this point. People hit the same problem when they finish reading and have nowhere useful to go except the browser back button.",
},
{
h2: "What a healthy page should do",
body: "Most pages should point somewhere sensible after the main content: a related article, a parent category, the homepage, or the next step in a flow. The exact destination depends on the page, but the pattern is the same. A useful page should help visitors continue, and it should help crawlers understand how this URL fits into the wider site instead of treating the page as an isolated endpoint.",
},
],
})}
</main>`,
}),
),
};
// 27 — deep pages: a click-chain 5+ levels from the homepage --------------
// catalog(1) -> deep/1(2) -> deep/2(3) -> deep/3(4) -> deep/4(5) -> treasure(6).
// Only deep/1 is linked from the catalog; each level links onward in its body,
// so the chain is the ONLY way down. Pages at depth >= 5 trip deep-page.
function deepWaypoint(level: number, next: string): Fixture {
const fx: Fixture = {
path: `/structure/deep/${level}`,
category: CAT,
name: `Deep click-path, level ${level}`,
summary: `Step ${level} of a long click-chain that leaves a page many clicks deep.`,
lesson:
"Pages many clicks from the homepage get crawled less and get less link strength. Flatten the path with links from higher-level pages.",
expectedIssues: [],
support: true,
linkedFromCatalog: level === 1, // only the entrance is on the catalog
inSitemap: false,
handler: () =>
htmlResponse(
renderPage({
fixture: fx,
title: `Deep click-path, level ${level}`,
metaDescription: `Level ${level} of a deliberately deep click-path. The only way on is the single link at the bottom of the page.`,
bodyHtml: `${article({
h1: `You are ${level} click${level === 1 ? "" : "s"} deep`,
lede: `The buried page is further down than it should be. Keep going.`,
sections: [
{
h2: "Why click-depth matters",
body: "Search engines spend more crawl budget on pages close to the homepage and less on pages behind many clicks. Depth is a rough measure of importance. If it takes six clicks to reach a page, the site's own structure is saying the page barely matters. Content you care about should not be that far down.",
},
{
h2: "The only way on",
body: "This page exists to add one more level of depth to the chain. There is exactly one link forward, below, and no shortcut from anywhere closer to the top. That is how a real site can leave an important page buried behind a single narrow trail of links.",
},
],
})}
<p class="next"><a href="${escapeHtml(next)}">Go to the next level down</a></p>`,
}),
),
};
return fx;
}
const treasure: Fixture = {
path: "/structure/deep/treasure",
category: CAT,
name: "Deep page (6 clicks down)",
summary: "Sits 6 clicks from the homepage, at the end of the click-chain.",
lesson:
"A page this deep is crawled rarely and gets almost no link strength. If it matters, link to it from much closer to the top.",
expectedIssues: ["deep-page"],
linkedFromCatalog: false,
inSitemap: false,
handler: () =>
htmlResponse(
renderPage({
fixture: treasure,
title: "The deeply buried page",
metaDescription:
"This page sits six clicks from the homepage, which is why search engines rarely reach it.",
bodyHtml: article({
h1: "A page buried six clicks deep",
lede: "You reached this in six clicks. A crawler would likely have stopped before now.",
sections: [
{
h2: "This far down, ranking is hard",
body: "This page sits five or more clicks from the homepage. That is deep enough that crawlers visit it rarely and pass it very little internal strength. If this were a product, an article, or a landing page you cared about, its position this far down would cap how well it can do.",
},
{
h2: "How to fix a buried page",
body: "The fix is not to delete the pages in between. It is to add shortcuts. Link to important deep pages straight from hubs, category pages, or the main navigation, so they sit two or three clicks from home instead of six. A flatter structure tells search engines the page matters and gives it a real chance to rank.",
},
],
}),
}),
),
};
const deep1 = deepWaypoint(1, "/structure/deep/2");
const deep2 = deepWaypoint(2, "/structure/deep/3");
const deep3 = deepWaypoint(3, "/structure/deep/4");
const deep4 = deepWaypoint(4, "/structure/deep/treasure");
export const structureFixtures: Fixture[] = [
orphan,
noOutgoingLinks,
deep1,
deep2,
deep3,
deep4,
treasure,
];

51
badseo/src/fixtures/types.ts Normal file
View File

@ -0,0 +1,51 @@
// The issue-id union comes straight from the OpenSEO audit engine so that
// every fixture's `expectedIssues` is type-checked against the real registry.
// Type-only import — erased at build time, never bundled into the Worker.
import type { AuditIssueType } from "../../../src/shared/audit-issues";
export type IssueId = AuditIssueType;
export interface FixtureContext {
/** Absolute origin the site is being served from (localhost or badseo.dev). */
origin: string;
/** The request currently being handled. */
request: Request;
/** The path that matched this fixture (may be one of `extraPaths`). */
path: string;
}
export interface Fixture {
/** Canonical URL path, e.g. "/on-page/missing-title". Must start with "/". */
path: string;
/**
* Extra paths that serve byte-identical output. Used to model duplicate
* pages living at several URLs. Each extra path is also crawlable.
*/
extraPaths?: string[];
/** Grouping shown on the catalog page. */
category: string;
/** Short human name, e.g. "Missing <title> tag". */
name: string;
/** One-line description of the mistake, shown in the on-page test panel. */
summary: string;
/** Optional longer educational note (why it matters / how to fix). */
lesson?: string;
/**
* The audit issue ids this page is engineered to trigger. This doubles as
* the assertion source-of-truth for the e2e harness. An empty array means
* "this page must come back clean" (used for healthy nav/support pages).
*/
expectedIssues: IssueId[];
/**
* Support pages (redirect hops, deep-link waypoints) that exist only to make
* another fixture reachable. They're crawled but not featured on the catalog
* and the harness asserts they are clean.
*/
support?: boolean;
/** Include this path in sitemap.xml. Default true. */
inSitemap?: boolean;
/** Link to this page from the catalog. Default true (orphans set false). */
linkedFromCatalog?: boolean;
/** Produce the HTTP response. Full control over status, headers, timing. */
handler: (ctx: FixtureContext) => Response | Promise<Response>;
}

163
badseo/src/index.ts Normal file
View File

@ -0,0 +1,163 @@
// badseo.dev — Cloudflare Worker entry point.
//
// A dependency-free Worker that serves a catalog of deliberately-broken SEO
// pages. Each fixture declares the audit issues it should trigger; this file
// wires the fixtures up to URLs, plus robots.txt and a sitemap.xml that lists
// the pages a crawler is meant to discover (including a few dead ones).
import { STYLESHEET } from "./styles";
import { OPENSEO_LOGO_PNG_BASE64 } from "./logo";
import { renderHome, renderCatalog } from "./pages";
import { renderShell, redirect } from "./lib";
import { TRAILING_SLASH_CANONICAL } from "./fixtures/redirects";
import {
allFixtures,
fixturePaths,
sitemapFixtures,
} from "./fixtures/registry";
import type { Fixture, FixtureContext } from "./fixtures/types";
// Build the path -> fixture table once at module load.
const routeTable = new Map<string, Fixture>();
for (const fixture of allFixtures) {
for (const path of fixturePaths(fixture)) {
routeTable.set(path, fixture);
}
}
const PLACEHOLDER_SVG = `<svg xmlns="http://www.w3.org/2000/svg" width="720" height="360" viewBox="0 0 720 360" role="img" aria-label="placeholder">
<rect width="720" height="360" fill="#ebe7e1"/>
<path d="M1 359 L719 1 M1 1 L719 359" stroke="#d8d1c8" stroke-width="1"/>
<rect x="0.5" y="0.5" width="719" height="359" fill="none" stroke="#d8d1c8"/>
<text x="360" y="188" text-anchor="middle" font-family="ui-monospace, monospace" font-size="24" fill="#7b7b78">placeholder image</text>
</svg>`;
function normalizePath(pathname: string): string {
if (pathname.length > 1 && pathname.endsWith("/")) {
return pathname.slice(0, -1);
}
return pathname;
}
function robotsTxt(origin: string): Response {
const body = `# badseo.dev is broken on purpose, but it lets crawlers in.
User-agent: *
Allow: /
Sitemap: ${origin}/sitemap.xml
`;
return new Response(body, {
headers: { "content-type": "text/plain; charset=utf-8" },
});
}
function sitemapXml(origin: string): Response {
// Note: /catalog is deliberately NOT in the sitemap. It's the hub the deep
// click-chain hangs off, and a sitemap-listed page is crawled at "null"
// click-depth — which would propagate down the chain and stop the deep-page
// fixture from ever reaching depth >= 5. Keeping the catalog link-only means
// the chain gets real, incrementing depths.
const paths = new Set<string>(["/"]);
for (const fixture of sitemapFixtures) {
for (const path of fixturePaths(fixture)) paths.add(path);
}
const urls = [...paths]
.map((path) => ` <url><loc>${origin}${path}</loc></url>`)
.join("\n");
const body = `<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
${urls}
</urlset>`;
return new Response(body, {
headers: { "content-type": "application/xml; charset=utf-8" },
});
}
function html(body: string, status = 200): Response {
return new Response(body, {
status,
headers: {
"content-type": "text/html; charset=utf-8",
"cache-control": "no-store",
},
});
}
function notFoundPage(): Response {
return html(
renderShell({
title: "404, not found | badseo.dev",
metaDescription: "That URL is not one of the pages on this site.",
bodyHtml: `<h1>404, not found</h1>
<p class="lede">This URL is not one of the broken pages on the site. It is just missing.</p>
<p>Go back to the <a href="/catalog">catalog</a> to see the pages that break on purpose.</p>`,
}),
404,
);
}
export default {
async fetch(request: Request): Promise<Response> {
const url = new URL(request.url);
// Derive the origin from the Host header, not url.origin: under `wrangler
// dev` with custom-domain routes configured, url.origin resolves to the
// production host (badseo.dev) even on localhost, which would make every
// sitemap/canonical URL cross-origin to a local crawler.
const host = request.headers.get("host") ?? url.host;
const origin = `${url.protocol}//${host}`;
const path = normalizePath(url.pathname);
// Trailing-slash canonical: the non-slash form 301-redirects to the slash
// form, which is served as the canonical 200 (via the fixture below, since
// normalizePath maps the slash form to the same route). Checked on the RAW
// pathname, before normalization, so the two forms behave differently. This
// reproduces the CMS redirect cycle from every-app/open-seo#61.
if (url.pathname === TRAILING_SLASH_CANONICAL) {
return redirect(`${TRAILING_SLASH_CANONICAL}/`, 301);
}
switch (path) {
case "/styles.css":
return new Response(STYLESHEET, {
headers: {
"content-type": "text/css; charset=utf-8",
"cache-control": "public, max-age=3600",
},
});
case "/img/placeholder.svg":
return new Response(PLACEHOLDER_SVG, {
headers: {
"content-type": "image/svg+xml; charset=utf-8",
"cache-control": "public, max-age=3600",
},
});
case "/openseo-logo.png":
return new Response(
Uint8Array.from(atob(OPENSEO_LOGO_PNG_BASE64), (c) =>
c.charCodeAt(0),
),
{
headers: {
"content-type": "image/png",
"cache-control": "public, max-age=86400",
},
},
);
case "/robots.txt":
return robotsTxt(origin);
case "/sitemap.xml":
return sitemapXml(origin);
case "/":
return html(renderHome());
case "/catalog":
return html(renderCatalog());
}
const fixture = routeTable.get(path);
if (fixture) {
const ctx: FixtureContext = { origin, request, path };
return fixture.handler(ctx);
}
return notFoundPage();
},
};

208
badseo/src/lib.ts Normal file
View File

@ -0,0 +1,208 @@
// Rendering primitives for badseo.dev.
//
// Everything the shared chrome emits (nav, footer, the "what this page tests"
// panel, the OpenSEO badge) is deliberately SEO-NEUTRAL: no <h1>–<h6> and no
// <img>. That way each fixture's headings and images are fully under the
// fixture's own control, and the audit measures exactly the defect we injected
// — not accidental noise from the layout.
import { AUDIT_ISSUE_TYPES } from "../../src/shared/audit-issues";
import type { Fixture, IssueId } from "./fixtures/types";
export function escapeHtml(input: string): string {
return input
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
/** Deterministic filler text so pages can clear the thin-content threshold. */
export function lorem(words: number): string {
const bank =
"the quick brown fox jumps over a lazy search engine while crawling deep into a sprawling website looking for signals headings titles descriptions and links that help people find genuinely useful content on the open web".split(
" ",
);
const out: string[] = [];
for (let i = 0; i < words; i++) out.push(bank[i % bank.length]);
return out.join(" ");
}
interface DocumentOptions {
/** Omit entirely to render NO <title> element (tests missing-title). */
title?: string;
/** Omit entirely to render NO meta description (tests missing-meta). */
metaDescription?: string;
/** <link rel="canonical"> href. */
canonical?: string;
/** <meta name="robots"> content. */
robotsMeta?: string;
/** Raw HTML injected at the end of <head> (extra tags, JSON-LD, etc.). */
headExtra?: string;
lang?: string;
bodyHtml: string;
}
/** Build a complete HTML document string with exact <head> control. */
export function renderDocument(opts: DocumentOptions): string {
const head: string[] = ['<meta charset="utf-8">'];
head.push(
'<meta name="viewport" content="width=device-width, initial-scale=1">',
);
if (opts.title !== undefined)
head.push(`<title>${escapeHtml(opts.title)}</title>`);
if (opts.metaDescription !== undefined)
head.push(
`<meta name="description" content="${escapeHtml(opts.metaDescription)}">`,
);
if (opts.canonical)
head.push(`<link rel="canonical" href="${escapeHtml(opts.canonical)}">`);
if (opts.robotsMeta)
head.push(`<meta name="robots" content="${escapeHtml(opts.robotsMeta)}">`);
head.push(
'<link rel="preconnect" href="https://fonts.googleapis.com">',
'<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>',
'<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap">',
);
head.push('<link rel="stylesheet" href="/styles.css">');
if (opts.headExtra) head.push(opts.headExtra);
return `<!doctype html>
<html lang="${opts.lang ?? "en"}">
<head>
${head.join("\n")}
</head>
<body>
${opts.bodyHtml}
</body>
</html>`;
}
function navHtml(): string {
return `<nav class="nav">
<a class="brand" href="/">badseo.dev</a>
<a class="nav-link" href="/catalog">Catalog</a>
</nav>`;
}
/**
* The sitewide backlink to openseo.so, pinned bottom-right on every page. The
* logo is a CSS background-image (not an <img>) so the shared chrome stays
* image-free and never affects the images-missing-alt check.
*/
function openseoBadge(): string {
return `<a class="openseo-badge" href="https://openseo.so" title="Audit a site with OpenSEO">
<span class="openseo-mark" aria-hidden="true"></span><span class="badge-label">Maintained by </span><strong>OpenSEO</strong>
</a>`;
}
function footerHtml(): string {
return `<footer class="foot"><div class="foot-inner">
<span>badseo.dev is maintained by OpenSEO. Every page here is broken on purpose.</span>
<span class="foot-links">
<a href="/catalog">Catalog</a>
<a href="https://github.com/every-app/open-seo">GitHub</a>
<a href="https://openseo.so">OpenSEO</a>
</span>
</div></footer>`;
}
/** Render the expected-issue chips, using the real audit-registry titles. */
function issueChips(issues: IssueId[]): string {
if (issues.length === 0) {
return `<span class="chip chip-clean">Should pass clean</span>`;
}
return issues
.map((id) => {
const d = AUDIT_ISSUE_TYPES[id];
return `<span class="chip chip-${d.severity}" title="${escapeHtml(
d.explanation,
)}">${escapeHtml(d.title)}</span>`;
})
.join("");
}
/**
* The on-page "what this page tests" panel. Heading-free and image-free so it
* never pollutes the signals under test. Skip it (`showPanel: false`) on the
* few pages whose word count is itself the thing being tested.
*/
function testPanel(fixture: Fixture): string {
const lesson = fixture.lesson
? `<p class="panel-lesson">${escapeHtml(fixture.lesson)}</p>`
: "";
return `<aside class="panel" aria-label="What this page tests">
<div class="panel-head">
<span class="panel-kicker">What this page tests</span>
<span class="panel-cat">${escapeHtml(fixture.category)}</span>
</div>
<p class="panel-summary">${escapeHtml(fixture.summary)}</p>
${lesson}
<div class="panel-chips">
<span class="chips-label">Audit should flag:</span>
${issueChips(fixture.expectedIssues)}
</div>
</aside>`;
}
function withChrome(inner: string): string {
return `${navHtml()}
${inner}
${footerHtml()}
${openseoBadge()}`;
}
interface PageOptions extends DocumentOptions {
fixture: Fixture;
showPanel?: boolean;
}
/** Compose chrome + optional test panel + fixture body into a full document. */
export function renderPage(opts: PageOptions): string {
const { fixture, showPanel = true, ...doc } = opts;
const panel = showPanel ? testPanel(fixture) : "";
const inner = `${panel}
<main class="main">
${doc.bodyHtml}
</main>`;
return renderDocument({ ...doc, bodyHtml: withChrome(inner) });
}
/** Chrome-wrapped page with NO test panel — for the home and catalog pages. */
export function renderShell(opts: DocumentOptions): string {
const inner = `<main class="main">
${opts.bodyHtml}
</main>`;
return renderDocument({ ...opts, bodyHtml: withChrome(inner) });
}
interface HtmlResponseOptions {
status?: number;
headers?: Record<string, string>;
/** Artificial delay before responding — tests slow-response. */
delayMs?: number;
}
export async function htmlResponse(
html: string,
opts: HtmlResponseOptions = {},
): Promise<Response> {
if (opts.delayMs && opts.delayMs > 0) {
await new Promise((resolve) => setTimeout(resolve, opts.delayMs));
}
return new Response(html, {
status: opts.status ?? 200,
headers: {
"content-type": "text/html; charset=utf-8",
"cache-control": "no-store",
...opts.headers,
},
});
}
/** A manual 3xx redirect (crawler records each hop as its own page row). */
export function redirect(location: string, status = 301): Response {
return new Response(null, {
status,
headers: { location, "cache-control": "no-store" },
});
}

5
badseo/src/logo.ts Normal file
View File

File diff suppressed because one or more lines are too long

110
badseo/src/pages.ts Normal file
View File

@ -0,0 +1,110 @@
// The two non-fixture pages: the homepage and the catalog. Both must audit
// CLEAN — they're the crawl entry point and the hub that links every fixture,
// so any accidental issue here would show up in the e2e run.
import { renderShell, escapeHtml } from "./lib";
import { AUDIT_ISSUE_TYPES } from "../../src/shared/audit-issues";
import {
allFixtures,
catalogLinkedFixtures,
categories,
duplicateUrlLinks,
fixturePaths,
} from "./fixtures/registry";
import type { Fixture } from "./fixtures/types";
const totalPaths = allFixtures.reduce((n, f) => n + fixturePaths(f).length, 0);
const distinctIssueTypes = new Set(allFixtures.flatMap((f) => f.expectedIssues))
.size;
export function renderHome(): string {
return renderShell({
title: "Technical SEO issues, by example | badseo.dev",
metaDescription:
"Every page here has one common technical SEO issue, from a missing title to a redirect loop, so you can test what your SEO crawler catches.",
bodyHtml: `<section class="hero">
<h1>A website demonstrating common technical SEO problems</h1>
<p class="lede">Every page on this site has one thing wrong with it, on purpose: a missing title, a redirect loop, a page nothing links to. Point an SEO crawler at the site and see which problems it catches.</p>
</section>
<div class="stat-row">
<div class="stat"><b>${allFixtures.length}</b><span>broken pages</span></div>
<div class="stat"><b>${distinctIssueTypes}</b><span>issue types</span></div>
<div class="stat"><b>${totalPaths}</b><span>crawlable URLs</span></div>
</div>
<h2>What's on it</h2>
<p>Each page breaks one thing and is otherwise fine, so an audit sees a single problem instead of a pile of them. Every page shows what it is testing and which issue an audit should report. Start with the <a href="/catalog">catalog</a>, or open the <a href="/kitchen-sink">kitchen-sink page</a>, which breaks six ways at once.</p>
<h2>Why it exists</h2>
<p>SEO tools all say they find broken titles, duplicate pages, redirect chains, and thin content. It is hard to check that they actually do. badseo.dev gives you a site that is broken in known ways, so you can run a crawler against it and compare what it finds to what is really there. We use it to test the OpenSEO audit.</p>
<h2>It's open source</h2>
<p>The site is open source. If there is a common SEO mistake it does not cover yet, you can add it. Each page is one small file that lists the issues it should trigger, so a new page also works as a test. It is maintained by the team behind <a href="https://openseo.so">OpenSEO</a>, an open-source SEO tool.</p>
<h2>Run an audit</h2>
<p><a href="https://openseo.so">OpenSEO</a> can crawl badseo.dev and report the issues on each page, along with how to fix them. It is a straightforward way to see what a crawler catches.</p>`,
});
}
function issueDots(fixture: Fixture): string {
return fixture.expectedIssues
.map((id) => {
const sev = AUDIT_ISSUE_TYPES[id].severity;
return `<span class="dot dot-${sev}" title="${escapeHtml(
AUDIT_ISSUE_TYPES[id].title,
)}"></span>`;
})
.join("");
}
function indexRow(fixture: Fixture): string {
return `<a class="index-row" href="${escapeHtml(fixture.path)}">
<span class="row-name">${escapeHtml(fixture.name)}</span>
<span class="row-sum">${escapeHtml(fixture.summary)}</span>
<span class="row-sev">${issueDots(fixture)}</span>
</a>`;
}
export function renderCatalog(): string {
const groups = categories
.map((category) => {
const rows = catalogLinkedFixtures
.filter((f) => f.category === category)
.map(indexRow)
.join("\n");
return `<section class="index-group">
<h2>${escapeHtml(category)}</h2>
<div class="index-list">
${rows}
</div>
</section>`;
})
.join("\n");
return renderShell({
title: "Technical SEO issues checklist | badseo.dev",
metaDescription:
"A checklist of common technical SEO issues, each with a live example page: head tags, duplicate content, redirects, HTTP status, speed, and site structure.",
bodyHtml: `<h1>Technical SEO issues, by category</h1>
<p class="lede">Every page on the site, grouped by the kind of technical SEO issue it shows. The dots are the issues an audit should report for that page.</p>
<p class="legend">
<span><span class="dot dot-critical"></span> critical</span>
<span><span class="dot dot-warning"></span> warning</span>
<span><span class="dot dot-info"></span> info</span>
</p>
${groups}
<section class="index-group">
<h2>Duplicate URLs</h2>
<p>The same page served again at a second address. This is the raw material for the duplicate-content check. They are linked here so a crawler reaches them and does not mistake them for orphans.</p>
<p>${duplicateUrlLinks
.map(
(d) =>
`<a href="${escapeHtml(d.path)}"><code>${escapeHtml(
d.path,
)}</code></a>`,
)
.join(" &middot; ")}</p>
</section>
<p style="margin-top:32px">A few pages are left off this list on purpose. An orphan page (<code>/structure/orphan</code>) is only in the sitemap, and some 404, 500, and 403 URLs are only in the sitemap too, so a crawler has to find them on its own.</p>`,
});
}

216
badseo/src/styles.ts Normal file
View File

@ -0,0 +1,216 @@
// Served verbatim at /styles.css. Kept as a module string so the Worker stays
// dependency-free. Palette + type mirror the OpenSEO marketing site (web/):
// a warm "cream" canvas, ink text, one orange accent, Inter, hairline borders,
// small radii, no shadows.
export const STYLESHEET = `
:root {
--canvas: #f5f1ec;
--surface: #ffffff;
--surface-2: #ebe7e1;
--ink: #111111;
--ink-muted: #626260;
--ink-subtle: #7b7b78;
--hairline: #d8d1c8;
--hairline-soft: #ebe7e1;
--orange: #ff5600;
--footer-bg: #eee8de;
--critical: #d23b1f;
--warning: #b26a00;
--info: #6b7280;
--mono: "JetBrains Mono", ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
--sans: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
}
* { box-sizing: border-box; }
html { -webkit-text-size-adjust: 100%; }
body {
margin: 0;
background: var(--canvas);
color: var(--ink);
font-family: var(--sans);
font-size: 16px;
line-height: 1.5;
-webkit-font-smoothing: antialiased;
text-rendering: optimizeLegibility;
}
a { color: var(--ink); text-decoration: none; }
a:hover { text-decoration: underline; text-underline-offset: 3px; }
.wrap, .main, .panel, .nav, .foot-inner { max-width: 820px; margin: 0 auto; padding-left: 24px; padding-right: 24px; }
/* ── nav ── */
.nav {
display: flex;
align-items: center;
gap: 20px;
padding-top: 20px;
padding-bottom: 20px;
}
.brand {
font-weight: 600;
letter-spacing: -0.01em;
color: var(--ink);
font-size: 17px;
}
.brand:hover { text-decoration: none; }
.nav-link { color: var(--ink-muted); font-weight: 500; margin-left: auto; font-size: 15px; }
.nav-link:hover { color: var(--ink); }
/* ── content ── */
.main { padding-top: 8px; padding-bottom: 48px; }
.main h1 {
font-size: clamp(30px, 4.6vw, 46px);
font-weight: 500;
line-height: 1.1;
letter-spacing: -0.025em;
margin: 12px 0 18px;
text-wrap: balance;
}
.main h2 {
font-size: 24px; font-weight: 500; letter-spacing: -0.017em;
margin: 40px 0 12px;
}
.main h3 { font-size: 19px; font-weight: 500; margin: 26px 0 8px; }
.main h4 { font-size: 16px; font-weight: 600; margin: 20px 0 6px; }
.main p { color: #2c2c2b; margin: 0 0 16px; }
.lede { font-size: 19px; line-height: 1.45; letter-spacing: -0.006em; color: var(--ink); }
.main a { color: var(--ink); text-decoration: underline; text-decoration-color: var(--hairline); text-underline-offset: 3px; }
.main a:hover { text-decoration-color: var(--ink); }
.main img {
max-width: 100%; border-radius: 10px; border: 1px solid var(--hairline);
display: block; margin: 20px 0; background: var(--surface);
}
.main code {
font-family: var(--mono); font-size: 13.5px;
background: var(--surface-2); padding: 2px 6px; border-radius: 5px;
}
.main ul, .main ol { color: #2c2c2b; padding-left: 20px; }
.main li { margin: 4px 0; }
.next { margin-top: 8px; }
.next a { font-weight: 500; }
/* ── test panel ── */
.panel {
margin-top: 8px;
margin-bottom: 32px;
background: var(--surface);
border: 1px solid var(--hairline);
border-radius: 12px;
padding: 18px 20px;
}
.panel-head { display: flex; align-items: baseline; gap: 10px; flex-wrap: wrap; }
.panel-kicker {
text-transform: uppercase; letter-spacing: 0.08em; font-size: 11px;
font-weight: 600; color: var(--ink-subtle);
}
.panel-cat {
margin-left: auto; font-size: 12px; color: var(--ink-subtle);
font-family: var(--mono);
}
.panel-summary { margin: 10px 0 0; font-size: 15px; color: var(--ink); }
.panel-lesson { margin: 6px 0 0; font-size: 14px; color: var(--ink-muted); }
.panel-chips { margin-top: 14px; display: flex; flex-wrap: wrap; gap: 8px; align-items: center; }
.chips-label { font-size: 12px; color: var(--ink-subtle); font-weight: 500; margin-right: 2px; }
.chip {
display: inline-flex; align-items: center; gap: 7px;
font-size: 12.5px; font-weight: 500; color: var(--ink);
padding: 3px 11px 3px 9px; border-radius: 999px;
background: var(--canvas); border: 1px solid var(--hairline);
}
.chip::before {
content: ""; width: 7px; height: 7px; border-radius: 50%; flex: none;
background: var(--info);
}
.chip-critical::before { background: var(--critical); }
.chip-warning::before { background: var(--warning); }
.chip-info::before { background: var(--info); }
.chip-clean { color: var(--ink-muted); }
.chip-clean::before { background: #3a9d5d; }
/* ── hero + home bits ── */
.hero { padding: 20px 0 4px; }
.stat-row { display: flex; gap: 40px; flex-wrap: wrap; margin: 26px 0 8px; }
.stat b { font-size: 30px; font-weight: 500; letter-spacing: -0.02em; display: block; line-height: 1.1; }
.stat span { color: var(--ink-muted); font-size: 14px; }
.callout {
background: var(--surface); border: 1px solid var(--hairline);
border-radius: 12px; padding: 16px 18px; margin: 24px 0;
}
.callout p { margin: 0; color: #2c2c2b; }
/* ── catalog index (no cards) ── */
.index-group { margin: 40px 0; }
.index-group > h2 { margin-bottom: 4px; }
.index-list { margin-top: 8px; border-top: 1px solid var(--hairline); }
.index-list .index-row {
display: grid;
grid-template-columns: minmax(160px, 240px) 1fr auto;
gap: 20px;
align-items: baseline;
padding: 14px 12px;
margin: 0 -12px;
border-radius: 8px;
border-bottom: 1px solid var(--hairline);
color: var(--ink);
text-decoration: none;
}
.index-list .index-row:hover {
background: var(--surface-2);
border-bottom-color: transparent;
}
.index-list .index-row:hover .row-name {
text-decoration: underline;
text-decoration-color: var(--ink);
text-underline-offset: 3px;
}
.row-name { font-weight: 500; }
.row-sum { color: var(--ink-muted); font-size: 14.5px; }
.row-sev { display: inline-flex; gap: 5px; align-items: center; padding-top: 4px; }
.dot { width: 8px; height: 8px; border-radius: 50%; display: inline-block; flex: none; }
.dot-critical { background: var(--critical); }
.dot-warning { background: var(--warning); }
.dot-info { background: var(--info); }
.legend { display: inline-flex; gap: 14px; flex-wrap: wrap; align-items: center; color: var(--ink-muted); font-size: 14px; }
.legend span { display: inline-flex; gap: 6px; align-items: center; }
@media (max-width: 640px) {
.index-row { grid-template-columns: 1fr auto; }
.row-sum { grid-column: 1 / -1; }
}
/* ── footer + OpenSEO badge ── */
/* The band fills to the bottom of the page (no body padding beneath it). Extra
bottom padding keeps the footer text clear of the fixed badge, which floats
in the empty band space below the row. */
.foot { background: var(--footer-bg); border-top: 1px solid var(--hairline); margin-top: 64px; }
.foot-inner {
padding: 26px 24px 76px; color: var(--ink-muted); font-size: 14px;
display: flex; justify-content: space-between; gap: 8px 24px; flex-wrap: wrap;
}
.foot-inner a { color: var(--ink-muted); }
.foot-inner a:hover { color: var(--ink); }
.foot-links { display: flex; gap: 18px; }
.openseo-badge {
position: fixed; right: 20px; bottom: 20px; z-index: 50;
display: inline-flex; align-items: center; gap: 8px;
background: var(--ink); color: #ffffff;
border-radius: 999px; padding: 11px 17px;
font-size: 14px; font-weight: 500;
}
.openseo-badge:hover { background: #000000; text-decoration: none; }
.openseo-badge strong { font-weight: 600; }
.openseo-mark {
width: 22px; height: 24px; flex: none;
background: url("/openseo-logo.png") center / contain no-repeat;
/* The source logo is a silver tree on transparent; render it white so it
reads on the dark pill with no backing chip. */
filter: brightness(0) invert(1);
}
@media (max-width: 640px) {
.openseo-badge .badge-label { display: none; }
.openseo-badge { padding: 10px 12px; }
}
`;

19
badseo/tsconfig.json Normal file
View File

@ -0,0 +1,19 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"lib": ["ES2022"],
"types": ["@cloudflare/workers-types"],
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"isolatedModules": true,
"noEmit": true,
"forceConsistentCasingInFileNames": true
},
// Only the Worker source is typechecked here. scripts/run-audit.ts imports
// the main app via its own path aliases and is run with tsx from the repo
// root, so it isn't part of this project's typecheck.
"include": ["src/**/*.ts"]
}

26
badseo/wrangler.jsonc Normal file
View File

@ -0,0 +1,26 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
"name": "badseo",
"main": "src/index.ts",
"compatibility_date": "2025-06-01",
"workers_dev": true,
"preview_urls": true,
"observability": {
"enabled": true,
},
// Custom-domain routes live under the `production` env so that plain
// `wrangler dev` serves on localhost (a top-level custom_domain route makes
// dev simulate the production host, which breaks local crawling).
// Deploy with: wrangler deploy --env production
"env": {
"production": {
"name": "badseo",
"workers_dev": true,
"observability": { "enabled": true },
"routes": [
{ "pattern": "badseo.dev", "custom_domain": true },
{ "pattern": "www.badseo.dev", "custom_domain": true },
],
},
},
}

9
knip.jsonc
View File

@ -23,7 +23,14 @@
// can't trace // can't trace
"src/server/lib/dataforseo/sections.ts", "src/server/lib/dataforseo/sections.ts",
], ],
"project": ["**/*.{js,mjs,ts,tsx}", "!src/routeTree.gen.ts", "!web/**"], // badseo/ is the standalone broken-SEO fixture worker (own deps/config),
// like web/ it isn't part of the app's module graph.
"project": [
"**/*.{js,mjs,ts,tsx}",
"!src/routeTree.gen.ts",
"!web/**",
"!badseo/**",
],
"ignore": ["drizzle-prod.config.ts"], "ignore": ["drizzle-prod.config.ts"],
// Disable Drizzle plugin - it tries to load drizzle.config.ts which imports cloudflare:workers // Disable Drizzle plugin - it tries to load drizzle.config.ts which imports cloudflare:workers
"drizzle": false, "drizzle": false,

11
skills-lock.json Normal file
View File

@ -0,0 +1,11 @@
{
"version": 1,
"skills": {
"webapp-testing": {
"source": "anthropics/skills",
"sourceType": "github",
"skillPath": "skills/webapp-testing/SKILL.md",
"computedHash": "ad5b1fc52807e9afa4635e59218a026b164dc58b6c3f41b0f7c644dcd6ccf572"
}
}
}

271
src/client/features/audit/results/IssuesView.tsx Normal file
View File

@ -0,0 +1,271 @@
import { useMemo, useState } from "react";
import { ChevronRight } from "lucide-react";
import {
getIssueDescriptor,
ISSUE_SEVERITY_ORDER,
type IssueSeverity,
} from "@/shared/audit-issues";
import type { AuditResultsData } from "@/client/features/audit/results/types";
type AuditIssueRow = AuditResultsData["issues"][number];
const MAX_RENDERED_URLS = 100;
const SEVERITY_DOT: Record<IssueSeverity, string> = {
critical: "bg-error",
warning: "bg-warning",
info: "bg-base-content/30",
};
const SEVERITY_RULE: Record<IssueSeverity, string> = {
critical: "border-l-error/60",
warning: "border-l-warning/60",
info: "border-l-base-content/20",
};
const SEVERITY_LABEL: Record<IssueSeverity, string> = {
critical: "Critical",
warning: "Warning",
info: "Info",
};
interface IssueGroup {
issueType: string;
severity: IssueSeverity;
title: string;
explanation: string;
howToFix: string;
issues: AuditIssueRow[];
}
export function resolveIssueSeverity(issue: {
issueType: string;
severity: string;
}): IssueSeverity {
const descriptor = getIssueDescriptor(issue.issueType);
if (descriptor) return descriptor.severity;
return issue.severity === "critical" || issue.severity === "warning"
? issue.severity
: "info";
}
function groupIssues(issues: AuditIssueRow[]): IssueGroup[] {
const groups = new Map<string, IssueGroup>();
for (const issue of issues) {
let group = groups.get(issue.issueType);
if (!group) {
const descriptor = getIssueDescriptor(issue.issueType);
group = {
issueType: issue.issueType,
severity: resolveIssueSeverity(issue),
title: descriptor?.title ?? issue.issueType,
explanation: descriptor?.explanation ?? "",
howToFix: descriptor?.howToFix ?? "",
issues: [],
};
groups.set(issue.issueType, group);
}
group.issues.push(issue);
}
return Array.from(groups.values()).toSorted(
(a, b) =>
ISSUE_SEVERITY_ORDER[a.severity] - ISSUE_SEVERITY_ORDER[b.severity] ||
b.issues.length - a.issues.length,
);
}
export function IssuesView({ issues }: { issues: AuditIssueRow[] }) {
const groups = useMemo(() => groupIssues(issues), [issues]);
const sections = useMemo(
() =>
(["critical", "warning", "info"] as const)
.map((severity) => ({
severity,
groups: groups.filter((group) => group.severity === severity),
}))
.filter((section) => section.groups.length > 0),
[groups],
);
if (issues.length === 0) {
return (
<div className="py-10 text-center text-base-content/60">
<p className="font-medium">No issues recorded for this audit.</p>
<p className="text-sm mt-1">
Either the site is in great shape, or this audit ran before issue
checks existed — run a new audit to get the full report.
</p>
</div>
);
}
return (
<div className="border border-base-300 rounded-lg overflow-hidden">
{sections.map((section) => (
<IssueSection key={section.severity} section={section} />
))}
</div>
);
}
function IssueSection({
section,
}: {
section: { severity: IssueSeverity; groups: IssueGroup[] };
}) {
const issueCount = section.groups.reduce(
(sum, group) => sum + group.issues.length,
0,
);
return (
<div className="border-t border-base-300 first:border-t-0">
<div className="flex items-center gap-2 bg-base-200/60 px-4 py-1.5 border-b border-base-300/60">
<span
className={`size-1.5 rounded-full ${SEVERITY_DOT[section.severity]}`}
/>
<span className="text-[11px] font-semibold uppercase tracking-wider text-base-content/60">
{SEVERITY_LABEL[section.severity]}
</span>
<span className="text-[11px] tabular-nums text-base-content/40">
{issueCount}
</span>
</div>
<div className="divide-y divide-base-300/60">
{section.groups.map((group) => (
<IssueRow key={group.issueType} group={group} />
))}
</div>
</div>
);
}
function IssueRow({ group }: { group: IssueGroup }) {
const [open, setOpen] = useState(false);
return (
<div
className={
open
? `border-l-2 ${SEVERITY_RULE[group.severity]} bg-base-200/20`
: "border-l-2 border-l-transparent"
}
>
<button
type="button"
className="w-full flex items-center gap-3 px-4 py-2.5 text-left hover:bg-base-200/40 transition-colors"
onClick={() => setOpen((value) => !value)}
aria-expanded={open}
>
<span
className={`size-2 shrink-0 rounded-full ${SEVERITY_DOT[group.severity]}`}
/>
<span className="text-sm font-medium flex-1 min-w-0 truncate">
{group.title}
</span>
<span className="text-xs tabular-nums text-base-content/50 shrink-0">
{group.issues.length} {group.issues.length === 1 ? "page" : "pages"}
</span>
<ChevronRight
className={`size-4 shrink-0 text-base-content/40 transition-transform ${
open ? "rotate-90" : ""
}`}
/>
</button>
{open && (
<div className="pl-9 pr-4 pb-4 pt-0.5 space-y-3">
{group.explanation && (
<p className="text-sm text-base-content/70 max-w-prose">
{group.explanation}
</p>
)}
{group.howToFix && (
<p className="text-sm max-w-prose">
<span className="font-medium">How to fix: </span>
<span className="text-base-content/80">{group.howToFix}</span>
</p>
)}
<AffectedUrlList issues={group.issues} />
</div>
)}
</div>
);
}
function AffectedUrlList({ issues }: { issues: AuditIssueRow[] }) {
const rendered = issues.slice(0, MAX_RENDERED_URLS);
const remaining = issues.length - rendered.length;
return (
<div className="max-h-[320px] overflow-y-auto rounded border border-base-300/60 bg-base-100">
{rendered.map((issue) => (
<div
key={issue.id}
className="px-3 py-1.5 text-sm flex flex-col gap-0.5 border-b border-base-300/50 last:border-b-0"
>
<a
className="link link-hover text-base-content/80 truncate"
href={issue.pageUrl}
target="_blank"
rel="noreferrer"
title={issue.pageUrl}
>
{issue.pageUrl}
</a>
<IssueDetails detailsJson={issue.detailsJson} />
</div>
))}
{remaining > 0 && (
<div className="px-3 py-2 text-xs text-base-content/50">
…and {remaining} more — export the issues CSV for the full list.
</div>
)}
</div>
);
}
function parseDetails(detailsJson: string): Array<[string, unknown]> | null {
try {
const parsed: unknown = JSON.parse(detailsJson);
if (
typeof parsed === "object" &&
parsed !== null &&
!Array.isArray(parsed)
) {
return Object.entries(parsed);
}
return null;
} catch {
return null;
}
}
function IssueDetails({ detailsJson }: { detailsJson: string | null }) {
const details = useMemo(
() => (detailsJson ? parseDetails(detailsJson) : null),
[detailsJson],
);
if (!details) return null;
const entries = details.filter(
([, value]) => value !== null && value !== undefined,
);
if (entries.length === 0) return null;
return (
<span className="text-xs text-base-content/50 truncate">
{entries
.map(([key, value]) => {
const rendered = Array.isArray(value)
? value.join(" → ")
: String(value);
return `${key}: ${rendered}`;
})
.join(" · ")}
</span>
);
}

252
src/client/features/audit/results/PagesTable.tsx Normal file
View File

@ -0,0 +1,252 @@
import { useMemo, useState } from "react";
import {
createColumnHelper,
type ColumnDef,
type SortingState,
} from "@tanstack/react-table";
import { ExternalLink } from "lucide-react";
import {
AppDataTable,
useAppTable,
} from "@/client/components/table/AppDataTable";
import { SortableHeader } from "@/client/components/table/SortableHeader";
import {
extractHostname,
extractPathname,
HttpStatusBadge,
} from "@/client/features/audit/shared";
import type { AuditResultsData } from "@/client/features/audit/results/types";
import {
countActiveFilters,
EmptyTableMessage,
PagesFilterBar,
TableFilterToggle,
} from "@/client/features/audit/results/AuditResultsTableFilters";
import {
EMPTY_PAGES_FILTERS,
filterPages,
nullableNumberSort,
nullableStringSort,
type PageRow,
type PagesFilters,
} from "@/client/features/audit/results/AuditResultsTableFilterLogic";
const pageColumnHelper = createColumnHelper<PageRow>();
/**
* Path shown in the URL/redirect cells. Redirect sources on another host
* (e.g. the apex domain 301ing to www) would otherwise render identically
* to their target, so include the host whenever it differs from the
* site's canonical host.
*/
function displayPath(url: string, canonicalHost: string): string {
const host = extractHostname(url);
const path = extractPathname(url);
return host === canonicalHost ? path : host + path;
}
/**
* The host most of the site's real (2xx) pages live on. The start URL's host
* is only a fallback: audits often start from the apex domain of a site that
* canonicalizes to www, and prefixing every row with the host is exactly the
* noise this display is meant to avoid.
*/
function predominantHost(pages: PageRow[], startUrl: string): string {
const counts = new Map<string, number>();
for (const page of pages) {
if (page.statusCode === null || page.statusCode >= 300) continue;
const host = extractHostname(page.url);
counts.set(host, (counts.get(host) ?? 0) + 1);
}
let best = extractHostname(startUrl);
let bestCount = 0;
for (const [host, count] of counts) {
if (count > bestCount) {
best = host;
bestCount = count;
}
}
return best;
}
function isRedirect(row: PageRow): boolean {
return (
row.statusCode !== null && row.statusCode >= 300 && row.statusCode < 400
);
}
/** Redirects and blocked/errored fetches have no analyzed content — their
* zero H1/word/image counts are an artifact, not a finding. */
function hasAnalyzedContent(row: PageRow): boolean {
return row.fetchClass === "ok" && !isRedirect(row);
}
const EmptyCell = () => <span className="text-xs text-base-content/40">-</span>;
function buildPagesColumns({
canonicalHost,
missingTitlePageIds,
}: {
canonicalHost: string;
missingTitlePageIds: Set<string>;
}): ColumnDef<PageRow>[] {
return [
pageColumnHelper.accessor("url", {
header: ({ column }) => <SortableHeader column={column} label="URL" />,
cell: ({ getValue }) => {
const url = getValue();
return (
<a
href={url}
target="_blank"
rel="noopener noreferrer"
className="link link-primary inline-flex items-center gap-1 text-xs"
>
<span className="truncate">{displayPath(url, canonicalHost)}</span>
<ExternalLink className="size-3 shrink-0" />
</a>
);
},
meta: { cellClassName: "max-w-[240px] truncate" },
}),
pageColumnHelper.accessor("statusCode", {
header: ({ column }) => <SortableHeader column={column} label="Status" />,
cell: ({ getValue }) => <HttpStatusBadge code={getValue()} />,
sortingFn: nullableNumberSort,
}),
pageColumnHelper.accessor("title", {
header: ({ column }) => <SortableHeader column={column} label="Title" />,
cell: ({ getValue, row }) => {
if (isRedirect(row.original)) {
const target = row.original.redirectUrl;
return (
<span className="text-xs text-base-content/60">
→ {target ? displayPath(target, canonicalHost) : "redirect"}
</span>
);
}
const title = getValue();
if (title) {
return <span className="break-words">{title}</span>;
}
// Red only when the engine flagged it — a 200 that isn't an HTML
// document (robots.txt, security.txt) legitimately has no title.
return missingTitlePageIds.has(row.original.id) ? (
<span className="text-error text-xs">missing</span>
) : (
<EmptyCell />
);
},
sortingFn: nullableStringSort,
meta: { cellClassName: "max-w-[360px]" },
}),
pageColumnHelper.accessor("h1Count", {
header: ({ column }) => <SortableHeader column={column} label="H1" />,
cell: ({ getValue, row }) =>
hasAnalyzedContent(row.original) ? getValue() : <EmptyCell />,
}),
pageColumnHelper.accessor("wordCount", {
header: ({ column }) => <SortableHeader column={column} label="Words" />,
cell: ({ getValue, row }) =>
hasAnalyzedContent(row.original) ? getValue() : <EmptyCell />,
}),
pageColumnHelper.display({
id: "images",
header: ({ column }) => <SortableHeader column={column} label="Images" />,
cell: ({ row }) => {
if (!hasAnalyzedContent(row.original)) return <EmptyCell />;
return row.original.imagesMissingAlt > 0 ? (
<span className="text-warning">
{row.original.imagesMissingAlt}/{row.original.imagesTotal}
</span>
) : (
row.original.imagesTotal
);
},
enableSorting: true,
sortingFn: (left, right) =>
left.original.imagesMissingAlt - right.original.imagesMissingAlt ||
left.original.imagesTotal - right.original.imagesTotal,
}),
pageColumnHelper.accessor("responseTimeMs", {
header: ({ column }) => <SortableHeader column={column} label="Speed" />,
cell: ({ getValue }) => {
const value = getValue();
return value ? (
<span className="text-xs">{value}ms</span>
) : (
<EmptyCell />
);
},
sortingFn: nullableNumberSort,
}),
];
}
export function PagesTable({
pages,
startUrl,
issues,
}: {
pages: AuditResultsData["pages"];
startUrl: string;
issues: AuditResultsData["issues"];
}) {
const [filters, setFilters] = useState<PagesFilters>(EMPTY_PAGES_FILTERS);
const [showFilters, setShowFilters] = useState(false);
// URL order reads as a site inventory; status-first would open the table
// on its most boring rows (redirects) whenever a site has no errors.
const [sorting, setSorting] = useState<SortingState>([
{ id: "url", desc: false },
]);
const activeFilterCount = countActiveFilters(filters, EMPTY_PAGES_FILTERS);
const filteredPages = useMemo(
() => filterPages(pages, filters),
[filters, pages],
);
const columns = useMemo(
() =>
buildPagesColumns({
canonicalHost: predominantHost(pages, startUrl),
missingTitlePageIds: new Set(
issues
.filter((issue) => issue.issueType === "missing-title")
.map((issue) => issue.pageId)
.filter((pageId): pageId is string => pageId !== null),
),
}),
[issues, pages, startUrl],
);
const table = useAppTable({
data: filteredPages,
columns,
state: { sorting },
onSortingChange: setSorting,
withSorting: true,
});
return (
<div className="space-y-3">
<TableFilterToggle
showFilters={showFilters}
onToggle={() => setShowFilters((current) => !current)}
activeFilterCount={activeFilterCount}
resultCount={filteredPages.length}
totalCount={pages.length}
/>
{showFilters ? (
<PagesFilterBar
filters={filters}
onChange={setFilters}
activeFilterCount={activeFilterCount}
onReset={() => setFilters(EMPTY_PAGES_FILTERS)}
/>
) : null}
<AppDataTable
table={table}
className="table table-sm"
empty={<EmptyTableMessage label="No pages match these filters." />}
/>
</div>
);
}

126
src/client/features/audit/results/ResultsTables.tsx
View File

@ -5,7 +5,6 @@ import {
type SortingState, type SortingState,
} from "@tanstack/react-table"; } from "@tanstack/react-table";
import { Link } from "@tanstack/react-router"; import { Link } from "@tanstack/react-router";
import { ExternalLink } from "lucide-react";
import { import {
AppDataTable, AppDataTable,
useAppTable, useAppTable,
@ -14,152 +13,27 @@ import { TableExportMenu } from "@/client/components/table/TableBulkActionBar";
import { SortableHeader } from "@/client/components/table/SortableHeader"; import { SortableHeader } from "@/client/components/table/SortableHeader";
import { import {
extractPathname, extractPathname,
HttpStatusBadge,
LighthouseScoreBadge, LighthouseScoreBadge,
} from "@/client/features/audit/shared"; } from "@/client/features/audit/shared";
import type { AuditResultsData } from "@/client/features/audit/results/types"; import type { AuditResultsData } from "@/client/features/audit/results/types";
import { import {
countActiveFilters, countActiveFilters,
EmptyTableMessage, EmptyTableMessage,
PagesFilterBar,
PerformanceFilterBar, PerformanceFilterBar,
TableFilterToggle, TableFilterToggle,
} from "@/client/features/audit/results/AuditResultsTableFilters"; } from "@/client/features/audit/results/AuditResultsTableFilters";
import { import {
EMPTY_PAGES_FILTERS,
EMPTY_PERFORMANCE_FILTERS, EMPTY_PERFORMANCE_FILTERS,
filterPages,
filterPerformanceRows, filterPerformanceRows,
isLighthouseFailure, isLighthouseFailure,
nullableNumberSort, nullableNumberSort,
nullableStringSort, nullableStringSort,
type PageRow,
type PagesFilters,
type PerformanceFilters, type PerformanceFilters,
type PerformanceRowData, type PerformanceRowData,
} from "@/client/features/audit/results/AuditResultsTableFilterLogic"; } from "@/client/features/audit/results/AuditResultsTableFilterLogic";
const pageColumnHelper = createColumnHelper<PageRow>();
const performanceColumnHelper = createColumnHelper<PerformanceRowData>(); const performanceColumnHelper = createColumnHelper<PerformanceRowData>();
const pagesColumns: ColumnDef<PageRow>[] = [
pageColumnHelper.accessor("url", {
header: ({ column }) => <SortableHeader column={column} label="URL" />,
cell: ({ getValue }) => {
const url = getValue();
return (
<a
href={url}
target="_blank"
rel="noopener noreferrer"
className="link link-primary inline-flex items-center gap-1 text-xs"
>
<span className="truncate">{extractPathname(url)}</span>
<ExternalLink className="size-3 shrink-0" />
</a>
);
},
meta: { cellClassName: "max-w-[240px] truncate" },
}),
pageColumnHelper.accessor("statusCode", {
header: ({ column }) => <SortableHeader column={column} label="Status" />,
cell: ({ getValue }) => <HttpStatusBadge code={getValue()} />,
sortingFn: nullableNumberSort,
}),
pageColumnHelper.accessor("title", {
header: ({ column }) => <SortableHeader column={column} label="Title" />,
cell: ({ getValue }) => {
const title = getValue();
return title ? (
<span title={title}>{title}</span>
) : (
<span className="text-error text-xs">missing</span>
);
},
sortingFn: nullableStringSort,
meta: { cellClassName: "max-w-[220px] truncate" },
}),
pageColumnHelper.accessor("h1Count", {
header: ({ column }) => <SortableHeader column={column} label="H1" />,
}),
pageColumnHelper.accessor("wordCount", {
header: ({ column }) => <SortableHeader column={column} label="Words" />,
}),
pageColumnHelper.display({
id: "images",
header: ({ column }) => <SortableHeader column={column} label="Images" />,
cell: ({ row }) =>
row.original.imagesMissingAlt > 0 ? (
<span className="text-warning">
{row.original.imagesMissingAlt}/{row.original.imagesTotal}
</span>
) : (
row.original.imagesTotal
),
enableSorting: true,
sortingFn: (left, right) =>
left.original.imagesMissingAlt - right.original.imagesMissingAlt ||
left.original.imagesTotal - right.original.imagesTotal,
}),
pageColumnHelper.accessor("responseTimeMs", {
header: ({ column }) => <SortableHeader column={column} label="Speed" />,
cell: ({ getValue }) => {
const value = getValue();
return value ? (
<span className="text-xs">{value}ms</span>
) : (
<span className="text-xs text-base-content/40">-</span>
);
},
sortingFn: nullableNumberSort,
}),
];
export function PagesTable({ pages }: { pages: AuditResultsData["pages"] }) {
const [filters, setFilters] = useState<PagesFilters>(EMPTY_PAGES_FILTERS);
const [showFilters, setShowFilters] = useState(false);
const [sorting, setSorting] = useState<SortingState>([
{ id: "statusCode", desc: true },
]);
const activeFilterCount = countActiveFilters(filters, EMPTY_PAGES_FILTERS);
const filteredPages = useMemo(
() => filterPages(pages, filters),
[filters, pages],
);
const table = useAppTable({
data: filteredPages,
columns: pagesColumns,
state: { sorting },
onSortingChange: setSorting,
withSorting: true,
});
return (
<div className="space-y-3">
<TableFilterToggle
showFilters={showFilters}
onToggle={() => setShowFilters((current) => !current)}
activeFilterCount={activeFilterCount}
resultCount={filteredPages.length}
totalCount={pages.length}
/>
{showFilters ? (
<PagesFilterBar
filters={filters}
onChange={setFilters}
activeFilterCount={activeFilterCount}
onReset={() => setFilters(EMPTY_PAGES_FILTERS)}
/>
) : null}
<AppDataTable
table={table}
className="table table-sm"
empty={<EmptyTableMessage label="No pages match these filters." />}
/>
</div>
);
}
export function PerformanceTable({ export function PerformanceTable({
auditId, auditId,
projectId, projectId,

267
src/client/features/audit/results/ResultsView.tsx
View File

@ -1,18 +1,23 @@
import { useMemo } from "react"; import { useMemo, type ReactNode } from "react";
import { StatCard } from "@/client/features/audit/shared"; import { ShieldAlert } from "lucide-react";
import { import {
exportIssues,
exportPages, exportPages,
exportPerformance, exportPerformance,
} from "@/client/features/audit/results/export"; } from "@/client/features/audit/results/export";
import type { AuditResultsData } from "@/client/features/audit/results/types"; import type { AuditResultsData } from "@/client/features/audit/results/types";
import { isLighthouseFailure } from "@/client/features/audit/results/AuditResultsTableFilterLogic"; import { isLighthouseFailure } from "@/client/features/audit/results/AuditResultsTableFilterLogic";
import {
IssuesView,
resolveIssueSeverity,
} from "@/client/features/audit/results/IssuesView";
import { PagesTable } from "@/client/features/audit/results/PagesTable";
import { import {
ExportDropdown, ExportDropdown,
PagesTable,
PerformanceTable, PerformanceTable,
} from "@/client/features/audit/results/ResultsTables"; } from "@/client/features/audit/results/ResultsTables";
type ResultsTab = "pages" | "performance"; type ResultsTab = "issues" | "pages" | "performance";
export function ResultsView({ export function ResultsView({
projectId, projectId,
@ -25,16 +30,39 @@ export function ResultsView({
tab: string; tab: string;
onTabChange: (tab: ResultsTab) => void; onTabChange: (tab: ResultsTab) => void;
}) { }) {
const { audit, pages, lighthouse } = data; const { audit, pages, lighthouse, issues } = data;
const hasPerformanceTab = lighthouse.length > 0; const hasPerformanceTab = lighthouse.length > 0;
const activeTab = hasPerformanceTab ? tab : "pages"; const activeTab =
tab === "performance" && !hasPerformanceTab ? "issues" : tab;
const stats = useResultStats(pages, lighthouse); const stats = useResultStats(pages, lighthouse);
const blockedCount = useMemo(
() => pages.filter((page) => page.fetchClass === "blocked").length,
[pages],
);
return ( return (
<> <>
<StatsGrid {blockedCount > 0 && (
<div className="flex items-start gap-3 rounded-lg border border-warning/30 bg-warning/5 px-4 py-3 text-sm">
<ShieldAlert className="mt-0.5 size-4 shrink-0 text-warning" />
<p>
<span className="font-medium">
We were blocked on {blockedCount}{" "}
{blockedCount === 1 ? "page" : "pages"}.
</span>{" "}
<span className="text-base-content/70">
The site's bot protection challenged our crawler, so those pages
couldn't be audited. If this is your site, allowlist the{" "}
<code className="font-mono">OpenSEO-Audit</code> user agent in
your WAF or bot-protection settings and re-run the audit.
</span>
</p>
</div>
)}
<StatsStrip
pagesCrawled={audit.pagesCrawled} pagesCrawled={audit.pagesCrawled}
totalPages={pages.length} issues={issues}
totalLighthouse={lighthouse.length} totalLighthouse={lighthouse.length}
averageResponseMs={stats.averageResponseMs} averageResponseMs={stats.averageResponseMs}
lighthouseSummary={stats.lighthouseSummary} lighthouseSummary={stats.lighthouseSummary}
@ -43,6 +71,7 @@ export function ResultsView({
<div className="card bg-base-100 border border-base-300"> <div className="card bg-base-100 border border-base-300">
<div className="card-body gap-3"> <div className="card-body gap-3">
<ResultsHeader <ResultsHeader
issueCount={issues.length}
pageCount={pages.length} pageCount={pages.length}
lighthouseCount={lighthouse.length} lighthouseCount={lighthouse.length}
hasPerformanceTab={hasPerformanceTab} hasPerformanceTab={hasPerformanceTab}
@ -53,11 +82,22 @@ export function ResultsView({
exportPerformance(lighthouse, pages, format); exportPerformance(lighthouse, pages, format);
return; return;
} }
if (activeTab === "issues") {
exportIssues(issues, format);
return;
}
exportPages(pages, format); exportPages(pages, format);
}} }}
/> />
{activeTab === "pages" && <PagesTable pages={pages} />} {activeTab === "issues" && <IssuesView issues={issues} />}
{activeTab === "pages" && (
<PagesTable
pages={pages}
startUrl={audit.startUrl}
issues={issues}
/>
)}
{activeTab === "performance" && lighthouse.length > 0 && ( {activeTab === "performance" && lighthouse.length > 0 && (
<PerformanceTable <PerformanceTable
auditId={audit.id} auditId={audit.id}
@ -117,6 +157,7 @@ function useResultStats(
} }
function ResultsHeader({ function ResultsHeader({
issueCount,
pageCount, pageCount,
lighthouseCount, lighthouseCount,
hasPerformanceTab, hasPerformanceTab,
@ -124,6 +165,7 @@ function ResultsHeader({
onTabChange, onTabChange,
onExport, onExport,
}: { }: {
issueCount: number;
pageCount: number; pageCount: number;
lighthouseCount: number; lighthouseCount: number;
hasPerformanceTab: boolean; hasPerformanceTab: boolean;
@ -132,49 +174,60 @@ function ResultsHeader({
onExport: (format: "csv" | "json" | "sheets") => void; onExport: (format: "csv" | "json" | "sheets") => void;
}) { }) {
const tabs: Array<{ tab: ResultsTab; label: string }> = [ const tabs: Array<{ tab: ResultsTab; label: string }> = [
{ tab: "issues", label: `Issues (${issueCount})` },
{ tab: "pages", label: `Pages (${pageCount})` }, { tab: "pages", label: `Pages (${pageCount})` },
{ tab: "performance", label: `Performance (${lighthouseCount})` }, ...(hasPerformanceTab
? [
{
tab: "performance" as const,
label: `Performance (${lighthouseCount})`,
},
]
: []),
]; ];
return ( return (
<div className="flex flex-col lg:flex-row lg:items-center justify-between gap-3"> <div className="flex flex-col lg:flex-row lg:items-center justify-between gap-3">
{hasPerformanceTab ? ( <div role="tablist" className="tabs tabs-border w-fit">
<div role="tablist" className="tabs tabs-border w-fit"> {tabs.map(({ label, tab }) => {
{tabs.map(({ label, tab }) => { const isActive = activeTab === tab;
const isActive = activeTab === tab;
return ( return (
<button <button
key={tab} key={tab}
type="button" type="button"
role="tab" role="tab"
aria-selected={isActive} aria-selected={isActive}
className={`tab ${isActive ? "tab-active" : ""}`} className={`tab ${isActive ? "tab-active" : ""}`}
onClick={() => onTabChange(tab)} onClick={() => onTabChange(tab)}
> >
{label} {label}
</button> </button>
); );
})} })}
</div> </div>
) : (
<h3 className="text-base font-medium">Pages ({pageCount})</h3>
)}
<ExportDropdown onExport={onExport} /> <ExportDropdown onExport={onExport} />
</div> </div>
); );
} }
function StatsGrid({ interface StatItem {
label: string;
value: string;
valueClass?: string;
sub?: ReactNode;
}
function StatsStrip({
pagesCrawled, pagesCrawled,
totalPages, issues,
totalLighthouse, totalLighthouse,
averageResponseMs, averageResponseMs,
lighthouseSummary, lighthouseSummary,
}: { }: {
pagesCrawled: number; pagesCrawled: number;
totalPages: number; issues: AuditResultsData["issues"];
totalLighthouse: number; totalLighthouse: number;
averageResponseMs: number; averageResponseMs: number;
lighthouseSummary: { lighthouseSummary: {
@ -184,54 +237,114 @@ function StatsGrid({
avgAccessibility: number | null; avgAccessibility: number | null;
}; };
}) { }) {
const severityCounts = useMemo(() => {
const counts = { critical: 0, warning: 0, info: 0 };
for (const issue of issues) {
counts[resolveIssueSeverity(issue)] += 1;
}
return counts;
}, [issues]);
const items: StatItem[] = [
{ label: "Pages crawled", value: String(pagesCrawled) },
{
label: "Issues found",
value: String(issues.length),
valueClass: issues.length === 0 ? "text-success" : "",
sub: issues.length > 0 && (
<span className="flex items-center gap-2.5">
<SeverityCount count={severityCounts.critical} dotClass="bg-error" />
<SeverityCount count={severityCounts.warning} dotClass="bg-warning" />
<SeverityCount
count={severityCounts.info}
dotClass="bg-base-content/30"
/>
</span>
),
},
{ label: "Avg response", value: `${averageResponseMs}ms` },
];
if (totalLighthouse > 0) {
items.push(
{ label: "Lighthouse tests", value: String(totalLighthouse) },
{
label: "Avg Lighthouse perf",
value:
lighthouseSummary.avgPerformance == null
? "-"
: String(lighthouseSummary.avgPerformance),
valueClass: scoreClass(lighthouseSummary.avgPerformance),
},
{
label: "Avg Lighthouse SEO",
value:
lighthouseSummary.avgSeo == null
? "-"
: String(lighthouseSummary.avgSeo),
valueClass: scoreClass(lighthouseSummary.avgSeo),
},
{
label: "Avg Lighthouse a11y",
value:
lighthouseSummary.avgAccessibility == null
? "-"
: String(lighthouseSummary.avgAccessibility),
valueClass: scoreClass(lighthouseSummary.avgAccessibility),
},
{
label: "Lighthouse failures",
value: String(lighthouseSummary.failed),
valueClass:
lighthouseSummary.failed > 0 ? "text-error" : "text-success",
},
);
}
const columnsClass =
items.length === 3
? "grid-cols-1 sm:grid-cols-3"
: "grid-cols-2 md:grid-cols-4";
return ( return (
<div className="grid grid-cols-2 md:grid-cols-4 gap-3"> <div
<StatCard label="Pages Crawled" value={String(pagesCrawled)} /> className={`grid ${columnsClass} gap-px rounded-lg border border-base-300 bg-base-300/70 overflow-hidden`}
<StatCard label="Total URLs" value={String(totalPages)} /> >
<StatCard label="Lighthouse Tests" value={String(totalLighthouse)} /> {items.map((item) => (
<StatCard label="Avg Response" value={`${averageResponseMs}ms`} /> <div key={item.label} className="bg-base-100 px-4 py-3">
{totalLighthouse > 0 && ( <p className="text-[11px] uppercase tracking-wider text-base-content/50">
<> {item.label}
<StatCard </p>
label="Avg Lighthouse Perf" <p
value={ className={`text-xl font-semibold mt-0.5 tabular-nums ${item.valueClass ?? ""}`}
lighthouseSummary.avgPerformance == null >
? "-" {item.value}
: String(lighthouseSummary.avgPerformance) </p>
} {item.sub && (
className={scoreClass(lighthouseSummary.avgPerformance)} <div className="text-xs text-base-content/60 mt-1">{item.sub}</div>
/> )}
<StatCard </div>
label="Avg Lighthouse SEO" ))}
value={
lighthouseSummary.avgSeo == null
? "-"
: String(lighthouseSummary.avgSeo)
}
className={scoreClass(lighthouseSummary.avgSeo)}
/>
<StatCard
label="Avg Lighthouse A11y"
value={
lighthouseSummary.avgAccessibility == null
? "-"
: String(lighthouseSummary.avgAccessibility)
}
className={scoreClass(lighthouseSummary.avgAccessibility)}
/>
<StatCard
label="Lighthouse Failures"
value={String(lighthouseSummary.failed)}
className={
lighthouseSummary.failed > 0 ? "text-error" : "text-success"
}
/>
</>
)}
</div> </div>
); );
} }
function SeverityCount({
count,
dotClass,
}: {
count: number;
dotClass: string;
}) {
if (count === 0) return null;
return (
<span className="flex items-center gap-1 tabular-nums">
<span className={`size-1.5 rounded-full ${dotClass}`} />
{count}
</span>
);
}
function scoreClass(score: number | null) { function scoreClass(score: number | null) {
if (score == null) return ""; if (score == null) return "";
if (score >= 90) return "text-success"; if (score >= 90) return "text-success";

54
src/client/features/audit/results/export.ts
View File

@ -1,8 +1,62 @@
import type { AuditResultsData } from "@/client/features/audit/results/types"; import type { AuditResultsData } from "@/client/features/audit/results/types";
import { getIssueDescriptor } from "@/shared/audit-issues";
import { buildCsv, type CsvValue, downloadCsv } from "@/client/lib/csv"; import { buildCsv, type CsvValue, downloadCsv } from "@/client/lib/csv";
import { downloadFile } from "@/client/lib/download"; import { downloadFile } from "@/client/lib/download";
import { exportTableToSheets } from "@/client/lib/exportToSheets"; import { exportTableToSheets } from "@/client/lib/exportToSheets";
const ISSUES_HEADERS = ["Severity", "Issue", "URL", "Details", "How To Fix"];
function issuesRows(issues: AuditResultsData["issues"]): CsvValue[][] {
return issues.map((issue) => {
const descriptor = getIssueDescriptor(issue.issueType);
return [
issue.severity,
descriptor?.title ?? issue.issueType,
issue.pageUrl,
issue.detailsJson ?? "",
descriptor?.howToFix ?? "",
];
});
}
export function exportIssues(
issues: AuditResultsData["issues"],
format: "csv" | "json" | "sheets",
) {
if (format === "json") {
const rows = issues.map((issue) => {
const descriptor = getIssueDescriptor(issue.issueType);
return {
severity: issue.severity,
issueType: issue.issueType,
issue: descriptor?.title ?? issue.issueType,
url: issue.pageUrl,
details: issue.detailsJson
? (JSON.parse(issue.detailsJson) as unknown)
: null,
howToFix: descriptor?.howToFix ?? null,
};
});
downloadFile(
JSON.stringify(rows, null, 2),
"audit-issues.json",
"application/json",
);
return;
}
if (format === "sheets") {
void exportTableToSheets({
headers: ISSUES_HEADERS,
rows: issuesRows(issues),
feature: "audit_issues",
});
return;
}
downloadCsv("audit-issues.csv", buildCsv(ISSUES_HEADERS, issuesRows(issues)));
}
const PAGES_HEADERS = [ const PAGES_HEADERS = [
"URL", "URL",
"Status", "Status",

21
src/client/features/audit/shared.tsx
View File

@ -78,24 +78,3 @@ export function LighthouseScoreBadge({ score }: { score: number | null }) {
score >= 90 ? "text-success" : score >= 50 ? "text-warning" : "text-error"; score >= 90 ? "text-success" : score >= 50 ? "text-warning" : "text-error";
return <span className={`font-medium text-sm ${color}`}>{score}</span>; return <span className={`font-medium text-sm ${color}`}>{score}</span>;
} }
export function StatCard({
label,
value,
className = "",
}: {
label: string;
value: string;
className?: string;
}) {
return (
<div className="card bg-base-100 border border-base-300">
<div className="card-body p-4">
<p className="text-xs uppercase tracking-wide text-base-content/60">
{label}
</p>
<p className={`text-2xl font-semibold ${className}`}>{value}</p>
</div>
</div>
);
}

13
src/routes/_project/p/$projectId/audit/index.tsx
View File

@ -72,7 +72,7 @@ function AuditDetail({
auditId: string; auditId: string;
tab: string; tab: string;
onBack: () => void; onBack: () => void;
onTabChange: (tab: "pages" | "performance") => void; onTabChange: (tab: "issues" | "pages" | "performance") => void;
}) { }) {
const statusQuery = useQuery({ const statusQuery = useQuery({
queryKey: ["audit-status", projectId, auditId], queryKey: ["audit-status", projectId, auditId],
@ -128,16 +128,17 @@ function AuditDetail({
<button className="btn btn-ghost btn-sm px-0" onClick={onBack}> <button className="btn btn-ghost btn-sm px-0" onClick={onBack}>
&larr; All audits &larr; All audits
</button> </button>
<div className="flex items-center justify-between"> <div className="flex flex-wrap items-center gap-x-3 gap-y-1">
<h1 className="text-2xl font-semibold">Site Audit</h1> <h1 className="text-2xl font-semibold">
{status ? extractHostname(status.startUrl) : "Site Audit"}
</h1>
{status?.status !== "running" && status && ( {status?.status !== "running" && status && (
<StatusBadge status={status.status} /> <StatusBadge status={status.status} />
)} )}
</div> </div>
{status && ( {status && (
<p className="text-sm text-base-content/70"> <p className="text-sm text-base-content/60">
{extractHostname(status.startUrl)} &middot; Started{" "} Site audit &middot; Started {formatStartedAt(status.startedAt)}
{formatStartedAt(status.startedAt)}
</p> </p>
)} )}
</div> </div>

32
src/server/lib/audit/issues/page-reporters.test.ts
View File

@ -5,7 +5,14 @@ import {
findRedirectChainsAndLoops, findRedirectChainsAndLoops,
type SlimPage, type SlimPage,
} from "@/server/lib/audit/issues/multipage-checks"; } from "@/server/lib/audit/issues/multipage-checks";
import type { CrawledPageResult } from "@/server/lib/audit/types"; import type { CrawledPageResult, PageLink } from "@/server/lib/audit/types";
const HEALTHY_LINK: PageLink = {
targetUrl: "https://example.com/catalog",
anchor: "Catalog",
isInternal: true,
isNofollow: false,
};
function makePage(overrides: Partial<CrawledPageResult>): CrawledPageResult { function makePage(overrides: Partial<CrawledPageResult>): CrawledPageResult {
return { return {
@ -37,7 +44,7 @@ function makePage(overrides: Partial<CrawledPageResult>): CrawledPageResult {
imagesTotal: 0, imagesTotal: 0,
imagesMissingAlt: 0, imagesMissingAlt: 0,
images: [], images: [],
links: [], links: [HEALTHY_LINK],
hasStructuredData: false, hasStructuredData: false,
hreflangTags: [], hreflangTags: [],
isIndexable: true, isIndexable: true,
@ -96,6 +103,17 @@ describe("runPageReporters", () => {
expect( expect(
issueTypes(makePage({ metaDescription: "x".repeat(200) })), issueTypes(makePage({ metaDescription: "x".repeat(200) })),
).toContain("meta-description-too-long"); ).toContain("meta-description-too-long");
expect(issueTypes(makePage({ metaDescription: "x".repeat(69) }))).toContain(
"meta-description-too-short",
);
expect(
issueTypes(makePage({ metaDescription: "x".repeat(70) })),
).not.toContain("meta-description-too-short");
expect(
runPageReporters(makePage({ metaDescription: "x".repeat(69) })).find(
(issue) => issue.issueType === "meta-description-too-short",
)?.details,
).toEqual({ length: 69 });
}); });
it("checks headings", () => { it("checks headings", () => {
@ -172,6 +190,16 @@ describe("runPageReporters", () => {
"deep-page", "deep-page",
); );
}); });
it("flags indexable pages with no outgoing links", () => {
expect(issueTypes(makePage({ links: [] }))).toContain("no-outgoing-links");
expect(
issueTypes(makePage({ links: [], isIndexable: false })),
).not.toContain("no-outgoing-links");
expect(issueTypes(makePage({ links: [HEALTHY_LINK] }))).not.toContain(
"no-outgoing-links",
);
});
}); });
function makeSlimPage(overrides: Partial<SlimPage>): SlimPage { function makeSlimPage(overrides: Partial<SlimPage>): SlimPage {

8
src/server/lib/audit/issues/page-reporters.ts
View File

@ -27,6 +27,7 @@ export interface DetectedIssue {
const TITLE_MAX_CHARS = 60; const TITLE_MAX_CHARS = 60;
const TITLE_MIN_CHARS = 10; const TITLE_MIN_CHARS = 10;
const META_DESCRIPTION_MAX_CHARS = 160; const META_DESCRIPTION_MAX_CHARS = 160;
const META_DESCRIPTION_MIN_CHARS = 70;
const THIN_CONTENT_WORDS = 150; const THIN_CONTENT_WORDS = 150;
const SLOW_RESPONSE_MS = 1500; const SLOW_RESPONSE_MS = 1500;
const DEEP_PAGE_DEPTH = 5; const DEEP_PAGE_DEPTH = 5;
@ -92,6 +93,10 @@ export function runPageReporters(page: CrawledPageResult): DetectedIssue[] {
report("meta-description-too-long", { report("meta-description-too-long", {
length: page.metaDescription.length, length: page.metaDescription.length,
}); });
} else if (page.metaDescription.length < META_DESCRIPTION_MIN_CHARS) {
report("meta-description-too-short", {
length: page.metaDescription.length,
});
} }
// Headings // Headings
@ -138,6 +143,9 @@ export function runPageReporters(page: CrawledPageResult): DetectedIssue[] {
} }
// Structure // Structure
if (page.isIndexable && page.links.length === 0) {
report("no-outgoing-links");
}
if (page.crawlDepth !== null && page.crawlDepth >= DEEP_PAGE_DEPTH) { if (page.crawlDepth !== null && page.crawlDepth >= DEEP_PAGE_DEPTH) {
report("deep-page", { crawlDepth: page.crawlDepth }); report("deep-page", { crawlDepth: page.crawlDepth });
} }

12
src/server/lib/audit/lighthouse.ts
View File

@ -1,4 +1,4 @@
import { detectUrlTemplate, normalizeUrl } from "./url-utils"; import { detectUrlTemplate, canonicalUrlKey } from "./url-utils";
import type { BillingCustomerContext } from "@/server/billing/subscription"; import type { BillingCustomerContext } from "@/server/billing/subscription";
import { createDataforseoClient } from "@/server/lib/dataforseo"; import { createDataforseoClient } from "@/server/lib/dataforseo";
import type { LighthouseResult, LighthouseStrategy } from "./types"; import type { LighthouseResult, LighthouseStrategy } from "./types";
@ -130,10 +130,12 @@ export function selectLighthouseSample(
// strategy === "auto": homepage + 1 per URL pattern, capped at 10 // strategy === "auto": homepage + 1 per URL pattern, capped at 10
const selected = new Set<string>(); const selected = new Set<string>();
// Always include the start URL / homepage. Page URLs are normalized; // Always include the start URL / homepage. Compare with canonicalUrlKey on
// normalize the start URL the same way or the comparison silently misses. // both sides so the match survives the redirects a site uses to reach its
const normalizedStart = normalizeUrl(startUrl) ?? startUrl; // canonical homepage: trailing-slash (/ -> // no, e.g. example.com ->
const startPage = validPages.find((p) => p.url === normalizedStart); // example.com/), www <-> non-www, and http -> https.
const startKey = canonicalUrlKey(startUrl);
const startPage = validPages.find((p) => canonicalUrlKey(p.url) === startKey);
if (startPage) selected.add(startPage.url); if (startPage) selected.add(startPage.url);
// Group by URL template pattern // Group by URL template pattern

37
src/server/lib/audit/url-utils.test.ts
View File

@ -1,5 +1,6 @@
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import {
canonicalUrlKey,
detectUrlTemplate, detectUrlTemplate,
getOrigin, getOrigin,
isSameOrigin, isSameOrigin,
@ -7,7 +8,7 @@ import {
} from "@/server/lib/audit/url-utils"; } from "@/server/lib/audit/url-utils";
describe("normalizeUrl", () => { describe("normalizeUrl", () => {
it("normalizes host/query/hash/trailing slash", () => { it("normalizes host/query/hash, preserves trailing slash", () => {
const value = normalizeUrl( const value = normalizeUrl(
"https://Example.COM/path/?b=2&a=1#section", "https://Example.COM/path/?b=2&a=1#section",
"https://fallback.com", "https://fallback.com",
@ -16,11 +17,45 @@ describe("normalizeUrl", () => {
expect(value).toBe("https://example.com/path/?a=1&b=2"); expect(value).toBe("https://example.com/path/?a=1&b=2");
}); });
it("preserves a trailing slash on path-only URLs", () => {
// A trailing slash is the canonical form on most CMSes; stripping it would
// rewrite the canonical URL into its own redirect source and cause a loop.
expect(normalizeUrl("https://example.com/services/")).toBe(
"https://example.com/services/",
);
});
it("preserves the absence of a trailing slash", () => {
expect(normalizeUrl("https://example.com/services")).toBe(
"https://example.com/services",
);
});
it("returns null for unsupported protocol", () => { it("returns null for unsupported protocol", () => {
expect(normalizeUrl("mailto:test@example.com")).toBeNull(); expect(normalizeUrl("mailto:test@example.com")).toBeNull();
}); });
}); });
describe("canonicalUrlKey", () => {
it("treats www and non-www as equal", () => {
expect(canonicalUrlKey("https://www.example.com/")).toBe(
canonicalUrlKey("https://example.com/"),
);
});
it("treats http and https as equal", () => {
expect(canonicalUrlKey("http://example.com/")).toBe(
canonicalUrlKey("https://example.com/"),
);
});
it("keeps the trailing-slash distinction in the path", () => {
expect(canonicalUrlKey("https://example.com/services")).not.toBe(
canonicalUrlKey("https://example.com/services/"),
);
});
});
describe("isSameOrigin", () => { describe("isSameOrigin", () => {
it("accepts www host equivalence", () => { it("accepts www host equivalence", () => {
expect( expect(

41
src/server/lib/audit/url-utils.ts
View File

@ -8,7 +8,12 @@
* - Strip fragments (#...) * - Strip fragments (#...)
* - Sort query parameters * - Sort query parameters
* - Lowercase the hostname * - Lowercase the hostname
* - Remove trailing slash (except for root path "/") * - Preserve trailing slashes. A trailing slash is the canonical form on most
* CMS platforms (WordPress etc.), which 301-redirect the non-slash version to
* it. Stripping it here would rewrite the canonical URL into its own redirect
* source, so the crawler would follow /path -> /path/ and strip back to /path
* forever — a 508 loop. Keeping /path and /path/ distinct lets the redirect
* resolve normally.
*/ */
export function normalizeUrl(url: string, base?: string): string | null { export function normalizeUrl(url: string, base?: string): string | null {
try { try {
@ -28,18 +33,38 @@ export function normalizeUrl(url: string, base?: string): string | null {
// Lowercase hostname // Lowercase hostname
parsed.hostname = parsed.hostname.toLowerCase(); parsed.hostname = parsed.hostname.toLowerCase();
// Remove trailing slash (but keep "/" for root) return parsed.toString();
let normalized = parsed.toString();
if (normalized.endsWith("/") && parsed.pathname !== "/") {
normalized = normalized.slice(0, -1);
}
return normalized;
} catch { } catch {
return null; return null;
} }
} }
/**
* A canonical key for URL equality checks that should survive the common
* redirect patterns a site uses to reach its canonical form:
* - trailing-slash redirects (/services -> /services/)
* - www <-> non-www (www.example.com -> example.com)
* - http -> https upgrades
*
* Forces https, drops a leading "www.", lowercases the hostname, sorts query
* params, and strips the fragment. Trailing slashes are intentionally left
* intact so two genuinely different paths never collapse together; this key is
* only for "is this effectively the same page as the start URL" comparisons
* (e.g. picking the homepage for the Lighthouse sample), not for crawl dedup.
*/
export function canonicalUrlKey(url: string): string {
try {
const parsed = new URL(url);
parsed.protocol = "https:";
parsed.hostname = parsed.hostname.toLowerCase().replace(/^www\./, "");
parsed.hash = "";
parsed.searchParams.sort();
return parsed.toString();
} catch {
return url.toLowerCase();
}
}
function getEffectivePort(parsed: URL): string { function getEffectivePort(parsed: URL): string {
if (parsed.port) return parsed.port; if (parsed.port) return parsed.port;
return parsed.protocol === "https:" ? "443" : "80"; return parsed.protocol === "https:" ? "443" : "80";

68
src/server/workflows/site-audit-workflow-helpers.ts
View File

@ -38,15 +38,6 @@ function classifyFetch(
return "ok"; return "ok";
} }
/** Resolve a Location header against its base without normalizing. */
function resolveRawUrl(location: string, base: string): string | null {
try {
return new URL(location, base).toString();
} catch {
return null;
}
}
/** Parse `Link: <url>; rel="canonical"` response headers. */ /** Parse `Link: <url>; rel="canonical"` response headers. */
function parseLinkHeaderCanonical( function parseLinkHeaderCanonical(
linkHeader: string | null, linkHeader: string | null,
@ -72,52 +63,31 @@ export async function crawlPage(
try { try {
// Manual redirect handling: each hop is recorded as its own page row and // Manual redirect handling: each hop is recorded as its own page row and
// the target is enqueued by the frontier, so chains/loops are detectable. // its target is enqueued by the frontier, so redirect chains and loops are
// Exception: redirects whose target normalizes to this same URL (e.g. // detectable from the recorded rows. Trailing-slash redirects (/docs ->
// /docs -> /docs/ on slash-canonical sites — our normalizer strips the // /docs/) need no special handling: normalizeUrl preserves trailing
// slash) are followed inline; recording them would create self-redirect // slashes, so /docs and /docs/ are distinct URLs and the redirect resolves
// rows the frontier can never resolve. // to its canonical target instead of cycling back to its own source.
let fetchUrl = url; const response = await fetch(url, {
let response: Response; headers: {
let hops = 0; "User-Agent": CRAWL_USER_AGENT,
for (;;) { Accept: "text/html,application/xhtml+xml",
response = await fetch(fetchUrl, { },
headers: { redirect: "manual",
"User-Agent": CRAWL_USER_AGENT, signal: AbortSignal.timeout(15_000),
Accept: "text/html,application/xhtml+xml", });
},
redirect: "manual",
signal: AbortSignal.timeout(15_000),
});
if (response.status < 300 || response.status >= 400) break;
const location = response.headers.get("location");
const rawTarget = location ? resolveRawUrl(location, fetchUrl) : null;
const normalizedTarget = location
? normalizeUrl(location, fetchUrl)
: null;
const isSelfAfterNormalization =
normalizedTarget === url &&
rawTarget !== null &&
rawTarget !== fetchUrl;
if (!isSelfAfterNormalization || hops >= 3) break;
fetchUrl = rawTarget;
hops += 1;
}
const responseTimeMs = Date.now() - startTime; const responseTimeMs = Date.now() - startTime;
const statusCode = response.status; const statusCode = response.status;
const xRobotsTag = response.headers.get("x-robots-tag"); const xRobotsTag = response.headers.get("x-robots-tag");
const headerCanonicalUrl = parseLinkHeaderCanonical( const headerCanonicalUrl = parseLinkHeaderCanonical(
response.headers.get("link"), response.headers.get("link"),
fetchUrl, url,
); );
if (statusCode >= 300 && statusCode < 400) { if (statusCode >= 300 && statusCode < 400) {
const location = response.headers.get("location"); const location = response.headers.get("location");
const redirectUrl = location ? normalizeUrl(location, fetchUrl) : null; const redirectUrl = location ? normalizeUrl(location, url) : null;
return emptyPageResult({ return emptyPageResult({
url, url,
statusCode, statusCode,
@ -154,14 +124,12 @@ export async function crawlPage(
}); });
} }
// Resolve links/canonical against the URL that actually served the
// content (it may carry a trailing slash the recorded URL doesn't).
// Dynamic import keeps cheerio (page-analyzer's HTML parser) out of the // Dynamic import keeps cheerio (page-analyzer's HTML parser) out of the
// worker's startup module graph: SiteAuditWorkflow is re-exported from // worker's startup module graph: SiteAuditWorkflow is re-exported from
// src/server.ts, so a static import would evaluate cheerio in every // src/server.ts, so a static import would evaluate cheerio in every
// isolate's baseline heap, not just when an audit actually crawls. // isolate's baseline heap, not just when an audit actually crawls.
const { analyzeHtml } = await import("@/server/lib/audit/page-analyzer"); const { analyzeHtml } = await import("@/server/lib/audit/page-analyzer");
const analysis = analyzeHtml(body, fetchUrl, statusCode, responseTimeMs); const analysis = analyzeHtml(body, url, statusCode, responseTimeMs);
const robotsDirectives = [analysis.robotsMeta, xRobotsTag] const robotsDirectives = [analysis.robotsMeta, xRobotsTag]
.filter(Boolean) .filter(Boolean)
.join(",") .join(",")
@ -179,7 +147,7 @@ export async function crawlPage(
title: analysis.title, title: analysis.title,
metaDescription: analysis.metaDescription, metaDescription: analysis.metaDescription,
canonicalUrl: analysis.canonical canonicalUrl: analysis.canonical
? (normalizeUrl(analysis.canonical, fetchUrl) ?? analysis.canonical) ? (normalizeUrl(analysis.canonical, url) ?? analysis.canonical)
: null, : null,
robotsMeta: analysis.robotsMeta, robotsMeta: analysis.robotsMeta,
xRobotsTag, xRobotsTag,
@ -187,7 +155,7 @@ export async function crawlPage(
ogTitle: analysis.ogTitle, ogTitle: analysis.ogTitle,
ogDescription: analysis.ogDescription, ogDescription: analysis.ogDescription,
ogImage: analysis.ogImage, ogImage: analysis.ogImage,
h1Count: analysis.h1s.length, h1Count: analysis.h1s.filter((h) => h.length > 0).length,
h2Count: headingCount(2), h2Count: headingCount(2),
h3Count: headingCount(3), h3Count: headingCount(3),
h4Count: headingCount(4), h4Count: headingCount(4),

18
src/shared/audit-issues.ts
View File

@ -6,7 +6,7 @@
* of these types by id. * of these types by id.
*/ */
type IssueSeverity = "critical" | "warning" | "info"; export type IssueSeverity = "critical" | "warning" | "info";
interface AuditIssueDescriptor { interface AuditIssueDescriptor {
severity: IssueSeverity; severity: IssueSeverity;
@ -152,6 +152,14 @@ export const AUDIT_ISSUE_TYPES = {
howToFix: howToFix:
"Link to this page from relevant pages (navigation, related content, hub pages), or remove it from the sitemap if it shouldn't be indexed.", "Link to this page from relevant pages (navigation, related content, hub pages), or remove it from the sitemap if it shouldn't be indexed.",
}, },
"no-outgoing-links": {
severity: "warning",
title: "Page has no outgoing links",
explanation:
"The page contains no links at all — a dead end. Link equity that flows into it stops there, crawlers have nowhere to go next, and users have to reach for the back button.",
howToFix:
"Add links to related pages, the parent category, or the homepage. If the page's navigation is rendered by JavaScript, make sure it also exists in the server-rendered HTML.",
},
"title-too-long": { "title-too-long": {
severity: "info", severity: "info",
title: "Title too long", title: "Title too long",
@ -176,6 +184,14 @@ export const AUDIT_ISSUE_TYPES = {
howToFix: howToFix:
"Trim the description to roughly 70–160 characters while keeping the core message and call to action.", "Trim the description to roughly 70–160 characters while keeping the core message and call to action.",
}, },
"meta-description-too-short": {
severity: "info",
title: "Meta description too short",
explanation:
"The meta description is under ~70 characters. Short descriptions waste the snippet space search results give you, and search engines often ignore them in favor of text pulled from the page.",
howToFix:
"Expand the description to roughly 70–160 characters that summarize the page and give a reason to click.",
},
"heading-order-skip": { "heading-order-skip": {
severity: "info", severity: "info",
title: "Heading levels skip", title: "Heading levels skip",

4
src/types/schemas/audit.ts
View File

@ -46,9 +46,9 @@ export const getCrawlProgressSchema = z.object({
// ─── URL search params schema for /p/$projectId/audit ──────────────────────── // ─── URL search params schema for /p/$projectId/audit ────────────────────────
const auditTabs = ["pages", "performance"] as const; const auditTabs = ["issues", "pages", "performance"] as const;
export const auditSearchSchema = z.object({ export const auditSearchSchema = z.object({
auditId: z.string().optional().catch(undefined), auditId: z.string().optional().catch(undefined),
tab: z.enum(auditTabs).catch("pages").default("pages"), tab: z.enum(auditTabs).catch("issues").default("issues"),
}); });