diff --git a/.env.example b/.env.example index ab00267..42d98cf 100644 --- a/.env.example +++ b/.env.example @@ -1,7 +1,7 @@ # ----------------------------------------------------------------------------- # Core settings # ----------------------------------------------------------------------------- -# Required for all modes. See README.md for setup instructions. +# Required for SEO data features. See docs/DATAFORSEO_API_KEY.md. # DATAFORSEO_API_KEY= # Optional app port @@ -23,7 +23,7 @@ # Defaults to cloudflare_access when unset. # AUTH_MODE=cloudflare_access -# Required when AUTH_MODE=cloudflare_access +# Required when AUTH_MODE=cloudflare_access. See docs/SELF_HOSTING_CLOUDFLARE.md. # TEAM_DOMAIN=https://your-team.cloudflareaccess.com # POLICY_AUD=your-cloudflare-access-aud-tag diff --git a/Dockerfile.selfhost b/Dockerfile.selfhost index f44eb1a..383650e 100644 --- a/Dockerfile.selfhost +++ b/Dockerfile.selfhost @@ -15,10 +15,18 @@ COPY . . EXPOSE 3001 +# Readiness probe against the unauthenticated setup/health endpoint. The long +# start period covers migrations plus the boot-time vite build below. +HEALTHCHECK --interval=30s --timeout=10s --start-period=300s --retries=3 \ + CMD node -e "fetch('http://127.0.0.1:'+(process.env.PORT||3001)+'/api/health').then(function(r){process.exit(r.ok?0:1)}).catch(function(){process.exit(1)})" + +# The preflight validates env BEFORE the slow steps, so misconfiguration fails +# in seconds with the exact fix instead of after a multi-minute build. +# # The build MUST run at container start, not image-build time: AUTH_MODE (and the # other client envs) are inlined into the client bundle by `vite build`, and the # self-hoster only chooses AUTH_MODE at runtime via Compose. Building here lets # that runtime value bake into the bundle. The repo .npmrc raises the V8 heap # ceiling (node-options) so the ~7400-module SSR build doesn't OOM under Node's # ~2GB default. -CMD ["sh", "-c", "echo 'OpenSEO sends an anonymous usage heartbeat (counts only). Disable: OPENSEO_TELEMETRY_DISABLED=1. Details: docs/SELF_HOSTING_DOCKER.md#telemetry' && pnpm run db:migrate:local && pnpm run build && pnpm exec vite preview --host 0.0.0.0 --port ${PORT:-3001}"] +CMD ["sh", "-c", "echo 'OpenSEO sends an anonymous usage heartbeat (counts only). Disable: OPENSEO_TELEMETRY_DISABLED=1. Details: docs/SELF_HOSTING_DOCKER.md#telemetry' && pnpm exec tsx scripts/selfhost-preflight.ts && pnpm run db:migrate:local && pnpm run build && pnpm exec vite preview --host 0.0.0.0 --port ${PORT:-3001}"] diff --git a/compose.yaml b/compose.yaml index dc0b5b9..0f32c25 100644 --- a/compose.yaml +++ b/compose.yaml @@ -2,6 +2,13 @@ services: open-seo: image: ${OPEN_SEO_IMAGE:-ghcr.io/every-app/open-seo:latest} restart: unless-stopped + # Forward EVERY .env value into the container. The explicit `environment:` + # list below still wins for the values it names — without this, any .env + # value not listed there (e.g. OPENROUTER_API_KEY) silently never reached + # the app. Compose errors if .env is missing; the quickstart's + # `cp .env.example .env` creates it. + env_file: + - .env # Anonymous usage heartbeat controls: docs/SELF_HOSTING_DOCKER.md#telemetry environment: # Required for local Docker self-hosting: exposes Compose env vars to cloudflare:workers bindings. @@ -17,6 +24,9 @@ services: - GOOGLE_CLIENT_ID=${GOOGLE_CLIENT_ID:-} - GOOGLE_CLIENT_SECRET=${GOOGLE_CLIENT_SECRET:-} - BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET:-} + # Optional: AI features (SAM, the in-app SEO agent) + - OPENROUTER_API_KEY=${OPENROUTER_API_KEY:-} + - OPENROUTER_MODEL=${OPENROUTER_MODEL:-} - VITE_SHOW_DEVTOOLS=false ports: - "127.0.0.1:${PORT:-3001}:${PORT:-3001}" diff --git a/docs/LOCAL_DEVELOPMENT.md b/docs/LOCAL_DEVELOPMENT.md index d5a7d49..a212853 100644 --- a/docs/LOCAL_DEVELOPMENT.md +++ b/docs/LOCAL_DEVELOPMENT.md @@ -22,6 +22,8 @@ Configure `.env.local`: `printf '%s' 'YOUR_LOGIN:YOUR_PASSWORD' | base64` +3. Set `AUTH_MODE=local_noauth` for normal local development. + Run locally: ```sh @@ -67,7 +69,6 @@ backend for installs that outgrow D1 — see - `AUTH_MODE=local_noauth`: local trusted mode, no auth check, injects `admin@localhost`. - `AUTH_MODE=hosted`: Better Auth-backed email/password mode. Requires Better Auth schema generation plus `BETTER_AUTH_SECRET` and `BETTER_AUTH_URL`. -Local scripts (`pnpm dev` and `pnpm dev:agents`) set `AUTH_MODE=local_noauth` automatically. -Use `AUTH_MODE=cloudflare_access pnpm dev` when you specifically want to test Access validation locally. +Dev scripts do not set `AUTH_MODE`, so you can test another mode by changing it in `.env.local`. For Cloudflare deployments, ensure Cloudflare Access is enabled on your Worker route/domain and provide `TEAM_DOMAIN` + `POLICY_AUD` in environment variables. diff --git a/docs/SELF_HOSTING_CLOUDFLARE.md b/docs/SELF_HOSTING_CLOUDFLARE.md index 6a5e543..af7772f 100644 --- a/docs/SELF_HOSTING_CLOUDFLARE.md +++ b/docs/SELF_HOSTING_CLOUDFLARE.md @@ -22,16 +22,31 @@ If deploy fails with `Cannot provision a KV Namespace with the title "open-seo" ## 2) Configure authentication and secrets -In the Cloudflare dashboard: +### Create the Access application -1. Go to `Compute` -> `Workers & Pages` -> your OpenSEO Worker. -2. Open `Settings`. -3. In `Domains & Routes`, enable `Cloudflare Access` for the `workers.dev` route. -4. Save the values shown by Cloudflare Access. -5. In `Variables & Secrets`, add: - - `POLICY_AUD` (from Access setup) - - `TEAM_DOMAIN` (domain from `JWKS_URL`, for example `https://your-team.cloudflareaccess.com`) - - `DATAFORSEO_API_KEY` (see [`DATAFORSEO_API_KEY.md`](./DATAFORSEO_API_KEY.md) for how to get one) +1. In the main Cloudflare dashboard, go to `Compute` -> `Workers & Pages` -> your OpenSEO Worker -> `Settings` -> `Domains & Routes`. Copy the `workers.dev` hostname. It looks like `open-seo..workers.dev`. +2. Open [Cloudflare Zero Trust](https://one.dash.cloudflare.com/). +3. Go to `Access controls` -> `Applications` -> `Create new application` -> `Self-hosted and private`. +4. Name the application `OpenSEO`. +5. Under `Destinations` -> `Public hostnames`, click `Switch to custom input` and paste the exact `workers.dev` hostname from step 1. Enter only the hostname, without `https://` or a path. +6. Under `Access policies`, click `Create new policy` and configure: + - `Policy name`: `Allow OpenSEO users` + - `Action`: `Allow` + - `Include` selector: `Emails` + - Value: your Cloudflare account email +7. Do not choose `Everyone`; it allows anyone to reach the application. +8. Leave the other policy settings at their defaults, save the policy, then save the application. + +### Collect the values + +- `POLICY_AUD`: in `Access controls` -> `Applications`, select `Configure` on your application, then copy the `Application Audience (AUD) Tag` from `Additional settings`. +- `TEAM_DOMAIN`: `https://.cloudflareaccess.com`. Your team name is shown in Zero Trust `Settings`. Include the `https://` prefix. +- `DATAFORSEO_API_KEY`: follow [`DATAFORSEO_API_KEY.md`](./DATAFORSEO_API_KEY.md). + +### Set them on the Worker + +1. Go to `Compute` -> `Workers & Pages` -> your OpenSEO Worker -> `Settings` -> `Variables & Secrets`. +2. Add `TEAM_DOMAIN`, `POLICY_AUD`, and `DATAFORSEO_API_KEY`. ## 3) Optional: add an R2 lifecycle rule @@ -51,7 +66,11 @@ Without a lifecycle rule, cached objects under `dataforseo-cache/` will accumula 2. Sign in with Cloudflare Access. 3. OpenSEO should load after login. -If login fails, re-check the three secrets and Access toggle. +If it doesn't, see Troubleshooting below. + +## Troubleshooting + +`https:///api/health` reports runtime configuration checks and database status. For server errors, open the Worker `Logs` or run `pnpm exec wrangler tail`. ## Next steps diff --git a/docs/SELF_HOSTING_CLOUDFLARE_MANUAL.md b/docs/SELF_HOSTING_CLOUDFLARE_MANUAL.md index a917e69..f4412ed 100644 --- a/docs/SELF_HOSTING_CLOUDFLARE_MANUAL.md +++ b/docs/SELF_HOSTING_CLOUDFLARE_MANUAL.md @@ -87,14 +87,7 @@ pnpm run deploy ## 6) Configure authentication and secrets -In the Cloudflare dashboard: - -1. Go to `Compute` -> `Workers & Pages` -> your OpenSEO Worker. -2. Open `Settings`. -3. In `Domains & Routes`, enable `Cloudflare Access` for the `workers.dev` route. -4. Save the values shown by Cloudflare Access. - -Then set the same values as Worker secrets with Wrangler: +Follow [Configure authentication and secrets](./SELF_HOSTING_CLOUDFLARE.md#2-configure-authentication-and-secrets), then set the values with Wrangler: ```bash pnpm exec wrangler secret put TEAM_DOMAIN @@ -102,8 +95,6 @@ pnpm exec wrangler secret put POLICY_AUD pnpm exec wrangler secret put DATAFORSEO_API_KEY ``` -Use the domain from `JWKS_URL` for `TEAM_DOMAIN`, for example `https://your-team.cloudflareaccess.com`. Use the Access application audience value for `POLICY_AUD`. See [`DATAFORSEO_API_KEY.md`](./DATAFORSEO_API_KEY.md) for how to get a DataForSEO key. - ## 7) Optional: add an R2 lifecycle rule DataForSEO API responses are cached in R2 under the `dataforseo-cache/` prefix. This step is optional, but recommended to automatically clean up expired cache objects: @@ -118,7 +109,7 @@ pnpm exec wrangler r2 bucket lifecycle add open-seo-YOUR_SUFFIX dataforseo-cache 2. Sign in with Cloudflare Access. 3. OpenSEO should load after login. -If login fails, re-check the three secrets, the Access toggle, and the binding values in `wrangler.jsonc`. +If login fails, check `/api/health`, the Worker logs, and the binding values in `wrangler.jsonc`. ## Next steps diff --git a/docs/SELF_HOSTING_DOCKER.md b/docs/SELF_HOSTING_DOCKER.md index 1615001..ab7860a 100644 --- a/docs/SELF_HOSTING_DOCKER.md +++ b/docs/SELF_HOSTING_DOCKER.md @@ -17,12 +17,15 @@ The default `compose.yaml` uses the published GHCR image: ```bash cp .env.example .env +``` + +Set `DATAFORSEO_API_KEY` in `.env` using the [DataForSEO setup guide](./DATAFORSEO_API_KEY.md), then start OpenSEO: + +```bash docker compose up -d ``` -Set `DATAFORSEO_API_KEY` in `.env`, then open `http://localhost:` (default `3001`). - -Docker Compose passes `.env` values into the container, and `compose.yaml` enables `CLOUDFLARE_INCLUDE_PROCESS_ENV=true` so the Cloudflare Vite runtime can read them as Worker bindings during local self-hosting. +Open `http://localhost:` (default `3001`). Each container start builds the app and may take 1-2 minutes; follow progress with `docker compose logs -f`. Optional env values: @@ -41,7 +44,7 @@ You can also persist it in `.env`. ## Telemetry -OpenSEO collects anonymized telemetry for core usage events: heartbeats with aggregate counts (installs, users, projects, feature usage) tied to a random install ID, sent every 5 minutes during the first two hours after install, then at most once daily. No URLs, keywords, prompts, emails, or IP-derived location are collected, and idle installs send nothing. +OpenSEO collects anonymized telemetry for core usage events: heartbeats with aggregate counts (installs, users, projects, feature usage) tied to a random install ID, sent every 5 minutes during the first two hours after install, then at most once daily. Telemetry also includes failed setup check names and statuses, never values or error messages. No URLs, keywords, prompts, emails, or IP-derived location are collected, and idle installs send nothing. To disable it, set `OPENSEO_TELEMETRY_DISABLED=1` (or `DO_NOT_TRACK=1`) in `.env`, then run `docker compose up -d --force-recreate open-seo`. @@ -83,11 +86,9 @@ docker compose pull && docker compose up -d docker compose down ``` -- Stop and remove volumes: +## Health and troubleshooting -```bash -docker compose down -v -``` +Startup checks appear in `docker compose logs` before the build. Once running, `/api/health` reports configuration and database status, and `docker compose ps` reports container health. ## Troubleshooting environment variables diff --git a/docs/self-host-auth-mode-spec.md b/docs/self-host-auth-mode-spec.md deleted file mode 100644 index e69de29..0000000 diff --git a/scripts/selfhost-preflight.ts b/scripts/selfhost-preflight.ts new file mode 100644 index 0000000..c196505 --- /dev/null +++ b/scripts/selfhost-preflight.ts @@ -0,0 +1,67 @@ +/** + * Container-start preflight for Docker self-hosting. Validates the environment + * before migrations and the vite build so misconfiguration fails in seconds + * with the exact fix. Run via: pnpm exec tsx scripts/selfhost-preflight.ts + * + * Exits non-zero on hard failures (invalid AUTH_MODE, missing auth config for + * the selected mode). Warnings and info lines never block startup. + */ +import process from "node:process"; +import { + formatPreflightReport, + runSelfhostPreflight, +} from "../src/lib/selfhost-preflight"; +import { isTelemetryOptOutValue } from "../src/shared/selfhost-checks"; +import { version } from "../package.json"; + +const SELF_HOST_POSTHOG_KEY = + "phc_xaXj4vE4LikxfvR7q6EHemAYNBSZW4hQkqor7fpf8aGT"; +const SELF_HOST_POSTHOG_HOST = "https://us.i.posthog.com"; + +function telemetryDisabled(): boolean { + return ( + isTelemetryOptOutValue(process.env.OPENSEO_TELEMETRY_DISABLED) || + isTelemetryOptOutValue(process.env.DO_NOT_TRACK) + ); +} + +// Anonymous "an install failed preflight" beacon: failed check names only, a +// throwaway distinct id, no env values. Without this, installs that never +// finish booting are invisible — the regular heartbeat needs a working app. +async function sendPreflightFailedBeacon(failedChecks: string[]) { + if (telemetryDisabled()) return; + + try { + await fetch(`${SELF_HOST_POSTHOG_HOST}/i/v0/e/`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + signal: AbortSignal.timeout(5000), + body: JSON.stringify({ + api_key: SELF_HOST_POSTHOG_KEY, + event: "self_host.preflight_failed", + distinct_id: crypto.randomUUID(), + properties: { + failedChecks, + version, + $process_person_profile: false, + }, + }), + }); + } catch { + // Telemetry must never affect startup. + } +} + +const result = runSelfhostPreflight(process.env); + +console.log("--- OpenSEO self-host preflight ---"); +console.log(formatPreflightReport(result)); + +if (result.failed) { + await sendPreflightFailedBeacon( + result.items + .filter((item) => item.level === "fail") + .map((item) => item.name), + ); + process.exit(1); +} diff --git a/src/client/components/AuthConfigErrorCard.tsx b/src/client/components/AuthConfigErrorCard.tsx index f0fbeb5..47b8bc5 100644 --- a/src/client/components/AuthConfigErrorCard.tsx +++ b/src/client/components/AuthConfigErrorCard.tsx @@ -1,7 +1,8 @@ import { ShieldAlert } from "lucide-react"; +import { isHostedClientAuthMode } from "@/lib/auth-mode"; -const README_CLOUDFLARE_ACCESS_URL = - "https://github.com/every-app/open-seo#cloudflare-deployment--access-setup"; +const CLOUDFLARE_SETUP_GUIDE_URL = + "https://github.com/every-app/open-seo/blob/main/docs/SELF_HOSTING_CLOUDFLARE.md#2-configure-authentication-and-secrets"; type AuthConfigErrorCardProps = { message: string; @@ -12,6 +13,8 @@ export function AuthConfigErrorCard({ message, onRetry, }: AuthConfigErrorCardProps) { + const isHostedMode = isHostedClientAuthMode(); + return (
@@ -24,14 +27,21 @@ export function AuthConfigErrorCard({ {message}
-

- Check the auth environment variables for your selected - AUTH_MODE. Cloudflare Access requires - TEAM_DOMAIN and - POLICY_AUD. Hosted mode requires - BETTER_AUTH_SECRET and - BETTER_AUTH_URL. -

+ {isHostedMode ? ( +

+ Hosted mode requires{" "} + BETTER_AUTH_SECRET + (32+ characters), BETTER_AUTH_URL, and + Google OAuth credentials on the deployment. +

+ ) : ( +

+ Cloudflare Access mode requires + TEAM_DOMAIN (a full https URL) and + POLICY_AUD set on the deployment, with + an Access application protecting this hostname. +

+ )}
{onRetry ? ( @@ -41,7 +51,7 @@ export function AuthConfigErrorCard({ ) : null} diff --git a/src/client/features/lighthouse/issues/LighthouseIssuesScreen.tsx b/src/client/features/lighthouse/issues/LighthouseIssuesScreen.tsx index 44d3fb3..7e94bed 100644 --- a/src/client/features/lighthouse/issues/LighthouseIssuesScreen.tsx +++ b/src/client/features/lighthouse/issues/LighthouseIssuesScreen.tsx @@ -6,6 +6,7 @@ import { getAuditLighthouseIssues, } from "@/serverFunctions/lighthouse"; import { downloadFile } from "@/client/lib/download"; +import { getStandardErrorMessage } from "@/client/lib/error-messages"; import { exportTableToSheets } from "@/client/lib/exportToSheets"; import type { CategoryTab, ExportPayload, LighthouseIssue } from "./types"; import { categoryLabel, issuesToCsv, issuesToTable } from "./utils"; @@ -69,10 +70,10 @@ export function LighthouseIssuesScreen(props: LighthouseIssuesScreenProps) { allIssues: issuesQuery.data?.issues ?? [], }); - const issuesErrorMessage = - issuesQuery.error instanceof Error - ? issuesQuery.error.message - : "Failed to load Lighthouse issues."; + const issuesErrorMessage = getStandardErrorMessage( + issuesQuery.error, + "Failed to load Lighthouse issues.", + ); const showsLegacyPayloadNotice = issuesQuery.data != null && !issuesQuery.data.hasIssueDetails; const emptyMessage = showsLegacyPayloadNotice diff --git a/src/client/lib/error-messages.test.ts b/src/client/lib/error-messages.test.ts index d8bdf6f..6f5b6d4 100644 --- a/src/client/lib/error-messages.test.ts +++ b/src/client/lib/error-messages.test.ts @@ -1,5 +1,8 @@ import { describe, expect, it } from "vitest"; -import { getStandardErrorMessage } from "@/client/lib/error-messages"; +import { + getErrorCode, + getStandardErrorMessage, +} from "@/client/lib/error-messages"; describe("getStandardErrorMessage", () => { it("maps known error codes to standard copy", () => { @@ -16,3 +19,33 @@ describe("getStandardErrorMessage", () => { ).toBe("DataForSEO task missing billing metadata. Response: {...}"); }); }); + +describe("coded error messages (CODE: detail)", () => { + const coded = new Error( + "AUTH_CONFIG_MISSING: TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com", + ); + + it("extracts the code from a coded message", () => { + expect(getErrorCode(coded)).toBe("AUTH_CONFIG_MISSING"); + }); + + it("shows the server detail instead of the generic text", () => { + expect(getStandardErrorMessage(coded)).toBe( + "TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com", + ); + }); + + it("keeps bare codes mapping to the standard copy", () => { + const bare = new Error("AUTH_CONFIG_MISSING"); + expect(getErrorCode(bare)).toBe("AUTH_CONFIG_MISSING"); + expect(getStandardErrorMessage(bare)).toContain("not configured"); + }); + + it("does not treat arbitrary colon messages as coded", () => { + const arbitrary = new Error("Something failed: try again"); + expect(getErrorCode(arbitrary)).toBeNull(); + expect(getStandardErrorMessage(arbitrary)).toBe( + "Something failed: try again", + ); + }); +}); diff --git a/src/client/lib/error-messages.ts b/src/client/lib/error-messages.ts index 2b6a8dd..bc05e99 100644 --- a/src/client/lib/error-messages.ts +++ b/src/client/lib/error-messages.ts @@ -32,17 +32,33 @@ const STANDARD_MESSAGES: Record = { "An unexpected error occurred. Please check server logs and try again.", }; +// Setup errors cross the wire as "CODE: detail" (see toClientError) so the +// user sees the server's specific guidance while code-driven UI (error cards, +// redirects) still keys off the code. +function splitCodedMessage( + message: string, +): { code: ErrorCode; detail: string } | null { + const separatorIndex = message.indexOf(": "); + if (separatorIndex === -1) return null; + const code = message.slice(0, separatorIndex); + if (!isErrorCode(code)) return null; + return { code, detail: message.slice(separatorIndex + 2) }; +} + export function getStandardErrorMessage( error: unknown, fallback: string = STANDARD_MESSAGES.INTERNAL_ERROR, ): string { if (!(error instanceof Error)) return fallback; if (isErrorCode(error.message)) return STANDARD_MESSAGES[error.message]; + const coded = splitCodedMessage(error.message); + if (coded) return coded.detail; if (error.message) return error.message; return fallback; } export function getErrorCode(error: unknown): ErrorCode | null { if (!(error instanceof Error)) return null; - return isErrorCode(error.message) ? error.message : null; + if (isErrorCode(error.message)) return error.message; + return splitCodedMessage(error.message)?.code ?? null; } diff --git a/src/lib/auth-mode.ts b/src/lib/auth-mode.ts index 6dfa903..30797f2 100644 --- a/src/lib/auth-mode.ts +++ b/src/lib/auth-mode.ts @@ -1,13 +1,32 @@ import { z } from "zod"; -type AuthMode = "cloudflare_access" | "local_noauth" | "hosted"; +export const AUTH_MODES = [ + "cloudflare_access", + "local_noauth", + "hosted", +] as const; -const authModeSchema = z - .enum(["cloudflare_access", "local_noauth", "hosted"]) - .catch("cloudflare_access"); +type AuthMode = (typeof AUTH_MODES)[number]; + +const authModeSchema = z.enum(AUTH_MODES); + +const warnedInvalidAuthModes = new Set(); export function getAuthMode(value: string | null | undefined): AuthMode { - return authModeSchema.parse(value); + const parsed = authModeSchema.safeParse(value); + if (parsed.success) return parsed.data; + + // Unset stays a silent fail-closed default; a SET-but-invalid value is a + // typo the operator needs to hear about — silently coercing it made every + // request fail with a Cloudflare Access error about a mode they never chose. + if (value && !warnedInvalidAuthModes.has(value)) { + warnedInvalidAuthModes.add(value); + console.error( + `Invalid AUTH_MODE "${value}" — falling back to "cloudflare_access". Valid values: ${AUTH_MODES.join(", ")}.`, + ); + } + + return "cloudflare_access"; } export function isHostedAuthMode(value: string | null | undefined) { diff --git a/src/lib/selfhost-preflight.test.ts b/src/lib/selfhost-preflight.test.ts new file mode 100644 index 0000000..8911e0d --- /dev/null +++ b/src/lib/selfhost-preflight.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, it } from "vitest"; +import { runSelfhostPreflight } from "./selfhost-preflight"; + +function itemFor( + result: ReturnType, + name: string, +) { + return result.items.find((item) => item.name === name); +} + +describe("runSelfhostPreflight", () => { + it("passes the stock Docker setup (local_noauth + DataForSEO key)", () => { + const result = runSelfhostPreflight({ + AUTH_MODE: "local_noauth", + DATAFORSEO_API_KEY: btoa("user@example.com:secret"), + }); + + expect(result.failed).toBe(false); + expect(itemFor(result, "AUTH_MODE")?.level).toBe("ok"); + expect(itemFor(result, "DATAFORSEO_API_KEY")?.level).toBe("ok"); + }); + + it("fails an invalid AUTH_MODE with the valid list", () => { + const result = runSelfhostPreflight({ AUTH_MODE: "local-noauth" }); + + expect(result.failed).toBe(true); + expect(itemFor(result, "AUTH_MODE")?.message).toContain( + "cloudflare_access, local_noauth, hosted", + ); + }); + + it("fails cloudflare_access mode without TEAM_DOMAIN and POLICY_AUD", () => { + const result = runSelfhostPreflight({}); + + expect(result.failed).toBe(true); + const item = itemFor(result, "AUTH_MODE"); + expect(item?.message).toContain("TEAM_DOMAIN and POLICY_AUD"); + expect(item?.message).toContain("AUTH_MODE is unset"); + }); + + it("fails a bare-hostname TEAM_DOMAIN with the https:// fix", () => { + const result = runSelfhostPreflight({ + AUTH_MODE: "cloudflare_access", + TEAM_DOMAIN: "your-team.cloudflareaccess.com", + POLICY_AUD: "aud-tag", + }); + + expect(result.failed).toBe(true); + expect(itemFor(result, "TEAM_DOMAIN")?.message).toContain("https://"); + }); + + it("warns on a DataForSEO key that is not base64 login:password", () => { + const result = runSelfhostPreflight({ + AUTH_MODE: "local_noauth", + DATAFORSEO_API_KEY: "raw-dashboard-key", + }); + + expect(result.failed).toBe(false); + expect(itemFor(result, "DATAFORSEO_API_KEY")?.level).toBe("warn"); + expect(itemFor(result, "DATAFORSEO_API_KEY")?.message).toContain("base64"); + }); + + it("warns that GSC stays disabled on a short BETTER_AUTH_SECRET", () => { + const result = runSelfhostPreflight({ + AUTH_MODE: "local_noauth", + GOOGLE_CLIENT_ID: "id", + GOOGLE_CLIENT_SECRET: "secret", + BETTER_AUTH_SECRET: "too-short", + }); + + expect(itemFor(result, "Search Console")?.level).toBe("warn"); + expect(itemFor(result, "Search Console")?.message).toContain("32"); + }); + + it("fails hosted mode listing every missing variable", () => { + const result = runSelfhostPreflight({ + AUTH_MODE: "hosted", + BETTER_AUTH_SECRET: "x".repeat(40), + }); + + expect(result.failed).toBe(true); + const item = itemFor(result, "AUTH_MODE"); + expect(item?.message).toContain("BETTER_AUTH_URL"); + expect(item?.message).toContain("GOOGLE_CLIENT_ID"); + expect(item?.message).not.toContain("BETTER_AUTH_SECRET,"); + }); + + it("mentions ALLOWED_HOST when unset", () => { + const result = runSelfhostPreflight({ AUTH_MODE: "local_noauth" }); + + expect(itemFor(result, "ALLOWED_HOST")?.level).toBe("info"); + expect(itemFor(result, "ALLOWED_HOST")?.message).toContain("reverse proxy"); + }); +}); diff --git a/src/lib/selfhost-preflight.ts b/src/lib/selfhost-preflight.ts new file mode 100644 index 0000000..d85db81 --- /dev/null +++ b/src/lib/selfhost-preflight.ts @@ -0,0 +1,277 @@ +import { AUTH_MODES } from "@/lib/auth-mode"; +import { + looksLikeDataForSeoKey, + MIN_BETTER_AUTH_SECRET_LENGTH, + validateTeamDomain, +} from "@/shared/selfhost-checks"; + +// Startup preflight for self-host containers: validate the environment BEFORE +// the multi-minute build/boot so misconfiguration fails in seconds with the +// exact fix, instead of surfacing minutes later as a generic in-app error. +// "fail" aborts startup; "warn" degrades a feature; "info" is orientation. + +type PreflightLevel = "ok" | "info" | "warn" | "fail"; + +type PreflightItem = { + // Stable identifier shared with /api/health's check map. + key: "auth" | "dataforseo" | "gsc" | "ai" | "runtime"; + name: string; + level: PreflightLevel; + message: string; +}; + +type PreflightResult = { + items: PreflightItem[]; + failed: boolean; +}; + +type EnvRecord = Record; + +function get(env: EnvRecord, name: string): string | undefined { + const value = env[name]?.trim(); + return value ? value : undefined; +} + +function checkAuthMode(env: EnvRecord, items: PreflightItem[]): void { + const rawMode = get(env, "AUTH_MODE"); + + if (rawMode && !(AUTH_MODES as readonly string[]).includes(rawMode)) { + items.push({ + key: "auth", + name: "AUTH_MODE", + level: "fail", + message: `"${rawMode}" is not a valid AUTH_MODE. Valid values: ${AUTH_MODES.join(", ")}.`, + }); + return; + } + + const mode = rawMode ?? "cloudflare_access"; + + if (mode === "local_noauth") { + items.push({ + key: "auth", + name: "AUTH_MODE", + level: "ok", + message: + "local_noauth — no auth, single admin user. Do not expose publicly without your own auth in front.", + }); + return; + } + + if (mode === "hosted") { + const missing = [ + "BETTER_AUTH_URL", + "BETTER_AUTH_SECRET", + "GOOGLE_CLIENT_ID", + "GOOGLE_CLIENT_SECRET", + ].filter((name) => !get(env, name)); + items.push( + missing.length + ? { + key: "auth", + name: "AUTH_MODE", + level: "fail", + message: `hosted mode requires ${missing.join(", ")}.`, + } + : { key: "auth", name: "AUTH_MODE", level: "ok", message: "hosted" }, + ); + return; + } + + // cloudflare_access (explicit or defaulted) + const teamDomain = get(env, "TEAM_DOMAIN"); + const policyAud = get(env, "POLICY_AUD"); + const modeLabel = rawMode + ? "cloudflare_access" + : "cloudflare_access (default — AUTH_MODE is unset)"; + + if (!teamDomain || !policyAud) { + const missing = [ + teamDomain ? null : "TEAM_DOMAIN", + policyAud ? null : "POLICY_AUD", + ] + .filter(Boolean) + .join(" and "); + items.push({ + key: "auth", + name: "AUTH_MODE", + level: "fail", + message: `${modeLabel} requires ${missing}. See docs/SELF_HOSTING_CLOUDFLARE.md — or set AUTH_MODE=local_noauth for a private, no-auth deployment.`, + }); + return; + } + + const teamDomainResult = validateTeamDomain(teamDomain); + if (!teamDomainResult.ok) { + items.push({ + key: "auth", + name: "TEAM_DOMAIN", + level: "fail", + message: teamDomainResult.message, + }); + return; + } + + items.push({ + key: "auth", + name: "AUTH_MODE", + level: "ok", + message: modeLabel, + }); +} + +function checkDataForSeo(env: EnvRecord, items: PreflightItem[]): void { + const key = get(env, "DATAFORSEO_API_KEY"); + + if (!key) { + items.push({ + key: "dataforseo", + name: "DATAFORSEO_API_KEY", + level: "warn", + message: + "Not set — all SEO data features will be unavailable until it is. It is the base64 of your DataForSEO login:password (NOT the dashboard API key). See docs/DATAFORSEO_API_KEY.md.", + }); + return; + } + + if (!looksLikeDataForSeoKey(key)) { + items.push({ + key: "dataforseo", + name: "DATAFORSEO_API_KEY", + level: "warn", + message: + "Set, but does not decode as base64 of login:password. If DataForSEO rejects it, encode your account email and API password: printf 'email:password' | base64.", + }); + return; + } + + items.push({ + key: "dataforseo", + name: "DATAFORSEO_API_KEY", + level: "ok", + message: "Set", + }); +} + +function checkOptionalFeatures(env: EnvRecord, items: PreflightItem[]): void { + const clientId = get(env, "GOOGLE_CLIENT_ID"); + const clientSecret = get(env, "GOOGLE_CLIENT_SECRET"); + const betterAuthSecret = get(env, "BETTER_AUTH_SECRET"); + + if (clientId || clientSecret) { + if (!clientId || !clientSecret) { + items.push({ + key: "gsc", + name: "Search Console", + level: "warn", + message: + "Only one of GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET is set — both are required.", + }); + } else if ( + !betterAuthSecret || + betterAuthSecret.length < MIN_BETTER_AUTH_SECRET_LENGTH + ) { + items.push({ + key: "gsc", + name: "Search Console", + level: "warn", + message: `Google credentials are set, but Search Console stays DISABLED until BETTER_AUTH_SECRET is at least ${MIN_BETTER_AUTH_SECRET_LENGTH} characters (it encrypts stored OAuth tokens).`, + }); + } else { + items.push({ + key: "gsc", + name: "Search Console", + level: "ok", + message: "Configured", + }); + } + } else { + items.push({ + key: "gsc", + name: "Search Console", + level: "info", + message: + "Not configured (optional). See docs/SELF_HOSTING_GOOGLE_SEARCH_CONSOLE.md.", + }); + } + + items.push( + get(env, "OPENROUTER_API_KEY") + ? { + key: "ai", + name: "AI features", + level: "ok", + message: "OPENROUTER_API_KEY set", + } + : { + key: "ai", + name: "AI features", + level: "info", + message: + "OPENROUTER_API_KEY not set (optional) — SAM, the in-app SEO agent, is disabled.", + }, + ); +} + +// Shared per-feature checks: the Docker preflight prints these at boot and +// /api/health (setup-status.ts) serves the same results at runtime, so the +// two can never drift. +export function runSelfhostChecks(env: EnvRecord): PreflightItem[] { + const items: PreflightItem[] = []; + checkAuthMode(env, items); + checkDataForSeo(env, items); + checkOptionalFeatures(env, items); + return items; +} + +export function runSelfhostPreflight(env: EnvRecord): PreflightResult { + const items = runSelfhostChecks(env); + + items.push( + get(env, "ALLOWED_HOST") + ? { + key: "runtime", + name: "ALLOWED_HOST", + level: "ok", + message: `Requests allowed for host ${get(env, "ALLOWED_HOST")}`, + } + : { + key: "runtime", + name: "ALLOWED_HOST", + level: "info", + message: + 'Not set — only localhost access will work. Behind a reverse proxy or tunnel, set ALLOWED_HOST=yourdomain.com or requests are blocked with Vite\'s "Blocked request" page.', + }, + ); + + items.push({ + key: "runtime", + name: "Scheduled checks", + level: "info", + message: + "Rank-tracking schedules do not run in Docker mode — trigger checks from the Rank Tracking page.", + }); + + return { items, failed: items.some((item) => item.level === "fail") }; +} + +const LEVEL_BADGES: Record = { + ok: "[ ok ]", + info: "[info]", + warn: "[warn]", + fail: "[FAIL]", +}; + +export function formatPreflightReport(result: PreflightResult): string { + const lines = result.items.map( + (item) => `${LEVEL_BADGES[item.level]} ${item.name}: ${item.message}`, + ); + + lines.push( + result.failed + ? "\nPreflight failed — fix the [FAIL] items above and restart. Nothing was started." + : "\nPreflight passed. The app now builds inside the container (~1-2 minutes on every start before it serves).", + ); + + return lines.join("\n"); +} diff --git a/src/middleware/ensure-user/accessTokenErrors.test.ts b/src/middleware/ensure-user/accessTokenErrors.test.ts new file mode 100644 index 0000000..33dfac6 --- /dev/null +++ b/src/middleware/ensure-user/accessTokenErrors.test.ts @@ -0,0 +1,74 @@ +import { errors as joseErrors } from "jose"; +import { describe, expect, it } from "vitest"; +import { classifyAccessVerificationError } from "./accessTokenErrors"; + +describe("classifyAccessVerificationError", () => { + it("maps audience mismatch to a POLICY_AUD config error", () => { + const error = classifyAccessVerificationError( + new joseErrors.JWTClaimValidationFailed( + 'unexpected "aud" claim value', + {}, + "aud", + "check_failed", + ), + ); + + expect(error.code).toBe("AUTH_CONFIG_MISSING"); + expect(error.message).toContain("POLICY_AUD"); + }); + + it("maps issuer mismatch to a TEAM_DOMAIN config error", () => { + const error = classifyAccessVerificationError( + new joseErrors.JWTClaimValidationFailed( + 'unexpected "iss" claim value', + {}, + "iss", + "check_failed", + ), + ); + + expect(error.code).toBe("AUTH_CONFIG_MISSING"); + expect(error.message).toContain("TEAM_DOMAIN"); + }); + + it("keeps expired tokens as UNAUTHENTICATED (re-auth fixes them)", () => { + const error = classifyAccessVerificationError( + new joseErrors.JWTExpired('"exp" claim timestamp check failed', {}), + ); + + expect(error.code).toBe("UNAUTHENTICATED"); + }); + + it("maps JWKS lookup failures to a TEAM_DOMAIN config error", () => { + const error = classifyAccessVerificationError( + new joseErrors.JWKSNoMatchingKey(), + ); + + expect(error.code).toBe("AUTH_CONFIG_MISSING"); + expect(error.message).toContain("TEAM_DOMAIN"); + }); + + it("maps network failures fetching the JWKS to a config error", () => { + const error = classifyAccessVerificationError( + new TypeError("fetch failed"), + ); + + expect(error.code).toBe("AUTH_CONFIG_MISSING"); + }); + + it("maps workerd-style plain network errors to a config error", () => { + const error = classifyAccessVerificationError( + new Error("Network connection lost"), + ); + + expect(error.code).toBe("AUTH_CONFIG_MISSING"); + }); + + it("keeps other jose failures (bad signature) as UNAUTHENTICATED", () => { + const error = classifyAccessVerificationError( + new joseErrors.JWSSignatureVerificationFailed(), + ); + + expect(error.code).toBe("UNAUTHENTICATED"); + }); +}); diff --git a/src/middleware/ensure-user/accessTokenErrors.ts b/src/middleware/ensure-user/accessTokenErrors.ts new file mode 100644 index 0000000..81fb6aa --- /dev/null +++ b/src/middleware/ensure-user/accessTokenErrors.ts @@ -0,0 +1,47 @@ +import { errors as joseErrors } from "jose"; +import { AppError } from "@/server/lib/errors"; + +// Maps a jwtVerify failure to the right AppError. Config mistakes (wrong +// POLICY_AUD, TEAM_DOMAIN pointing at the wrong team, unreachable JWKS) must +// surface as AUTH_CONFIG_MISSING with guidance — collapsing them into bare +// UNAUTHENTICATED puts self-hosters in a sign-in loop with no signal anywhere, +// since UNAUTHENTICATED is a non-reportable code. Token-level failures +// (expired, bad signature) stay UNAUTHENTICATED: re-authenticating fixes them. +export function classifyAccessVerificationError(error: unknown): AppError { + if (error instanceof joseErrors.JWTExpired) { + return new AppError("UNAUTHENTICATED"); + } + + if (error instanceof joseErrors.JWTClaimValidationFailed) { + if (error.claim === "aud") { + return new AppError( + "AUTH_CONFIG_MISSING", + "Cloudflare Access token rejected: audience mismatch. POLICY_AUD does not match your Access application's AUD tag — copy it from Zero Trust -> Access controls -> Applications -> Configure -> Additional settings.", + ); + } + if (error.claim === "iss") { + return new AppError( + "AUTH_CONFIG_MISSING", + "Cloudflare Access token rejected: issuer mismatch. TEAM_DOMAIN does not match the Cloudflare team that issued the token — check it against your team domain in Zero Trust settings.", + ); + } + return new AppError("UNAUTHENTICATED"); + } + + if ( + error instanceof joseErrors.JWKSNoMatchingKey || + error instanceof joseErrors.JWKSInvalid || + error instanceof joseErrors.JWKSTimeout || + // The caller only classifies errors thrown by jwtVerify itself, so a + // non-jose error can only come from the remote JWKS fetch (TypeError in + // browsers/node, plain Error like "Network connection lost" in workerd). + !(error instanceof joseErrors.JOSEError) + ) { + return new AppError( + "AUTH_CONFIG_MISSING", + "Could not verify the Cloudflare Access token against TEAM_DOMAIN's signing keys. Check that TEAM_DOMAIN is your team's https://.cloudflareaccess.com domain.", + ); + } + + return new AppError("UNAUTHENTICATED"); +} diff --git a/src/middleware/ensure-user/cloudflareAccess.ts b/src/middleware/ensure-user/cloudflareAccess.ts index 54b7394..f1ba663 100644 --- a/src/middleware/ensure-user/cloudflareAccess.ts +++ b/src/middleware/ensure-user/cloudflareAccess.ts @@ -1,6 +1,8 @@ import { env } from "cloudflare:workers"; -import { createRemoteJWKSet, jwtVerify } from "jose"; +import { createRemoteJWKSet, jwtVerify, type JWTPayload } from "jose"; import { AppError } from "@/server/lib/errors"; +import { validateTeamDomain } from "@/shared/selfhost-checks"; +import { classifyAccessVerificationError } from "./accessTokenErrors"; import { resolveDelegatedContext } from "./delegated"; import type { EnsuredUserContext } from "./types"; @@ -25,22 +27,13 @@ function getJwks(teamDomain: string) { } function getValidatedTeamDomain(teamDomain: string) { - const normalizedTeamDomain = teamDomain.trim().replace(/\/+$/, ""); + const result = validateTeamDomain(teamDomain); - try { - const parsed = new URL(normalizedTeamDomain); - - if (parsed.protocol !== "https:") { - throw new Error("TEAM_DOMAIN must use https"); - } - - return parsed.origin; - } catch { - throw new AppError( - "AUTH_CONFIG_MISSING", - "TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com", - ); + if (!result.ok) { + throw new AppError("AUTH_CONFIG_MISSING", result.message); } + + return result.origin; } export async function resolveCloudflareAccessContext( @@ -52,37 +45,54 @@ export async function resolveCloudflareAccessContext( const policyAud = env.POLICY_AUD?.trim() || null; if (!teamDomain || !policyAud) { + const missing = [ + teamDomain ? null : "TEAM_DOMAIN", + policyAud ? null : "POLICY_AUD", + ] + .filter(Boolean) + .join(" and "); throw new AppError( "AUTH_CONFIG_MISSING", - "Missing Cloudflare Access configuration", + `Missing Cloudflare Access configuration: set ${missing} on the deployment. See docs/SELF_HOSTING_CLOUDFLARE.md.`, ); } const token = headers.get("cf-access-jwt-assertion"); if (!token) { - throw new AppError("UNAUTHENTICATED"); + // With Access enabled in front of the deployment, every request carries + // this header — its absence means Access is not actually protecting the + // route, which is a setup problem, not a signed-out user. + throw new AppError( + "AUTH_CONFIG_MISSING", + "No Cloudflare Access token on the request. Cloudflare Access is not enabled in front of this deployment — add an Access application covering this hostname in Zero Trust, or set AUTH_MODE=local_noauth if you intend to run without auth on a private network.", + ); } + // Only the token verification itself is classified — anything thrown past + // this block (user resolution, DB access) is an app fault, and classifying + // it here would mislabel a DB outage as an auth-config problem. + let payload: JWTPayload; try { const jwks = getJwks(teamDomain); - const { payload } = await jwtVerify(token, jwks, { + ({ payload } = await jwtVerify(token, jwks, { issuer: teamDomain, audience: policyAud, - }); - const userId = typeof payload.sub === "string" ? payload.sub : null; - const userEmail = typeof payload.email === "string" ? payload.email : null; - - if (!userId || !userEmail) { - throw new AppError("UNAUTHENTICATED"); - } - - return resolveDelegatedContext(userId, userEmail); + })); } catch (error) { - if (error instanceof AppError) { - throw error; - } + // The classified AppError carries operator guidance; log the raw jose + // error too, since it is the only place the underlying cause survives. + console.error("Cloudflare Access token verification failed:", error); + throw classifyAccessVerificationError(error); + } + + const userId = typeof payload.sub === "string" ? payload.sub : null; + const userEmail = typeof payload.email === "string" ? payload.email : null; + + if (!userId || !userEmail) { throw new AppError("UNAUTHENTICATED"); } + + return resolveDelegatedContext(userId, userEmail); } diff --git a/src/routeTree.gen.ts b/src/routeTree.gen.ts index 3c07882..66db33a 100644 --- a/src/routeTree.gen.ts +++ b/src/routeTree.gen.ts @@ -17,6 +17,7 @@ import { Route as AuthRouteImport } from './routes/_auth' import { Route as ProjectRouteRouteImport } from './routes/_project/route' import { Route as AppRouteRouteImport } from './routes/_app/route' import { Route as AppIndexRouteImport } from './routes/_app/index' +import { Route as ApiHealthRouteImport } from './routes/api/health' import { Route as AuthenticatedSubscribeRouteImport } from './routes/_authenticated.subscribe' import { Route as AuthenticatedOauthConsentRouteImport } from './routes/_authenticated.oauth-consent' import { Route as AuthSignUpRouteImport } from './routes/_auth.sign-up' @@ -88,6 +89,11 @@ const AppIndexRoute = AppIndexRouteImport.update({ path: '/', getParentRoute: () => AppRouteRoute, } as any) +const ApiHealthRoute = ApiHealthRouteImport.update({ + id: '/api/health', + path: '/api/health', + getParentRoute: () => rootRouteImport, +} as any) const AuthenticatedSubscribeRoute = AuthenticatedSubscribeRouteImport.update({ id: '/subscribe', path: '/subscribe', @@ -289,6 +295,7 @@ export interface FileRoutesByFullPath { '/sign-up': typeof AuthSignUpRoute '/oauth-consent': typeof AuthenticatedOauthConsentRoute '/subscribe': typeof AuthenticatedSubscribeRoute + '/api/health': typeof ApiHealthRoute '/p/$projectId': typeof ProjectPProjectIdRouteRouteWithChildren '/help/dataforseo-api-key': typeof AppHelpDataforseoApiKeyRoute '/help/openrouter-api-key': typeof AppHelpOpenrouterApiKeyRoute @@ -329,6 +336,7 @@ export interface FileRoutesByTo { '/sign-up': typeof AuthSignUpRoute '/oauth-consent': typeof AuthenticatedOauthConsentRoute '/subscribe': typeof AuthenticatedSubscribeRoute + '/api/health': typeof ApiHealthRoute '/help/dataforseo-api-key': typeof AppHelpDataforseoApiKeyRoute '/help/openrouter-api-key': typeof AppHelpOpenrouterApiKeyRoute '/onboarding/chat': typeof AuthenticatedOnboardingChatRoute @@ -370,6 +378,7 @@ export interface FileRoutesById { '/_auth/sign-up': typeof AuthSignUpRoute '/_authenticated/oauth-consent': typeof AuthenticatedOauthConsentRoute '/_authenticated/subscribe': typeof AuthenticatedSubscribeRoute + '/api/health': typeof ApiHealthRoute '/_app/': typeof AppIndexRoute '/_project/p/$projectId': typeof ProjectPProjectIdRouteRouteWithChildren '/_app/help/dataforseo-api-key': typeof AppHelpDataforseoApiKeyRoute @@ -413,6 +422,7 @@ export interface FileRouteTypes { | '/sign-up' | '/oauth-consent' | '/subscribe' + | '/api/health' | '/p/$projectId' | '/help/dataforseo-api-key' | '/help/openrouter-api-key' @@ -453,6 +463,7 @@ export interface FileRouteTypes { | '/sign-up' | '/oauth-consent' | '/subscribe' + | '/api/health' | '/help/dataforseo-api-key' | '/help/openrouter-api-key' | '/onboarding/chat' @@ -493,6 +504,7 @@ export interface FileRouteTypes { | '/_auth/sign-up' | '/_authenticated/oauth-consent' | '/_authenticated/subscribe' + | '/api/health' | '/_app/' | '/_project/p/$projectId' | '/_app/help/dataforseo-api-key' @@ -529,6 +541,7 @@ export interface RootRouteChildren { ResetPasswordRoute: typeof ResetPasswordRoute VerifyEmailRoute: typeof VerifyEmailRoute Char91DotwellKnownChar93OpenaiAppsChallengeRoute: typeof Char91DotwellKnownChar93OpenaiAppsChallengeRoute + ApiHealthRoute: typeof ApiHealthRoute ApiAuthSplatRoute: typeof ApiAuthSplatRoute ApiAutumnSplatRoute: typeof ApiAutumnSplatRoute ApiGscOauthCallbackRoute: typeof ApiGscOauthCallbackRoute @@ -592,6 +605,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof AppIndexRouteImport parentRoute: typeof AppRouteRoute } + '/api/health': { + id: '/api/health' + path: '/api/health' + fullPath: '/api/health' + preLoaderRoute: typeof ApiHealthRouteImport + parentRoute: typeof rootRouteImport + } '/_authenticated/subscribe': { id: '/_authenticated/subscribe' path: '/subscribe' @@ -984,6 +1004,7 @@ const rootRouteChildren: RootRouteChildren = { VerifyEmailRoute: VerifyEmailRoute, Char91DotwellKnownChar93OpenaiAppsChallengeRoute: Char91DotwellKnownChar93OpenaiAppsChallengeRoute, + ApiHealthRoute: ApiHealthRoute, ApiAuthSplatRoute: ApiAuthSplatRoute, ApiAutumnSplatRoute: ApiAutumnSplatRoute, ApiGscOauthCallbackRoute: ApiGscOauthCallbackRoute, diff --git a/src/routes/api/health.ts b/src/routes/api/health.ts new file mode 100644 index 0000000..baafcf9 --- /dev/null +++ b/src/routes/api/health.ts @@ -0,0 +1,29 @@ +import { createFileRoute } from "@tanstack/react-router"; +import { env } from "cloudflare:workers"; +import { isHostedAuthMode } from "@/lib/auth-mode"; + +// Unauthenticated setup/health endpoint for self-hosters: reports per-feature +// configuration status (statuses and guidance only — never secret values) so +// "container is up but misconfigured" is diagnosable with one curl. The Docker +// HEALTHCHECK probes it. Hosted mode returns a bare ok and no config detail. +async function handleHealthRequest(): Promise { + if (isHostedAuthMode(env.AUTH_MODE)) { + return Response.json({ status: "ok" }); + } + + const { getSelfHostSetupStatus } = await import("@/server/lib/setup-status"); + const setup = await getSelfHostSetupStatus(); + const hasError = Object.values(setup.checks).some( + (check) => check.status === "error", + ); + + return Response.json({ status: hasError ? "issues" : "ok", ...setup }); +} + +export const Route = createFileRoute("/api/health")({ + server: { + handlers: { + GET: () => handleHealthRequest(), + }, + }, +}); diff --git a/src/server/features/gsc/oauth-config.ts b/src/server/features/gsc/oauth-config.ts index cc0e2ab..09c8ee4 100644 --- a/src/server/features/gsc/oauth-config.ts +++ b/src/server/features/gsc/oauth-config.ts @@ -1,4 +1,5 @@ import { getOptionalEnvValue } from "@/server/lib/runtime-env"; +import { MIN_BETTER_AUTH_SECRET_LENGTH } from "@/shared/selfhost-checks"; type GscOAuthClientConfig = { clientId: string; @@ -24,5 +25,5 @@ export async function hasSelfHostedGscConfig(): Promise { if (!(await getGscOAuthClientConfig())) return false; const secret = (await getOptionalEnvValue("BETTER_AUTH_SECRET"))?.trim(); - return Boolean(secret && secret.length >= 32); + return Boolean(secret && secret.length >= MIN_BETTER_AUTH_SECRET_LENGTH); } diff --git a/src/server/lib/errors.test.ts b/src/server/lib/errors.test.ts index 361d2fe..22a374c 100644 --- a/src/server/lib/errors.test.ts +++ b/src/server/lib/errors.test.ts @@ -18,4 +18,23 @@ describe("toClientError", () => { expect(error.message).toBe("PAYMENT_REQUIRED"); }); + + it("passes setup-error detail through as CODE: detail", () => { + const error = toClientError( + new AppError( + "AUTH_CONFIG_MISSING", + "TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com", + ), + ); + + expect(error.message).toBe( + "AUTH_CONFIG_MISSING: TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com", + ); + }); + + it("keeps a detail-less setup error as its bare code", () => { + const error = toClientError(new AppError("AUTH_CONFIG_MISSING")); + + expect(error.message).toBe("AUTH_CONFIG_MISSING"); + }); }); diff --git a/src/server/lib/errors.ts b/src/server/lib/errors.ts index 610a69a..313777c 100644 --- a/src/server/lib/errors.ts +++ b/src/server/lib/errors.ts @@ -19,7 +19,21 @@ export function asAppError(error: unknown): AppError | null { return null; } +// Codes whose server-side message is safe and useful to show the user. +// Setup errors only: their messages are static guidance ("TEAM_DOMAIN must be +// a full https URL…") that self-hosters need to fix their deployment, and the +// alternative is a generic card that makes every misconfiguration look the +// same. Everything else stays stripped to its bare code. +const CLIENT_DETAIL_ERROR_CODES = new Set(["AUTH_CONFIG_MISSING"]); + export function toClientError(error: unknown): Error { const appError = asAppError(error); + if ( + appError && + CLIENT_DETAIL_ERROR_CODES.has(appError.code) && + appError.message !== appError.code + ) { + return new Error(`${appError.code}: ${appError.message}`); + } return new Error(appError?.code ?? "INTERNAL_ERROR"); } diff --git a/src/server/lib/self-host-telemetry.test.ts b/src/server/lib/self-host-telemetry.test.ts index 10a72f1..bedef23 100644 --- a/src/server/lib/self-host-telemetry.test.ts +++ b/src/server/lib/self-host-telemetry.test.ts @@ -62,6 +62,7 @@ function createHarness( isNonProductionBuild: () => false, claimHeartbeat, collectCounts: async () => emptyCounts, + collectSetupIssues: async () => [], sendHeartbeat, markHeartbeatSent, getDbBackend: () => "d1", @@ -156,6 +157,27 @@ describe("maybeSendSelfHostHeartbeat", () => { expect(harness.sendHeartbeat).not.toHaveBeenCalled(); }); + it('still sends when the disable flags are explicitly "0"/"false"', async () => { + vi.stubEnv("OPENSEO_TELEMETRY_DISABLED", "0"); + vi.stubEnv("DO_NOT_TRACK", "false"); + const harness = createHarness(); + + await runHeartbeat(harness); + + expect(harness.sendHeartbeat).toHaveBeenCalledTimes(1); + }); + + it("includes the setup-issue summary in heartbeat properties", async () => { + const harness = createHarness(); + harness.dependencies.collectSetupIssues = async () => ["dataforseo:error"]; + + await runHeartbeat(harness); + + expect(harness.sendHeartbeat.mock.calls[0]?.[1]).toMatchObject({ + setupIssues: ["dataforseo:error"], + }); + }); + it("does not send from non-production builds (dev, test, preview)", async () => { const harness = createHarness(); // Omit the isNonProductionBuild override: the production gate reads diff --git a/src/server/lib/self-host-telemetry.ts b/src/server/lib/self-host-telemetry.ts index 0296817..95f02bd 100644 --- a/src/server/lib/self-host-telemetry.ts +++ b/src/server/lib/self-host-telemetry.ts @@ -18,6 +18,8 @@ import { getOptionalEnvValue, isHostedServerAuthMode, } from "@/server/lib/runtime-env"; +import { getSetupIssueSummary } from "@/server/lib/setup-status"; +import { isTelemetryOptOutValue } from "@/shared/selfhost-checks"; const SELF_HOST_POSTHOG_KEY = "phc_xaXj4vE4LikxfvR7q6EHemAYNBSZW4hQkqor7fpf8aGT"; @@ -75,6 +77,9 @@ type HeartbeatProperties = HeartbeatCounts & { firstRun: boolean; minutesSinceInstall?: number; mcpToolCalls: number; + // Unhealthy setup checks as "check:status" pairs (e.g. "dataforseo:error"). + // Enumerable values only — never free-text detail. + setupIssues: string[]; $process_person_profile: false; }; @@ -83,6 +88,7 @@ export type SelfHostTelemetryDependencies = { isNonProductionBuild: () => boolean; claimHeartbeat: (now: Date) => Promise; collectCounts: () => Promise; + collectSetupIssues: () => Promise; sendHeartbeat: ( installId: string, properties: HeartbeatProperties, @@ -115,8 +121,16 @@ function isNonProductionBuild() { async function telemetryIsDisabled() { if (await isHostedServerAuthMode()) return true; - if (await getOptionalEnvValue("OPENSEO_TELEMETRY_DISABLED")) return true; - if (await getOptionalEnvValue("DO_NOT_TRACK")) return true; + if ( + isTelemetryOptOutValue( + await getOptionalEnvValue("OPENSEO_TELEMETRY_DISABLED"), + ) + ) { + return true; + } + if (isTelemetryOptOutValue(await getOptionalEnvValue("DO_NOT_TRACK"))) { + return true; + } return false; } @@ -254,6 +268,7 @@ const productionDependencies: SelfHostTelemetryDependencies = { isNonProductionBuild, claimHeartbeat, collectCounts, + collectSetupIssues: getSetupIssueSummary, sendHeartbeat, markHeartbeatSent, getDbBackend: getDatabaseProvider, @@ -290,6 +305,7 @@ export async function maybeSendSelfHostHeartbeat( const authMode = getAuthMode(await getOptionalEnvValue("AUTH_MODE")); const counts = await dependencies.collectCounts(); + const setupIssues = await dependencies.collectSetupIssues(); const prevVersion = state.lastVersion && state.lastVersion !== dependencies.version ? state.lastVersion @@ -308,6 +324,7 @@ export async function maybeSendSelfHostHeartbeat( ...(minutesSinceInstall === undefined ? {} : { minutesSinceInstall }), ...counts, mcpToolCalls: state.mcpToolCallCount, + setupIssues, $process_person_profile: false, }); await dependencies.markHeartbeatSent( diff --git a/src/server/lib/setup-status.ts b/src/server/lib/setup-status.ts new file mode 100644 index 0000000..8e0c021 --- /dev/null +++ b/src/server/lib/setup-status.ts @@ -0,0 +1,89 @@ +import { count } from "drizzle-orm"; +import { version } from "../../../package.json"; +import { db } from "@/db"; +import { projects } from "@/db/schema"; +import { getAuthMode } from "@/lib/auth-mode"; +import { runSelfhostChecks } from "@/lib/selfhost-preflight"; +import { getOptionalEnvValue } from "@/server/lib/runtime-env"; + +// "error" blocks core functionality; "warn" degrades a feature. +type SetupCheck = { + status: "ok" | "warn" | "error"; + detail?: string; +}; + +type SelfHostSetupStatus = { + version: string; + authMode: string; + checks: Record; +}; + +// The same env vars the Docker preflight validates; /api/health re-runs the +// shared checks against runtime env so the two reports can never drift. +const CHECK_ENV_VARS = [ + "AUTH_MODE", + "TEAM_DOMAIN", + "POLICY_AUD", + "DATAFORSEO_API_KEY", + "GOOGLE_CLIENT_ID", + "GOOGLE_CLIENT_SECRET", + "BETTER_AUTH_SECRET", + "OPENROUTER_API_KEY", +] as const; + +const LEVEL_TO_STATUS = { + ok: "ok", + info: "ok", + warn: "warn", + fail: "error", +} as const; + +async function checkDatabase(): Promise { + try { + await db.select({ value: count() }).from(projects); + return { status: "ok" }; + } catch (error) { + // This endpoint is unauthenticated: never surface raw driver messages + // (they can name hosts and DB users). The real error goes to the logs. + console.error("health check: database query failed", error); + return { + status: "error", + detail: "Database query failed — check server logs.", + }; + } +} + +export async function getSelfHostSetupStatus(options?: { + skipDatabaseCheck?: boolean; +}): Promise { + const env = Object.fromEntries( + await Promise.all( + CHECK_ENV_VARS.map( + async (name) => [name, await getOptionalEnvValue(name)] as const, + ), + ), + ); + + const checks: Record = {}; + for (const item of runSelfhostChecks(env)) { + checks[item.key] = { + status: LEVEL_TO_STATUS[item.level], + detail: item.message, + }; + } + checks.database = options?.skipDatabaseCheck + ? { status: "ok" } + : await checkDatabase(); + + return { version, authMode: getAuthMode(env.AUTH_MODE), checks }; +} + +// Compact "which checks are unhealthy" list for telemetry: check keys with +// their status, no free-text details (details can name env vars; keep events +// to enumerable values only). +export async function getSetupIssueSummary(): Promise { + const status = await getSelfHostSetupStatus({ skipDatabaseCheck: true }); + return Object.entries(status.checks) + .filter(([, check]) => check.status !== "ok") + .map(([key, check]) => `${key}:${check.status}`); +} diff --git a/src/shared/selfhost-checks.test.ts b/src/shared/selfhost-checks.test.ts new file mode 100644 index 0000000..49a6ec5 --- /dev/null +++ b/src/shared/selfhost-checks.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it } from "vitest"; +import { + isTelemetryOptOutValue, + looksLikeDataForSeoKey, + validateTeamDomain, +} from "./selfhost-checks"; + +describe("validateTeamDomain", () => { + it("accepts a full https team domain", () => { + expect( + validateTeamDomain("https://your-team.cloudflareaccess.com"), + ).toEqual({ ok: true, origin: "https://your-team.cloudflareaccess.com" }); + }); + + it("trims whitespace and trailing slashes", () => { + expect( + validateTeamDomain(" https://your-team.cloudflareaccess.com/ "), + ).toEqual({ ok: true, origin: "https://your-team.cloudflareaccess.com" }); + }); + + it("rejects a bare hostname and tells the user to add https://", () => { + const result = validateTeamDomain("your-team.cloudflareaccess.com"); + expect(result.ok).toBe(false); + if (!result.ok) { + expect(result.message).toContain("https://"); + expect(result.message).toContain( + 'add the https:// prefix to "your-team.cloudflareaccess.com"', + ); + } + }); + + it("rejects http://", () => { + const result = validateTeamDomain("http://your-team.cloudflareaccess.com"); + expect(result.ok).toBe(false); + }); + + it("rejects empty values", () => { + expect(validateTeamDomain("").ok).toBe(false); + }); +}); + +describe("looksLikeDataForSeoKey", () => { + it("accepts base64 of login:password", () => { + expect(looksLikeDataForSeoKey(btoa("user@example.com:secret"))).toBe(true); + }); + + it("rejects a raw dashboard API key", () => { + expect(looksLikeDataForSeoKey("0123456789abcdef0123")).toBe(false); + }); + + it("rejects base64 without a colon", () => { + expect(looksLikeDataForSeoKey(btoa("no-colon-here"))).toBe(false); + }); + + it("tolerates surrounding whitespace", () => { + expect(looksLikeDataForSeoKey(` ${btoa("a:b")} `)).toBe(true); + }); +}); + +describe("isTelemetryOptOutValue", () => { + it("treats unset and empty as opted in", () => { + expect(isTelemetryOptOutValue(undefined)).toBe(false); + expect(isTelemetryOptOutValue(null)).toBe(false); + expect(isTelemetryOptOutValue("")).toBe(false); + }); + + it('treats "1" and "true" as opted out', () => { + expect(isTelemetryOptOutValue("1")).toBe(true); + expect(isTelemetryOptOutValue("true")).toBe(true); + }); + + it('treats explicit "0"/"false"/"no"/"off" as opted in', () => { + expect(isTelemetryOptOutValue("0")).toBe(false); + expect(isTelemetryOptOutValue("false")).toBe(false); + expect(isTelemetryOptOutValue("no")).toBe(false); + expect(isTelemetryOptOutValue("OFF")).toBe(false); + }); +}); diff --git a/src/shared/selfhost-checks.ts b/src/shared/selfhost-checks.ts new file mode 100644 index 0000000..4eadc40 --- /dev/null +++ b/src/shared/selfhost-checks.ts @@ -0,0 +1,53 @@ +// Pure validators for self-host configuration values. Shared by the runtime +// (auth middleware, /api/health, telemetry) and the Docker preflight script, +// so every surface applies the exact same rules and wording. + +export const MIN_BETTER_AUTH_SECRET_LENGTH = 32; + +type TeamDomainResult = + | { ok: true; origin: string } + | { ok: false; message: string }; + +export function validateTeamDomain(value: string): TeamDomainResult { + const normalized = value.trim().replace(/\/+$/, ""); + + try { + const parsed = new URL(normalized); + + if (parsed.protocol !== "https:") { + throw new Error("TEAM_DOMAIN must use https"); + } + + return { ok: true, origin: parsed.origin }; + } catch { + return { + ok: false, + message: + "TEAM_DOMAIN must be a full https URL like https://your-team.cloudflareaccess.com" + + (normalized && !normalized.includes("://") + ? ` — add the https:// prefix to "${normalized}"` + : ""), + }; + } +} + +// DATAFORSEO_API_KEY is NOT the key shown in the DataForSEO dashboard — it is +// base64("login:password"). Decoding it and finding a colon is a cheap sanity +// check that catches the most common paste mistake without a paid API call. +export function looksLikeDataForSeoKey(value: string): boolean { + try { + return atob(value.trim()).includes(":"); + } catch { + return false; + } +} + +// OPENSEO_TELEMETRY_DISABLED / DO_NOT_TRACK semantics: any value except an +// explicit "off" string disables telemetry (fail toward privacy), but +// "0"/"false"/"no"/"off" mean what the operator wrote — telemetry stays on. +export function isTelemetryOptOutValue( + value: string | undefined | null, +): boolean { + if (!value) return false; + return !["0", "false", "no", "off"].includes(value.trim().toLowerCase()); +} diff --git a/web/content/docs/self-hosting/docker.md b/web/content/docs/self-hosting/docker.md index 960bf75..052b0bb 100644 --- a/web/content/docs/self-hosting/docker.md +++ b/web/content/docs/self-hosting/docker.md @@ -24,12 +24,15 @@ Clone the repo, then: git clone https://github.com/every-app/open-seo.git cd open-seo cp .env.example .env +``` + +Set `DATAFORSEO_API_KEY` in `.env` using the [DataForSEO setup guide](/docs/self-hosting#dataforseo-api-key-setup), then start OpenSEO: + +```bash docker compose up -d ``` -Set `DATAFORSEO_API_KEY` in `.env`, then open `http://localhost:` (default `3001`). - -Docker Compose passes `.env` values into the container, and `compose.yaml` enables `CLOUDFLARE_INCLUDE_PROCESS_ENV=true` so the Cloudflare Vite runtime can read them as Worker bindings during local self-hosting. +Open `http://localhost:` (default `3001`). Each container start builds the app and may take 1-2 minutes; follow progress with `docker compose logs -f`. Optional env values: @@ -48,7 +51,7 @@ You can also persist it in `.env`. ## Telemetry -OpenSEO collects anonymized telemetry for core usage events: heartbeats with aggregate counts (installs, users, projects, feature usage) tied to a random install ID, sent every 5 minutes during the first two hours after install, then at most once daily. No URLs, keywords, prompts, emails, or IP-derived location are collected, and idle installs send nothing. +OpenSEO collects anonymized telemetry for core usage events: heartbeats with aggregate counts (installs, users, projects, feature usage) tied to a random install ID, sent every 5 minutes during the first two hours after install, then at most once daily. Telemetry also includes failed setup check names and statuses, never values or error messages. No URLs, keywords, prompts, emails, or IP-derived location are collected, and idle installs send nothing. To disable it, set `OPENSEO_TELEMETRY_DISABLED=1` (or `DO_NOT_TRACK=1`) in `.env`, then run `docker compose up -d --force-recreate open-seo`. @@ -90,11 +93,9 @@ Stop: docker compose down ``` -Stop and remove volumes: +## Health and troubleshooting -```bash -docker compose down -v -``` +Startup checks appear in `docker compose logs` before the build. Once running, `/api/health` reports configuration and database status, and `docker compose ps` reports container health. ## Troubleshooting environment variables