From 2acf94334614e2c13673fe1900128f742e07f373 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Sun, 8 Mar 2026 15:21:01 -0400 Subject: [PATCH 1/7] Standalone OpenSEO docker --- .env.example | 23 ++-- README.md | 6 +- SELF_HOSTING_DOCKER.md | 122 +++++--------------- self-host/Dockerfile.selfhost | 5 + self-host/docker-compose.yml | 40 +------ self-host/scripts/validate-selfhost-env.mjs | 35 +----- worker-configuration.d.ts | 3 +- 7 files changed, 54 insertions(+), 180 deletions(-) diff --git a/.env.example b/.env.example index 0242fc7..1526740 100644 --- a/.env.example +++ b/.env.example @@ -15,22 +15,15 @@ VITE_APP_ID=open-seo # # PORT=3001 # ------------------------------------------------------------- -# --- Docker self-hosting only (see SELF_HOSTING_DOCKER.md) --- -# For Docker, set GATEWAY_URL and VITE_GATEWAY_URL to: -# http://every-app-gateway.localhost:3000 -# VITE_GATEWAY_URL=http://every-app-gateway.localhost:3000 -# GATEWAY_URL=http://every-app-gateway.localhost:3000 -# -# Optional: pin to a specific Gateway release tag. -# By default the latest release is pulled automatically (recommended). -# GATEWAY_RELEASE_TAG= +# --- Docker self-hosting only (standalone local mode) --- +# See SELF_HOSTING_DOCKER.md +# Docker compose runs with BYPASS_GATEWAY_LOCAL_ONLY=true. +# Optional in .env.local because compose already sets it. +# BYPASS_GATEWAY_LOCAL_ONLY=true +# Optional override (defaults to open-seo): +# VITE_APP_ID=open-seo # # See README.md for instructions for how to get this. # DATAFORSEO_API_KEY= -# Required for Gateway auth when running Docker self-host. -# Generate with: pnpm run docker:generate-secrets -# Validate with: pnpm run docker:check-env -# BETTER_AUTH_SECRET= -# JWT_PRIVATE_KEY="" -# JWT_PUBLIC_KEY="" +# Legacy gateway auth keys are not needed in standalone Docker mode. diff --git a/README.md b/README.md index 613f2af..d820b00 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ OpenSEO is an SEO tool for _the people_. If tools like Semrush or Ahrefs are too - [Community](#community) - [Pricing / Costs (Free + API costs)](#pricing--costs) - [Self Hosting (Deploy on Cloudflare) \[5-10 minutes\]](#self-hosting-deploy-on-cloudflare-5-10-minutes) -- [Docker Self Hosting (Gateway + OpenSEO)](#docker-self-hosting-gateway--openseo) +- [Docker Self Hosting (Standalone OpenSEO)](#docker-self-hosting-standalone-openseo) - [Local Development](#local-development) - [Contributing](#contributing) - [SEO API Cost Reference](#seo-api-cost-reference) @@ -144,9 +144,9 @@ npx wrangler secret put DATAFORSEO_API_KEY Now you're all set! Go back to the gateway, click on the OpenSEO app, and start getting better at SEO! -## Docker Self Hosting (Gateway + OpenSEO) +## Docker Self Hosting (Standalone OpenSEO) -If you want a single Docker Compose command that runs both Every App Gateway and OpenSEO together, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). +If you want a single Docker Compose command that runs OpenSEO locally without Gateway auth, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). This runtime uses local dev servers to emulate Cloudflare Worker bindings. It is intended for **local use only** — do not expose ports directly to the public internet. For remote access, use [Tailscale](https://tailscale.com/). For internet-facing deployments, use the Cloudflare deployment path above. See the [security and runtime caveats](./SELF_HOSTING_DOCKER.md#security-and-runtime-caveats) in the Docker guide for details. diff --git a/SELF_HOSTING_DOCKER.md b/SELF_HOSTING_DOCKER.md index e562119..59d1339 100644 --- a/SELF_HOSTING_DOCKER.md +++ b/SELF_HOSTING_DOCKER.md @@ -1,30 +1,21 @@ -# Docker Self-Hosting +# Docker Self-Hosting (Standalone OpenSEO) -This guide runs both Every App Gateway and OpenSEO with one Docker Compose command. +This guide runs OpenSEO as a standalone local service without Every App Gateway. -OpenSEO is an app built on Every App so things like authentication and user management are delegated to the Gateway. Every App is built with deployment to Cloudflare as its target with the goal of making self hosting more accessible to people not already running home labs. Because of that design principle, self hosting with docker is a bit complicated right now, but will hopefully will get smoother over time. - -The stack uses local Cloudflare-compatible runtime behavior (`wrangler` + Vite/worker runtime) so bindings and auth behavior stay close to Workers while running on your own machine or server. +In this mode, OpenSEO runs with `BYPASS_GATEWAY_LOCAL_ONLY=true`, so authentication and Gateway-managed user accounts are disabled. ## Prerequisites - Docker Desktop (or Docker Engine + Docker Compose) -## Runtime model - -- OpenSEO runs from this repository source in containerized local runtime mode. -- Gateway is built from `apps/every-app-gateway` source at a release tag in https://github.com/every-app/every-app - -Default gateway release policy: - -- Source: `every-app/every-app` releases -- Tag: latest release (recommended) - -You can optionally pin to a specific release by setting `GATEWAY_RELEASE_TAG` in your env file. - ## Security and runtime caveats -This stack runs local dev servers to emulate Cloudflare Worker bindings — this is currently the best way to self-host outside Cloudflare, but it means dev-only surfaces (HMR, verbose errors, broader file-serving) are exposed on the serving ports. Do not expose these ports directly to the public internet. If you need remote access, use [Tailscale](https://tailscale.com/) instead of a public tunnel. For internet-facing deployments, use the [Cloudflare deployment path](./README.md#self-hosting-deploy-on-cloudflare-5-10-minutes). +This stack runs local dev servers to emulate Cloudflare Worker bindings and is intended for local use only. + +- Do not expose these ports directly to the public internet. +- There is no Gateway auth in this mode. +- If you need remote access, use [Tailscale](https://tailscale.com/) instead of a public tunnel. +- For internet-facing deployments with auth, use the [Cloudflare deployment path](./README.md#self-hosting-deploy-on-cloudflare-5-10-minutes). ## 1) Configure env values @@ -36,24 +27,15 @@ cp .env.example .env.local Set values as needed in `.env.local`. -Important values: +Required: -- `GATEWAY_URL` and `VITE_GATEWAY_URL` should be set to `http://every-app-gateway.localhost:3000` for local Docker networking and JWT issuer consistency. - - Add a host entry for `every-app-gateway.localhost` if needed: - - macOS/Linux: add `127.0.0.1 every-app-gateway.localhost` to `/etc/hosts` - - Windows: add `127.0.0.1 every-app-gateway.localhost` to `C:\Windows\System32\drivers\etc\hosts` - - If you use a different host or port, set both `GATEWAY_URL` and `VITE_GATEWAY_URL` to the same origin. -- `DATAFORSEO_API_KEY` is required for OpenSEO SEO-data workflows. - - See [README: DataForSEO API Key Setup](./README.md#dataforseo-api-key-setup-5-minutes). -- `BETTER_AUTH_SECRET`, `JWT_PRIVATE_KEY`, and `JWT_PUBLIC_KEY` are required for gateway auth (see how to generate them below) +- `DATAFORSEO_API_KEY` -Generate auth values with: +Optional: -```bash -pnpm run docker:generate-secrets -``` - -Copy the printed lines into `.env.local`. +- `OPENAI_API_KEY` +- `VITE_APP_ID` (defaults to `open-seo`) +- `BYPASS_GATEWAY_LOCAL_ONLY=true` (Docker compose already sets this) Validate env before startup: @@ -61,85 +43,43 @@ Validate env before startup: pnpm run docker:check-env ``` -## 2) Start both services with one command +## 2) Start OpenSEO ```bash pnpm run docker:up ``` -URLs: +URL: -- Gateway: `http://every-app-gateway.localhost:3000` - OpenSEO: `http://localhost:3001` -Gateway boot behavior: +Boot behavior: -- Resolves the latest gateway release tag (unless explicitly pinned). -- Pulls gateway source for that tag, installs dependencies during image build, and runs in local runtime mode. +- Uses dependencies installed during image build. - Applies local D1 migrations on start. -- Persists local gateway Wrangler/D1 state in Docker volume `every_app_gateway_wrangler_state`. - -OpenSEO boot behavior: - -- Uses dependencies installed during image build, then applies local D1 migrations on start. -- Starts local dev runtime (Vite). See [Security and runtime caveats](#security-and-runtime-caveats) above. - -## 3) Bootstrap Gateway and app access - -1. Open `http://every-app-gateway.localhost:3000/sign-up` and create the owner account. -2. In Gateway admin (`/admin/apps`), add OpenSEO: - - App ID: `open-seo` - - App URL: `http://localhost:3001` - - Or whatever port you have this running at -3. Start using OpenSEO by accessing it through Gateway: `http://every-app-gateway.localhost:3000/`. - -## Optional: Run only one service - -Run gateway only: - -```bash -pnpm run docker:check-env -docker compose -f self-host/docker-compose.yml --env-file .env.local up --build gateway -``` - -Run OpenSEO only (expects gateway already reachable at `GATEWAY_URL`): - -```bash -pnpm run docker:check-env -docker compose -f self-host/docker-compose.yml --env-file .env.local up --build open-seo -``` - -## Updating gateway version - -By default this stack pulls the latest published gateway release (recommended). - -To pin to a specific gateway release instead: - -1. Set `GATEWAY_RELEASE_TAG` in `.env.local` (for example `gateway-v0.1.11`). -2. Rebuild gateway: - -```bash -docker compose -f self-host/docker-compose.yml --env-file .env.local build --no-cache gateway -docker compose -f self-host/docker-compose.yml --env-file .env.local up -d gateway -``` - -When tracking latest, rebuild gateway with `--no-cache` to pull newer gateway source for the latest tag. +- Starts local dev runtime (Vite). ## Troubleshooting -- `Issuer must be provided` or `signature verification failed`: make sure `GATEWAY_URL` and `VITE_GATEWAY_URL` both point to `http://every-app-gateway.localhost:3000`, then clear browser cookies/storage for `localhost` and `every-app-gateway.localhost` and sign in again. - OpenSEO env values seem stale: restart OpenSEO: ```bash docker compose -f self-host/docker-compose.yml --env-file .env.local up -d --build open-seo ``` +- If migrations fail on first run, rebuild and retry: + +```bash +pnpm run docker:down +pnpm run docker:up +``` + ## Stop and cleanup Stop stack: ```bash -docker compose -f self-host/docker-compose.yml --env-file .env.local down +pnpm run docker:down ``` Stop and remove Docker volumes: @@ -147,9 +87,3 @@ Stop and remove Docker volumes: ```bash docker compose -f self-host/docker-compose.yml --env-file .env.local down -v ``` - -To reset only the gateway local DB state explicitly: - -```bash -docker volume rm every_app_gateway_wrangler_state -``` diff --git a/self-host/Dockerfile.selfhost b/self-host/Dockerfile.selfhost index 3c6d30a..ac8bcb2 100644 --- a/self-host/Dockerfile.selfhost +++ b/self-host/Dockerfile.selfhost @@ -2,6 +2,11 @@ FROM node:22 ENV PNPM_HOME=/pnpm ENV PATH=$PNPM_HOME:$PATH +ENV PORT=3001 +ENV BYPASS_GATEWAY_LOCAL_ONLY=true +ENV VITE_APP_ID=open-seo +ENV VITE_GATEWAY_URL=http://localhost:3001 +ENV GATEWAY_URL=http://localhost:3001 WORKDIR /app diff --git a/self-host/docker-compose.yml b/self-host/docker-compose.yml index 34cc193..efe11b0 100644 --- a/self-host/docker-compose.yml +++ b/self-host/docker-compose.yml @@ -1,28 +1,4 @@ services: - gateway: - build: - context: .. - dockerfile: self-host/Dockerfile.gateway-selfhost - args: - GATEWAY_RELEASE_TAG: ${GATEWAY_RELEASE_TAG:-} - env_file: - - ../.env.local - command: - [ - "sh", - "-c", - "pnpm run db:migrate:local && pnpm exec vite dev --host 0.0.0.0 --port 3000", - ] - ports: - - "127.0.0.1:3000:3000" - volumes: - - ../.env.local:/app/.env:ro - - gateway_wrangler_state:/app/.wrangler/state - networks: - selfhost: - aliases: - - every-app-gateway.localhost - open-seo: build: context: .. @@ -30,9 +6,10 @@ services: working_dir: /app environment: - PORT=3001 - - VITE_APP_ID=${VITE_APP_ID} - - VITE_GATEWAY_URL=${VITE_GATEWAY_URL} - - GATEWAY_URL=${GATEWAY_URL} + - BYPASS_GATEWAY_LOCAL_ONLY=true + - VITE_APP_ID=${VITE_APP_ID:-open-seo} + - VITE_GATEWAY_URL=${VITE_GATEWAY_URL:-http://localhost:3001} + - GATEWAY_URL=${GATEWAY_URL:-http://localhost:3001} - DATAFORSEO_API_KEY=${DATAFORSEO_API_KEY} - OPENAI_API_KEY=${OPENAI_API_KEY:-} - VITE_SHOW_DEVTOOLS=false @@ -42,22 +19,13 @@ services: "-c", "pnpm run db:migrate:local && pnpm exec vite dev --host 0.0.0.0 --port 3001", ] - depends_on: - - gateway ports: - "127.0.0.1:3001:3001" volumes: - ..:/app - open_seo_node_modules:/app/node_modules - open_seo_pnpm_store:/pnpm/store - networks: - - selfhost - -networks: - selfhost: volumes: - gateway_wrangler_state: - name: every_app_gateway_wrangler_state open_seo_node_modules: open_seo_pnpm_store: diff --git a/self-host/scripts/validate-selfhost-env.mjs b/self-host/scripts/validate-selfhost-env.mjs index c454c6b..586af15 100644 --- a/self-host/scripts/validate-selfhost-env.mjs +++ b/self-host/scripts/validate-selfhost-env.mjs @@ -45,15 +45,7 @@ if (!existsSync(envPath)) { const env = parseEnvFile(envPath); -const requiredKeys = [ - "GATEWAY_URL", - "VITE_GATEWAY_URL", - "VITE_APP_ID", - "DATAFORSEO_API_KEY", - "BETTER_AUTH_SECRET", - "JWT_PRIVATE_KEY", - "JWT_PUBLIC_KEY", -]; +const requiredKeys = ["DATAFORSEO_API_KEY"]; const missingKeys = requiredKeys.filter((key) => isBlank(env[key])); if (missingKeys.length > 0) { @@ -61,32 +53,13 @@ if (missingKeys.length > 0) { for (const key of missingKeys) { console.error(`- ${key}`); } - console.error("\nGenerate auth keys with: pnpm run docker:generate-secrets"); process.exit(1); } -if (env.GATEWAY_URL !== env.VITE_GATEWAY_URL) { - console.error( - "GATEWAY_URL and VITE_GATEWAY_URL must match for auth issuer consistency.", +if (isBlank(env.BYPASS_GATEWAY_LOCAL_ONLY)) { + console.warn( + "Warning: BYPASS_GATEWAY_LOCAL_ONLY is not set in .env.local. Docker sets it automatically.", ); - process.exit(1); -} - -if (!env.JWT_PRIVATE_KEY.includes("BEGIN PRIVATE KEY")) { - console.error("JWT_PRIVATE_KEY does not look like a PEM private key."); - process.exit(1); -} - -if (!env.JWT_PUBLIC_KEY.includes("BEGIN PUBLIC KEY")) { - console.error("JWT_PUBLIC_KEY does not look like a PEM public key."); - process.exit(1); -} - -if (env.BETTER_AUTH_SECRET.trim().length < 32) { - console.error( - "BETTER_AUTH_SECRET is too short. Generate a new one with docker:generate-secrets.", - ); - process.exit(1); } console.log(`Env validation passed: ${envPath}`); diff --git a/worker-configuration.d.ts b/worker-configuration.d.ts index a62f0c8..6064aa9 100644 --- a/worker-configuration.d.ts +++ b/worker-configuration.d.ts @@ -12,6 +12,7 @@ declare namespace Cloudflare { VITE_APP_ID: string; VITE_GATEWAY_URL: string; GATEWAY_URL: string; + GATEWAY_APP_API_TOKEN: string; EVERY_APP_GATEWAY: Fetcher /* every-app-gateway */; SITE_AUDIT_WORKFLOW: Workflow[0]['payload']>; } @@ -21,7 +22,7 @@ type StringifyValues> = { [Binding in keyof EnvType]: EnvType[Binding] extends string ? EnvType[Binding] : string; }; declare namespace NodeJS { - interface ProcessEnv extends StringifyValues> {} + interface ProcessEnv extends StringifyValues> {} } // Begin runtime types From 71da6709248ba14237135073dc5cc16b709f4b00 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:20:23 -0400 Subject: [PATCH 2/7] Simplify Docker self-hosting commands and env flow --- .env.example | 2 +- .worktreeinclude | 3 - README.md | 9 ++- SELF_HOSTING_DOCKER.md | 29 ++++----- self-host/docker-compose.yml => compose.yaml | 4 +- package.json | 5 +- .../scripts/generate-selfhost-secrets.mjs | 2 +- self-host/scripts/validate-selfhost-env.mjs | 65 ------------------- 8 files changed, 24 insertions(+), 95 deletions(-) delete mode 100644 .worktreeinclude rename self-host/docker-compose.yml => compose.yaml (96%) delete mode 100644 self-host/scripts/validate-selfhost-env.mjs diff --git a/.env.example b/.env.example index 1526740..dc686dc 100644 --- a/.env.example +++ b/.env.example @@ -18,7 +18,7 @@ VITE_APP_ID=open-seo # --- Docker self-hosting only (standalone local mode) --- # See SELF_HOSTING_DOCKER.md # Docker compose runs with BYPASS_GATEWAY_LOCAL_ONLY=true. -# Optional in .env.local because compose already sets it. +# Optional in .env because compose already sets it. # BYPASS_GATEWAY_LOCAL_ONLY=true # Optional override (defaults to open-seo): # VITE_APP_ID=open-seo diff --git a/.worktreeinclude b/.worktreeinclude deleted file mode 100644 index bb876dc..0000000 --- a/.worktreeinclude +++ /dev/null @@ -1,3 +0,0 @@ -.env.local -.wrangler -node_modules/ diff --git a/README.md b/README.md index d820b00..a959bdd 100644 --- a/README.md +++ b/README.md @@ -146,9 +146,14 @@ Now you're all set! Go back to the gateway, click on the OpenSEO app, and start ## Docker Self Hosting (Standalone OpenSEO) -If you want a single Docker Compose command that runs OpenSEO locally without Gateway auth, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). +Quickstart: -This runtime uses local dev servers to emulate Cloudflare Worker bindings. It is intended for **local use only** — do not expose ports directly to the public internet. For remote access, use [Tailscale](https://tailscale.com/). For internet-facing deployments, use the Cloudflare deployment path above. See the [security and runtime caveats](./SELF_HOSTING_DOCKER.md#security-and-runtime-caveats) in the Docker guide for details. +1. `cp .env.example .env` +2. Set `DATAFORSEO_API_KEY` in `.env` +3. `docker compose up --build` +4. Open `http://localhost:3001` + +For runtime details, caveats, and troubleshooting, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). ## Local Development diff --git a/SELF_HOSTING_DOCKER.md b/SELF_HOSTING_DOCKER.md index 59d1339..78825e4 100644 --- a/SELF_HOSTING_DOCKER.md +++ b/SELF_HOSTING_DOCKER.md @@ -10,22 +10,21 @@ In this mode, OpenSEO runs with `BYPASS_GATEWAY_LOCAL_ONLY=true`, so authenticat ## Security and runtime caveats -This stack runs local dev servers to emulate Cloudflare Worker bindings and is intended for local use only. +This stack is local-first and uses dev runtimes to emulate Cloudflare Worker bindings. - Do not expose these ports directly to the public internet. -- There is no Gateway auth in this mode. -- If you need remote access, use [Tailscale](https://tailscale.com/) instead of a public tunnel. -- For internet-facing deployments with auth, use the [Cloudflare deployment path](./README.md#self-hosting-deploy-on-cloudflare-5-10-minutes). +- There is no built-in Gateway auth in this mode. +- If you expose it beyond localhost, put it behind the same authentication layer you use for your other self-hosted services (or use the [Cloudflare deployment path](./README.md#self-hosting-deploy-on-cloudflare-5-10-minutes)). ## 1) Configure env values From the repository root: ```bash -cp .env.example .env.local +cp .env.example .env ``` -Set values as needed in `.env.local`. +Set values as needed in `.env`. Required: @@ -37,16 +36,10 @@ Optional: - `VITE_APP_ID` (defaults to `open-seo`) - `BYPASS_GATEWAY_LOCAL_ONLY=true` (Docker compose already sets this) -Validate env before startup: - -```bash -pnpm run docker:check-env -``` - ## 2) Start OpenSEO ```bash -pnpm run docker:up +docker compose up --build ``` URL: @@ -64,14 +57,14 @@ Boot behavior: - OpenSEO env values seem stale: restart OpenSEO: ```bash -docker compose -f self-host/docker-compose.yml --env-file .env.local up -d --build open-seo +docker compose up -d --build open-seo ``` - If migrations fail on first run, rebuild and retry: ```bash -pnpm run docker:down -pnpm run docker:up +docker compose down +docker compose up --build ``` ## Stop and cleanup @@ -79,11 +72,11 @@ pnpm run docker:up Stop stack: ```bash -pnpm run docker:down +docker compose down ``` Stop and remove Docker volumes: ```bash -docker compose -f self-host/docker-compose.yml --env-file .env.local down -v +docker compose down -v ``` diff --git a/self-host/docker-compose.yml b/compose.yaml similarity index 96% rename from self-host/docker-compose.yml rename to compose.yaml index efe11b0..cf37fe0 100644 --- a/self-host/docker-compose.yml +++ b/compose.yaml @@ -1,7 +1,7 @@ services: open-seo: build: - context: .. + context: . dockerfile: self-host/Dockerfile.selfhost working_dir: /app environment: @@ -22,7 +22,7 @@ services: ports: - "127.0.0.1:3001:3001" volumes: - - ..:/app + - .:/app - open_seo_node_modules:/app/node_modules - open_seo_pnpm_store:/pnpm/store diff --git a/package.json b/package.json index a8ea8d4..b55f8d8 100644 --- a/package.json +++ b/package.json @@ -20,9 +20,8 @@ "db:migrate:prod": "npx everyapp app remote-d1-shell -- drizzle-kit migrate --config=drizzle-prod.config.ts", "db:studio:local": "drizzle-kit studio", "db:studio:prod": "npx everyapp app remote-d1-shell -- drizzle-kit studio --config=drizzle-prod.config.ts", - "docker:check-env": "node ./self-host/scripts/validate-selfhost-env.mjs", - "docker:up": "pnpm run docker:check-env && docker compose -f self-host/docker-compose.yml --env-file .env.local up --build", - "docker:down": "docker compose -f self-host/docker-compose.yml --env-file .env.local down", + "docker:up": "docker compose up --build", + "docker:down": "docker compose down", "docker:generate-secrets": "node ./self-host/scripts/generate-selfhost-secrets.mjs", "knip": "knip", "ci": "prettier --check . && knip && tsc --noEmit && oxlint ." diff --git a/self-host/scripts/generate-selfhost-secrets.mjs b/self-host/scripts/generate-selfhost-secrets.mjs index 391ff0e..f771af0 100644 --- a/self-host/scripts/generate-selfhost-secrets.mjs +++ b/self-host/scripts/generate-selfhost-secrets.mjs @@ -12,7 +12,7 @@ const { privateKey, publicKey } = generateKeyPairSync("rsa", { }); console.log( - "# Copy these lines into .env.local\n# See SELF_HOSTING_DOCKER.md for setup instructions", + "# Copy these lines into .env\n# See SELF_HOSTING_DOCKER.md for setup instructions", ); console.log(`BETTER_AUTH_SECRET=${betterAuthSecret}`); console.log(`JWT_PRIVATE_KEY="${escapeForEnv(privateKey)}"`); diff --git a/self-host/scripts/validate-selfhost-env.mjs b/self-host/scripts/validate-selfhost-env.mjs deleted file mode 100644 index 586af15..0000000 --- a/self-host/scripts/validate-selfhost-env.mjs +++ /dev/null @@ -1,65 +0,0 @@ -import { existsSync, readFileSync } from "node:fs"; - -function parseEnvFile(path) { - const raw = readFileSync(path, "utf8"); - const out = {}; - - for (const line of raw.split(/\r?\n/)) { - const trimmed = line.trim(); - if (trimmed.length === 0 || trimmed.startsWith("#")) { - continue; - } - - const equalsIndex = trimmed.indexOf("="); - if (equalsIndex <= 0) { - continue; - } - - const key = trimmed.slice(0, equalsIndex).trim(); - let value = trimmed.slice(equalsIndex + 1).trim(); - - if ( - (value.startsWith('"') && value.endsWith('"')) || - (value.startsWith("'") && value.endsWith("'")) - ) { - value = value.slice(1, -1); - } - - out[key] = value.replace(/\\n/g, "\n"); - } - - return out; -} - -function isBlank(value) { - return typeof value !== "string" || value.trim().length === 0; -} - -const envPath = process.argv[2] || ".env.local"; - -if (!existsSync(envPath)) { - console.error(`Missing env file: ${envPath}`); - console.error("Create it with: cp .env.example .env.local"); - process.exit(1); -} - -const env = parseEnvFile(envPath); - -const requiredKeys = ["DATAFORSEO_API_KEY"]; - -const missingKeys = requiredKeys.filter((key) => isBlank(env[key])); -if (missingKeys.length > 0) { - console.error("Missing required keys in env file:"); - for (const key of missingKeys) { - console.error(`- ${key}`); - } - process.exit(1); -} - -if (isBlank(env.BYPASS_GATEWAY_LOCAL_ONLY)) { - console.warn( - "Warning: BYPASS_GATEWAY_LOCAL_ONLY is not set in .env.local. Docker sets it automatically.", - ); -} - -console.log(`Env validation passed: ${envPath}`); From 2d01ce5196ac22cde9eff254c224fb29e9c64646 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:25:47 -0400 Subject: [PATCH 3/7] Remove obsolete Docker helper scripts --- README.md | 2 +- SELF_HOSTING_DOCKER.md | 10 ++++++++-- package.json | 3 --- .../scripts/generate-selfhost-secrets.mjs | 19 ------------------- 4 files changed, 9 insertions(+), 25 deletions(-) delete mode 100644 self-host/scripts/generate-selfhost-secrets.mjs diff --git a/README.md b/README.md index a959bdd..97cbfa7 100644 --- a/README.md +++ b/README.md @@ -150,7 +150,7 @@ Quickstart: 1. `cp .env.example .env` 2. Set `DATAFORSEO_API_KEY` in `.env` -3. `docker compose up --build` +3. `docker compose up` 4. Open `http://localhost:3001` For runtime details, caveats, and troubleshooting, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). diff --git a/SELF_HOSTING_DOCKER.md b/SELF_HOSTING_DOCKER.md index 78825e4..543cc87 100644 --- a/SELF_HOSTING_DOCKER.md +++ b/SELF_HOSTING_DOCKER.md @@ -39,7 +39,7 @@ Optional: ## 2) Start OpenSEO ```bash -docker compose up --build +docker compose up ``` URL: @@ -57,13 +57,19 @@ Boot behavior: - OpenSEO env values seem stale: restart OpenSEO: ```bash -docker compose up -d --build open-seo +docker compose up -d open-seo ``` - If migrations fail on first run, rebuild and retry: ```bash docker compose down +docker compose up +``` + +If you update dependencies or Docker build config, force a rebuild: + +```bash docker compose up --build ``` diff --git a/package.json b/package.json index b55f8d8..27dbb15 100644 --- a/package.json +++ b/package.json @@ -20,9 +20,6 @@ "db:migrate:prod": "npx everyapp app remote-d1-shell -- drizzle-kit migrate --config=drizzle-prod.config.ts", "db:studio:local": "drizzle-kit studio", "db:studio:prod": "npx everyapp app remote-d1-shell -- drizzle-kit studio --config=drizzle-prod.config.ts", - "docker:up": "docker compose up --build", - "docker:down": "docker compose down", - "docker:generate-secrets": "node ./self-host/scripts/generate-selfhost-secrets.mjs", "knip": "knip", "ci": "prettier --check . && knip && tsc --noEmit && oxlint ." }, diff --git a/self-host/scripts/generate-selfhost-secrets.mjs b/self-host/scripts/generate-selfhost-secrets.mjs deleted file mode 100644 index f771af0..0000000 --- a/self-host/scripts/generate-selfhost-secrets.mjs +++ /dev/null @@ -1,19 +0,0 @@ -import { generateKeyPairSync, randomBytes } from "node:crypto"; - -function escapeForEnv(value) { - return value.replace(/\r?\n/g, "\\n"); -} - -const betterAuthSecret = randomBytes(32).toString("base64"); -const { privateKey, publicKey } = generateKeyPairSync("rsa", { - modulusLength: 2048, - publicKeyEncoding: { type: "spki", format: "pem" }, - privateKeyEncoding: { type: "pkcs8", format: "pem" }, -}); - -console.log( - "# Copy these lines into .env\n# See SELF_HOSTING_DOCKER.md for setup instructions", -); -console.log(`BETTER_AUTH_SECRET=${betterAuthSecret}`); -console.log(`JWT_PRIVATE_KEY="${escapeForEnv(privateKey)}"`); -console.log(`JWT_PUBLIC_KEY="${escapeForEnv(publicKey)}"`); From d490dde51f66c025c89c060457624c4f55343c52 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:37:58 -0400 Subject: [PATCH 4/7] Simplify Docker self-hosting configuration --- .env.example | 6 ++++-- README.md | 6 +++--- SELF_HOSTING_DOCKER.md | 8 ++++---- compose.yaml | 11 +++++------ knip.jsonc | 2 -- self-host/Dockerfile.gateway-selfhost | 22 ---------------------- self-host/scripts/download-archive.mjs | 20 -------------------- self-host/scripts/resolve-gateway-tag.mjs | 23 ----------------------- 8 files changed, 16 insertions(+), 82 deletions(-) delete mode 100644 self-host/Dockerfile.gateway-selfhost delete mode 100644 self-host/scripts/download-archive.mjs delete mode 100644 self-host/scripts/resolve-gateway-tag.mjs diff --git a/.env.example b/.env.example index dc686dc..bff168a 100644 --- a/.env.example +++ b/.env.example @@ -15,9 +15,11 @@ VITE_APP_ID=open-seo # # PORT=3001 # ------------------------------------------------------------- -# --- Docker self-hosting only (standalone local mode) --- +# --- Docker self-hosting only (local mode) --- # See SELF_HOSTING_DOCKER.md # Docker compose runs with BYPASS_GATEWAY_LOCAL_ONLY=true. +# Optional host/container port (defaults to 3001): +# PORT=3001 # Optional in .env because compose already sets it. # BYPASS_GATEWAY_LOCAL_ONLY=true # Optional override (defaults to open-seo): @@ -26,4 +28,4 @@ VITE_APP_ID=open-seo # # See README.md for instructions for how to get this. # DATAFORSEO_API_KEY= -# Legacy gateway auth keys are not needed in standalone Docker mode. +# Legacy gateway auth keys are not needed in Docker self-host mode. diff --git a/README.md b/README.md index 97cbfa7..9725b4d 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ OpenSEO is an SEO tool for _the people_. If tools like Semrush or Ahrefs are too - [Community](#community) - [Pricing / Costs (Free + API costs)](#pricing--costs) - [Self Hosting (Deploy on Cloudflare) \[5-10 minutes\]](#self-hosting-deploy-on-cloudflare-5-10-minutes) -- [Docker Self Hosting (Standalone OpenSEO)](#docker-self-hosting-standalone-openseo) +- [Docker Self Hosting](#docker-self-hosting) - [Local Development](#local-development) - [Contributing](#contributing) - [SEO API Cost Reference](#seo-api-cost-reference) @@ -144,14 +144,14 @@ npx wrangler secret put DATAFORSEO_API_KEY Now you're all set! Go back to the gateway, click on the OpenSEO app, and start getting better at SEO! -## Docker Self Hosting (Standalone OpenSEO) +## Docker Self Hosting Quickstart: 1. `cp .env.example .env` 2. Set `DATAFORSEO_API_KEY` in `.env` 3. `docker compose up` -4. Open `http://localhost:3001` +4. Open `http://localhost:` (default `3001`) For runtime details, caveats, and troubleshooting, see [`SELF_HOSTING_DOCKER.md`](./SELF_HOSTING_DOCKER.md). diff --git a/SELF_HOSTING_DOCKER.md b/SELF_HOSTING_DOCKER.md index 543cc87..bc0581e 100644 --- a/SELF_HOSTING_DOCKER.md +++ b/SELF_HOSTING_DOCKER.md @@ -1,6 +1,6 @@ -# Docker Self-Hosting (Standalone OpenSEO) +# Docker Self-Hosting -This guide runs OpenSEO as a standalone local service without Every App Gateway. +This guide runs OpenSEO as a local service without Every App Gateway. In this mode, OpenSEO runs with `BYPASS_GATEWAY_LOCAL_ONLY=true`, so authentication and Gateway-managed user accounts are disabled. @@ -32,7 +32,7 @@ Required: Optional: -- `OPENAI_API_KEY` +- `PORT` (defaults to `3001`) - `VITE_APP_ID` (defaults to `open-seo`) - `BYPASS_GATEWAY_LOCAL_ONLY=true` (Docker compose already sets this) @@ -44,7 +44,7 @@ docker compose up URL: -- OpenSEO: `http://localhost:3001` +- OpenSEO: `http://localhost:` (defaults to `3001`) Boot behavior: diff --git a/compose.yaml b/compose.yaml index cf37fe0..1b5231f 100644 --- a/compose.yaml +++ b/compose.yaml @@ -5,22 +5,21 @@ services: dockerfile: self-host/Dockerfile.selfhost working_dir: /app environment: - - PORT=3001 + - PORT=${PORT:-3001} - BYPASS_GATEWAY_LOCAL_ONLY=true - VITE_APP_ID=${VITE_APP_ID:-open-seo} - - VITE_GATEWAY_URL=${VITE_GATEWAY_URL:-http://localhost:3001} - - GATEWAY_URL=${GATEWAY_URL:-http://localhost:3001} + - VITE_GATEWAY_URL=${VITE_GATEWAY_URL:-http://localhost:${PORT:-3001}} + - GATEWAY_URL=${GATEWAY_URL:-http://localhost:${PORT:-3001}} - DATAFORSEO_API_KEY=${DATAFORSEO_API_KEY} - - OPENAI_API_KEY=${OPENAI_API_KEY:-} - VITE_SHOW_DEVTOOLS=false command: [ "sh", "-c", - "pnpm run db:migrate:local && pnpm exec vite dev --host 0.0.0.0 --port 3001", + "pnpm run db:migrate:local && pnpm exec vite dev --host 0.0.0.0 --port ${PORT:-3001}", ] ports: - - "127.0.0.1:3001:3001" + - "127.0.0.1:${PORT:-3001}:${PORT:-3001}" volumes: - .:/app - open_seo_node_modules:/app/node_modules diff --git a/knip.jsonc b/knip.jsonc index dc80a9e..a63d875 100644 --- a/knip.jsonc +++ b/knip.jsonc @@ -8,8 +8,6 @@ "drizzle.config.ts", // DB index re-exports schema for convenience "src/db/index.ts", - // Docker self-host runtime entrypoint scripts - "self-host/scripts/*.mjs", ], "project": ["**/*.{js,mjs,ts,tsx}", "!src/routeTree.gen.ts"], "ignore": [ diff --git a/self-host/Dockerfile.gateway-selfhost b/self-host/Dockerfile.gateway-selfhost deleted file mode 100644 index bdaf80b..0000000 --- a/self-host/Dockerfile.gateway-selfhost +++ /dev/null @@ -1,22 +0,0 @@ -FROM node:22 - -ENV PNPM_HOME=/pnpm -ENV PATH=$PNPM_HOME:$PATH - -WORKDIR /app - -RUN corepack enable - -COPY self-host/scripts/resolve-gateway-tag.mjs self-host/scripts/download-archive.mjs /tmp/scripts/ - -ARG GATEWAY_RELEASE_TAG= -RUN gateway_tag=$(node /tmp/scripts/resolve-gateway-tag.mjs) \ - && echo "Downloading Gateway source for tag: ${gateway_tag}" \ - && node /tmp/scripts/download-archive.mjs "https://github.com/every-app/every-app/archive/refs/tags/${gateway_tag}.tar.gz" /tmp/every-app-source.tar.gz \ - && mkdir -p /tmp/every-app-source \ - && tar -xzf /tmp/every-app-source.tar.gz -C /tmp/every-app-source \ - && source_root=$(find /tmp/every-app-source -mindepth 1 -maxdepth 1 -type d | head -n 1) \ - && cp -R "$source_root/apps/every-app-gateway/." /app \ - && rm -rf /tmp/every-app-source /tmp/every-app-source.tar.gz /tmp/scripts - -RUN pnpm install --frozen-lockfile diff --git a/self-host/scripts/download-archive.mjs b/self-host/scripts/download-archive.mjs deleted file mode 100644 index 22c5785..0000000 --- a/self-host/scripts/download-archive.mjs +++ /dev/null @@ -1,20 +0,0 @@ -// Downloads a file from a URL and writes it to a local path. -// Usage: node self-host/scripts/download-archive.mjs - -import { writeFileSync } from "node:fs"; - -const [url, outputPath] = process.argv.slice(2); - -if (!url || !outputPath) { - console.error( - "Usage: node self-host/scripts/download-archive.mjs ", - ); - process.exit(1); -} - -const res = await fetch(url); -if (!res.ok) { - throw new Error("Failed to download source archive: " + String(res.status)); -} - -writeFileSync(outputPath, Buffer.from(await res.arrayBuffer())); diff --git a/self-host/scripts/resolve-gateway-tag.mjs b/self-host/scripts/resolve-gateway-tag.mjs deleted file mode 100644 index 2880785..0000000 --- a/self-host/scripts/resolve-gateway-tag.mjs +++ /dev/null @@ -1,23 +0,0 @@ -// Resolves the gateway release tag to use for the selfhost Docker build. -// If GATEWAY_RELEASE_TAG is set, prints it and exits. -// Otherwise fetches the latest release tag from the GitHub API. - -const tag = process.env.GATEWAY_RELEASE_TAG; -if (tag) { - process.stdout.write(tag); - process.exit(0); -} - -const res = await fetch( - "https://api.github.com/repos/every-app/every-app/releases/latest", -); -if (!res.ok) { - throw new Error("Failed to resolve latest release: " + String(res.status)); -} - -const parsed = await res.json(); -if (!parsed.tag_name) { - throw new Error("Missing tag_name in latest release payload"); -} - -process.stdout.write(parsed.tag_name); From 71b593615cfc3f9c64a2c2a010519497ae7deb04 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:41:49 -0400 Subject: [PATCH 5/7] Remove redundant env defaults from selfhost Dockerfile --- self-host/Dockerfile.selfhost | 5 ----- 1 file changed, 5 deletions(-) diff --git a/self-host/Dockerfile.selfhost b/self-host/Dockerfile.selfhost index ac8bcb2..3c6d30a 100644 --- a/self-host/Dockerfile.selfhost +++ b/self-host/Dockerfile.selfhost @@ -2,11 +2,6 @@ FROM node:22 ENV PNPM_HOME=/pnpm ENV PATH=$PNPM_HOME:$PATH -ENV PORT=3001 -ENV BYPASS_GATEWAY_LOCAL_ONLY=true -ENV VITE_APP_ID=open-seo -ENV VITE_GATEWAY_URL=http://localhost:3001 -ENV GATEWAY_URL=http://localhost:3001 WORKDIR /app From d60276de9e6194a1911d6203b3adc08313256270 Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:47:26 -0400 Subject: [PATCH 6/7] Remove Cloudflare self-hosting section from README --- README.md | 92 +++++-------------------------------------------------- 1 file changed, 7 insertions(+), 85 deletions(-) diff --git a/README.md b/README.md index 9725b4d..bcc08a8 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ OpenSEO is an SEO tool for _the people_. If tools like Semrush or Ahrefs are too - [Roadmap](#roadmap) - [Community](#community) - [Pricing / Costs (Free + API costs)](#pricing--costs) -- [Self Hosting (Deploy on Cloudflare) \[5-10 minutes\]](#self-hosting-deploy-on-cloudflare-5-10-minutes) +- [DataForSEO API Key Setup](#dataforseo-api-key-setup) - [Docker Self Hosting](#docker-self-hosting) - [Local Development](#local-development) - [Contributing](#contributing) @@ -63,68 +63,7 @@ There are two separate things: For cost estimates, see [DataForSEO API Cost Reference](#seo-api-cost-reference). -## Self Hosting (Deploy on Cloudflare) [5-10 minutes] - -> [!TIP] -> If anything in this section is confusing or unfamiliar like running terminal commands, copy this link into ChatGPT or Claude and ask it explain. - -OpenSEO is built on [Every App](https://github.com/every-app/every-app), a platform for easily self-hosting open source apps like OpenSEO in your own Cloudflare account. Cloudflare enables much more powerful functionality than is possible running on your own computer or on a VPS. - -_Windows Users_ - -This has not been tested on Windows. Please let me know if you run into problems. Using WSL will likely work better. Also, try using [fly.io Sprites](https://sprites.dev/) to get a linux sandbox for free if you get totally stuck. - -### Video Walkthrough - -This video walks through setting up the Gateway and self hosting OpenSEO. If you run into any problems, reference the [Community](#community) section for how to reach out. - -https://github.com/user-attachments/assets/e40d5089-971f-43c9-85ff-1213aea35156 - -### Prerequisites - -If you don't want to make a Cloudflare account yet (its easy!) and just want to test out OpenSEO, skip to the [Run Locally](#seo-api-cost-reference) section. - -1. Install [Node.js](https://nodejs.org/) (includes `npx`). -2. Create a Cloudflare account: [dash.cloudflare.com/sign-up](https://dash.cloudflare.com/sign-up) -3. Authenticate Wrangler: - -```sh -npx wrangler login -``` - -4. Deploy the Every App Gateway (one-time per account): - -```sh -npx everyapp gateway deploy -``` - -5. Create an Account - -- Follow the link output by the last command to create an account. You will access OpenSEO through this account. - -### Self Host OpenSEO - -Deploy the app to cloudflare. - -1. Clone the repo to your machine - -```sh -git clone https://github.com/every-app/open-seo.git -``` - -2. Switch to the directory - -```sh -cd open-seo -``` - -3. Self host via the Every App CLI - -```sh -npx everyapp app deploy -``` - -#### DataForSEO API Key Setup [5 minutes] +## DataForSEO API Key Setup OpenSEO use DataForSEO to get the SEO info. You need an API key to connect OpenSEO to the service. @@ -136,13 +75,10 @@ OpenSEO use DataForSEO to get the SEO info. You need an API key to connect OpenS printf '%s' 'YOUR_LOGIN:YOUR_PASSWORD' | base64 ``` -4. Set this as a secret in Cloudflare. Use the value from the previous step when prompted. +4. Set this as `DATAFORSEO_API_KEY` in your environment file: -```sh -npx wrangler secret put DATAFORSEO_API_KEY -``` - -Now you're all set! Go back to the gateway, click on the OpenSEO app, and start getting better at SEO! +- Docker self-hosting: `.env` +- Local development: `.env.local` ## Docker Self Hosting @@ -177,7 +113,7 @@ cp .env.example .env.local pnpm install # Initialize local DB schema (required on a fresh machine) pnpm run db:migrate:local -# This runs in BYPASS_GATEWAY mode so that you don't need to set up the Every App gateway. This is fine for local use and best for local dev / quick testing since you don't need to access the app through the gatway +# This runs in BYPASS_GATEWAY mode for local use and quick testing. pnpm dev:agents ``` @@ -187,21 +123,7 @@ pnpm dev:agents When using a git worktree, portless prefixes the branch name, for example `http://feature-name.open-seo.localhost:1355`. -Running locally is the fastest way to test core flows. In the future, local mode will not include some Cloudflare-backed capabilities (for example cron-based rank tracking and infrastructure-powered performance improvements for heavier audits). - -### Local Development (In Gateway) - -If you want auth or any other gateway only features enabled for local dev, you must access through your Gateway and set a devUrl on the app. - -See the next section which is a workflow for agents which works much better with worktrees and doens't require the Gateway. - -```sh -# This configures your .env.local, installs dependencies and runs migrations against your local database. -npx everyapp app setup-local - -# terminal 1: start once and keep running -pnpm dev:agents -``` +Running locally is the fastest way to test core flows. ### Local Development Workflow (for coding agents) From 0bd61ae26422b700157e4dd719cd4e9a20607a8b Mon Sep 17 00:00:00 2001 From: Ben Senescu Date: Mon, 9 Mar 2026 23:49:16 -0400 Subject: [PATCH 7/7] Link portless in local development docs --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index bcc08a8..2399487 100644 --- a/README.md +++ b/README.md @@ -119,9 +119,9 @@ pnpm dev:agents `pnpm dev` runs on `http://localhost:3001` by default (or `PORT` from `.env.local`). -`pnpm dev:agents` runs through portless at `http://open-seo.localhost:1355` by default. +`pnpm dev:agents` runs through [portless](https://github.com/vercel-labs/portless) at `http://open-seo.localhost:1355` by default. -When using a git worktree, portless prefixes the branch name, for example `http://feature-name.open-seo.localhost:1355`. +When using a git worktree, [portless](https://github.com/vercel-labs/portless) prefixes the branch name, for example `http://feature-name.open-seo.localhost:1355`. Running locally is the fastest way to test core flows.