diff --git a/HANDOFF.md b/HANDOFF.md new file mode 100644 index 0000000..4cc8a56 --- /dev/null +++ b/HANDOFF.md @@ -0,0 +1,150 @@ +# Handoff / Resume Notes + +Read this first if you're picking this project back up — whether that's +Claude Code in a future session or a human developer. The phase-by-phase +build plan is in `CommunityOS_Implementation_Plan_for_Claude_Code.md`; this +file is the "where things actually stand" companion to it. + +## Status at a glance + +| Phase | What it is | Status | +|---|---|---| +| 0 | Repo scaffold, Docker dev stack, CI + brand-leak guardrails | ✅ Done, committed | +| 1 | `community_membership` | ✅ Done, committed | +| 2 | `event_qr_ticketing` | ✅ Done, committed | +| 3 | `community_school` | ✅ Done, committed | +| 4 | `community_classifieds` + `community_benefits` | ✅ Done, committed | +| 5 | `community_interac` (Interac e-Transfer payment provider) | ✅ Done, committed | +| 6 | `community_theme_base` + `community_portal` | ✅ Done, committed | +| 7 | `tncsc_deployment` (client config layer) | ✅ Done, committed | +| 8 | TNCSC data migration scripts | 🟡 Barely started, **not committed** — see below | +| 9 | Packaging for resale + production go-live | ⬜ Not started | + +Every phase 0–7 commit was verified against a **live** Odoo 19 + Postgres 16 +container before being committed — not just written and assumed to work. +Run `git log --oneline` for the full commit list; each commit message +documents what was built and how it was verified, including several real +Odoo 19 API changes that don't match older docs/tutorials (see "Gotchas" +below for the index). + +**No git remote is configured.** This repo only exists on this machine +right now. Before handing off to a team, push it somewhere (GitHub/GitLab/ +etc.) — see "Handing off to a team" below. + +## Resuming the dev environment + +```bash +cd deploy +docker compose up -d +``` + +- Odoo: http://localhost:8069, database `communityos_dev`, login `admin` / + password `admin`. +- This dev database has been used for **all** testing across every phase — + it has real posted accounting entries in it (from Interac transaction + confirmations, membership invoices, etc.). That's *why* Phase 7's + `tncsc_deployment` module can't be installed on it directly (Odoo + correctly refuses to change a company's currency once journal entries + exist) — it was instead verified on throwaway fresh databases + (`tncsc_freshN`, all dropped after verification). A real TNCSC production + database should be a genuinely fresh install, not this dev one. +- **After any module code change**, the *running* `deploy-odoo-1` container + has a stale in-memory registry until you either upgrade the module (`-u`) + and restart the container, or just restart it: + ```bash + docker compose restart odoo + ``` + This bit repeatedly during development — don't skip it when testing. + +## Phase 8 — exactly where it was left off + +Two files exist, committed as WIP (see the `wip:` commit), inert (not +imported by anything, not installed, no manifest references them): + +- `scripts/_migration_common.py` — shared JSON-RPC helpers (auth, execute_kw, + CSV normalization, a `MigrationReport` class) for the migration scripts. + Finished, not yet tested against anything. +- `scripts/migrate_members.py` — first pass at the member migration script + (idempotent upsert by email, dry-run flag, tier lookup by code, report + CSV). Written but **not tested at all** — no live run against Odoo yet. + +Still to do for Phase 8 (see the plan's Phase 8 section): +- `scripts/migrate_students.py` — not started. +- `scripts/migrate_opening_balances.py` — not started. +- Test all three against synthetic/sample CSV data on a live instance, + the same way `scripts/migrate_classroom.py` was tested (see that file + and Phase 3's commit for the pattern: dry-run first, then a real run, + then a second real run to prove idempotency). +- Real TNCSC member/student data doesn't exist yet — these scripts were + intentionally designed against a *documented, assumed* CSV schema (see + each script's docstring) since the real export format isn't known yet. + Confirm the assumed column names match the real export before relying on + them, or adjust the scripts to match. + +To pick this back up: either continue `migrate_members.py`'s testing and +write the other two scripts, or decide the assumed CSV schemas are wrong +and redesign first once real export samples exist. + +## Phase 9 — not started + +Two sub-parts with very different risk profiles: +- **9-A (productize)**: static description pages, READMEs, CHANGELOGs, + release tagging for the product modules. Safe to do autonomously. +- **9-B (go-live)**: SSL, DNS cutover, firewall rules, backups on a **real + production server**. This needs a human in the loop with actual server + access/credentials — don't let an agent run this unattended. + +## Gotchas discovered this build (Odoo 19 API drift + environment quirks) + +This Odoo 19 build (`19.0-20260817`, essentially a nightly) diverges from +older tutorials/docs in several places that cost real debugging time. Each +is documented in detail in the commit message where it was found — this is +just the index so you know to search `git log` for it instead of +re-discovering it: + +- `res.groups.category_id` → replaced by `privilege_id` → + `res.groups.privilege.category_id` (Session 1-A commit). +- Settings pages use a new `//` structure inherited + from `base.res_config_settings_view_form`, not the old raw-div layout + (Session 1-A commit). +- `ir.cron` dropped `numbercall` entirely (Session 1-B commit). +- `event.event` has no `state` field anymore (replaced by a + `stage_id`/`event.stage` kanban system) (Phase 2 commit). +- Route `type='json'` should be `type='jsonrpc'` (`'json'` still works but + is deprecated) (Phase 2 commit). +- Search-view `` elements for "Group By" sections no longer accept a + `string` attribute, only `name` (Phase 3 / Phase 4 commits). +- `res.groups.users` → renamed to `user_ids` (Phase 4 commit). +- `res.config.settings` fields backed by `config_parameter=` only support + boolean/integer/float/char/selection/many2one/datetime — **not** + `Binary`. Using it on a Binary field breaks *every* Settings tab, not + just the one you're adding, since they share one model (`fix:` commit + after Phase 6). +- Anonymous visitors hitting an `auth='user'` + `website=True` route in + this specific build crash with a raw 500 instead of a clean redirect to + `/web/login` (`Request._serve_db`'s `finally: self.env = None` runs + before the website error handler tries to build the redirect). Worked + around by switching those routes to `auth='public'` with a manual + `if request.env.user._is_public(): return request.redirect(...)` check + (`fix:` commit after Phase 6 — this is the fix to reach for if you add + *new* member-only pages later, don't reintroduce `auth='user'` on a + `website=True` route without it). +- Setting `res.company.country_id` on a brand-new company auto-schedules + this build's own chart-template installer via a precommit hook, which + races an explicit `try_loading()` call made in the same install + transaction and silently reverts it afterwards. `ir.cron` code execution + is also sandboxed and forbids direct attribute assignment (`STORE_ATTR`) + — use `.write(...)`. Both fully explained in the Phase 7 commit message, + fix pattern is in `addons/tncsc_deployment/models/res_company.py`. + +## Handing off to a team + +1. **Push this repo to a real remote** (GitHub/GitLab/etc.) — right now it + only exists locally. +2. Point them at `CommunityOS_Implementation_Plan_for_Claude_Code.md` (the + plan) and this file (current status). +3. They'll need Docker Desktop (or equivalent) to run `deploy/docker-compose.yml` + locally — no other local dependencies. +4. If they're using Claude Code to continue: just point it at this repo and + this file. It doesn't need conversation history — everything load-bearing + is either in a commit message or in this document. diff --git a/README.md b/README.md index a8a7604..75b803f 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,8 @@ mode reload is enabled in `deploy/odoo.conf`). package, and a `static/description/index.html` listing page. See `CommunityOS_Implementation_Plan_for_Claude_Code.md` for the full, -phase-by-phase build plan. +phase-by-phase build plan, and `HANDOFF.md` for exactly where the build +currently stands, how to resume it, and known gotchas. ## CI diff --git a/scripts/_migration_common.py b/scripts/_migration_common.py new file mode 100644 index 0000000..722e109 --- /dev/null +++ b/scripts/_migration_common.py @@ -0,0 +1,94 @@ +"""Shared JSON-RPC helpers for the standalone migration scripts in this +directory (migrate_members.py, migrate_students.py, +migrate_opening_balances.py, migrate_classroom.py). + +Deliberately stdlib-only (no external dependencies) so these scripts can run +on any machine with Python 3, without needing the project's own virtualenv +or the Odoo MCP server - they talk to a deployed Odoo instance purely over +JSON-RPC. +""" +import csv +import json +import re +import urllib.request +from datetime import datetime + + +def jsonrpc_call(url, service, method, args): + payload = { + 'jsonrpc': '2.0', + 'method': 'call', + 'params': {'service': service, 'method': method, 'args': args}, + } + request = urllib.request.Request( + f'{url}/jsonrpc', + data=json.dumps(payload).encode('utf-8'), + headers={'Content-Type': 'application/json'}, + ) + with urllib.request.urlopen(request) as response: + result = json.loads(response.read()) + if 'error' in result: + raise RuntimeError(result['error'].get('data', {}).get('message') or result['error']) + return result['result'] + + +def authenticate(url, db, username, password): + uid = jsonrpc_call(url, 'common', 'authenticate', [db, username, password, {}]) + if not uid: + raise RuntimeError('Authentication failed - check --url/--db/--username/--password') + return uid + + +def execute_kw(url, db, uid, password, model, method, args, kwargs=None): + full_args = [db, uid, password, model, method, args] + if kwargs is not None: + return jsonrpc_call(url, 'object', 'execute_kw', full_args + [kwargs]) + return jsonrpc_call(url, 'object', 'execute_kw', full_args) + + +def normalize_email(value): + return (value or '').strip().lower() or None + + +def normalize_phone(value): + value = (value or '').strip() + if not value: + return None + keep_plus = value.startswith('+') + digits = re.sub(r'\D', '', value) + return ('+' if keep_plus else '') + digits + + +def normalize_date(value): + """Parse a handful of common export date formats into ISO (YYYY-MM-DD).""" + value = (value or '').strip() + if not value: + return None + for fmt in ('%Y-%m-%d', '%m/%d/%Y', '%d/%m/%Y', '%Y/%m/%d', '%B %d, %Y', '%b %d, %Y'): + try: + return datetime.strptime(value, fmt).date().isoformat() + except ValueError: + continue + return None + + +class MigrationReport: + """Accumulates per-row outcomes and writes them to a report CSV.""" + + def __init__(self): + self.rows = [] + + def add(self, key, action, reason=''): + self.rows.append({'key': key, 'action': action, 'reason': reason}) + + def write(self, path): + with open(path, 'w', newline='', encoding='utf-8') as handle: + writer = csv.DictWriter(handle, fieldnames=['key', 'action', 'reason']) + writer.writeheader() + writer.writerows(self.rows) + + def summary(self): + counts = {} + for row in self.rows: + counts[row['action']] = counts.get(row['action'], 0) + 1 + return counts diff --git a/scripts/migrate_members.py b/scripts/migrate_members.py new file mode 100644 index 0000000..3a10094 --- /dev/null +++ b/scripts/migrate_members.py @@ -0,0 +1,146 @@ +#!/usr/bin/env python3 +"""Migrate existing TNCSC members into Community OS Membership. + +Input CSV columns (source system export - adjust the header names below to +match the real export once available, the column *names* are what matter, +not their order): + + full_name,email,phone,tier_code,membership_start,membership_expiry,is_volunteer + + - full_name: required. + - email: required, used as the dedupe/match key. + - phone: optional, normalized to digits (with a leading + kept if present). + - tier_code: required, must match an existing community.membership.tier + code (e.g. IND, FAM, STU, SEN, LIFE for the TNCSC deployment). + - membership_start / membership_expiry: optional, several common date + formats are accepted and normalized to ISO (YYYY-MM-DD). + - is_volunteer: optional, "1"/"true"/"yes" (case-insensitive) = True. + +Behaviour: + - Rows missing full_name/email, with an unrecognized tier_code, or that + duplicate an email already seen earlier in the SAME file are marked + 'failed' with a reason and skipped - they never reach Odoo. + - Existing members are matched by email (case-insensitive) and updated; + new ones are created. Re-running the same file is safe: it always + converges on the same end state rather than creating duplicates. + - A member with a tier and no explicit state that isn't already active + is activated (generates a member ID) after being created/updated. + +Usage: + python scripts/migrate_members.py \\ + --url https://your-odoo-host --db communityos_dev \\ + --username admin --password admin \\ + --csv data/raw/members_export.csv \\ + --report data/raw/members_migration_report.csv \\ + [--dry-run] +""" +import argparse +import csv +import sys +from pathlib import Path + +from _migration_common import ( + MigrationReport, authenticate, execute_kw, normalize_date, normalize_email, normalize_phone, +) + + +def read_rows(csv_path): + with open(csv_path, newline='', encoding='utf-8') as handle: + yield from csv.DictReader(handle) + + +def is_truthy(value): + return (value or '').strip().lower() in ('1', 'true', 'yes', 'y') + + +def main(): + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument('--url', required=True) + parser.add_argument('--db', required=True) + parser.add_argument('--username', required=True) + parser.add_argument('--password', required=True) + parser.add_argument('--csv', required=True, type=Path) + parser.add_argument('--report', required=True, type=Path) + parser.add_argument('--dry-run', action='store_true') + args = parser.parse_args() + + if not args.csv.exists(): + print(f"CSV file not found: {args.csv}", file=sys.stderr) + return 1 + + uid = authenticate(args.url, args.db, args.username, args.password) + report = MigrationReport() + + def rpc(model, method, call_args, kwargs=None): + return execute_kw(args.url, args.db, uid, args.password, model, method, call_args, kwargs) + + tier_by_code = {} + for tier in rpc('community.membership.tier', 'search_read', [[]], {'fields': ['code']}): + tier_by_code[tier['code']] = tier['id'] + + seen_emails = set() + + for row_number, row in enumerate(read_rows(args.csv), start=2): + name = (row.get('full_name') or '').strip() + email = normalize_email(row.get('email')) + key = email or f'row {row_number}' + + if not name or not email: + report.add(key, 'failed', 'missing full_name or email') + continue + if email in seen_emails: + report.add(key, 'failed', 'duplicate email in source file') + continue + seen_emails.add(email) + + tier_code = (row.get('tier_code') or '').strip() + tier_id = tier_by_code.get(tier_code) + if tier_code and not tier_id: + report.add(key, 'failed', f"unknown tier_code '{tier_code}'") + continue + + vals = { + 'name': name, + 'email': email, + 'phone': normalize_phone(row.get('phone')), + 'is_volunteer': is_truthy(row.get('is_volunteer')), + } + if tier_id: + vals['membership_tier_id'] = tier_id + start = normalize_date(row.get('membership_start')) + if start: + vals['membership_start'] = start + expiry = normalize_date(row.get('membership_expiry')) + if expiry: + vals['membership_expiry'] = expiry + + existing = rpc('res.partner', 'search', [[('email', '=', email)]], {'limit': 1}) + + if args.dry_run: + report.add(key, 'would_update' if existing else 'would_create') + continue + + if existing: + rpc('res.partner', 'write', [existing, vals]) + partner_id = existing[0] + action = 'updated' + else: + partner_id = rpc('res.partner', 'create', [vals]) + action = 'created' + + if tier_id: + partner_state = rpc('res.partner', 'read', [[partner_id], ['membership_state']])[0]['membership_state'] + if partner_state not in ('active', 'renewal_due'): + rpc('res.partner', 'action_activate_membership', [[partner_id]]) + + report.add(key, action) + + report.write(args.report) + summary = report.summary() + print(f"Done. {summary}") + print(f"Report written to {args.report}") + return 1 if summary.get('failed') else 0 + + +if __name__ == '__main__': + sys.exit(main())