docs: add HANDOFF.md, commit Phase 8 WIP
HANDOFF.md is the "where things actually stand" companion to the plan document: a phase-by-phase status table, how to spin the dev environment back up (including the "restart the container after any module change or you'll hit a stale registry" gotcha that bit repeatedly this session), an index of every real Odoo 19 API-drift gotcha discovered during the build (pointing at the commit that documents each in full rather than duplicating it), and instructions for pushing this repo to a remote before handing it to a team (none is configured yet - this repo only exists locally). Also commits the two Phase 8 files that existed only as uncommitted local changes (scripts/_migration_common.py, a first pass at scripts/migrate_members.py) so they survive a git clone rather than being fragile local-only WIP. Neither is wired into anything or tested yet - migrate_members.py has no live run against Odoo, and migrate_students.py / migrate_opening_balances.py don't exist yet. Paused here at the user's request. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
metatroncubeswdev
parent
d05d80c4df
commit
8c8043a49a
150
HANDOFF.md
Normal file
150
HANDOFF.md
Normal file
@ -0,0 +1,150 @@
|
||||
# Handoff / Resume Notes
|
||||
|
||||
Read this first if you're picking this project back up — whether that's
|
||||
Claude Code in a future session or a human developer. The phase-by-phase
|
||||
build plan is in `CommunityOS_Implementation_Plan_for_Claude_Code.md`; this
|
||||
file is the "where things actually stand" companion to it.
|
||||
|
||||
## Status at a glance
|
||||
|
||||
| Phase | What it is | Status |
|
||||
|---|---|---|
|
||||
| 0 | Repo scaffold, Docker dev stack, CI + brand-leak guardrails | ✅ Done, committed |
|
||||
| 1 | `community_membership` | ✅ Done, committed |
|
||||
| 2 | `event_qr_ticketing` | ✅ Done, committed |
|
||||
| 3 | `community_school` | ✅ Done, committed |
|
||||
| 4 | `community_classifieds` + `community_benefits` | ✅ Done, committed |
|
||||
| 5 | `community_interac` (Interac e-Transfer payment provider) | ✅ Done, committed |
|
||||
| 6 | `community_theme_base` + `community_portal` | ✅ Done, committed |
|
||||
| 7 | `tncsc_deployment` (client config layer) | ✅ Done, committed |
|
||||
| 8 | TNCSC data migration scripts | 🟡 Barely started, **not committed** — see below |
|
||||
| 9 | Packaging for resale + production go-live | ⬜ Not started |
|
||||
|
||||
Every phase 0–7 commit was verified against a **live** Odoo 19 + Postgres 16
|
||||
container before being committed — not just written and assumed to work.
|
||||
Run `git log --oneline` for the full commit list; each commit message
|
||||
documents what was built and how it was verified, including several real
|
||||
Odoo 19 API changes that don't match older docs/tutorials (see "Gotchas"
|
||||
below for the index).
|
||||
|
||||
**No git remote is configured.** This repo only exists on this machine
|
||||
right now. Before handing off to a team, push it somewhere (GitHub/GitLab/
|
||||
etc.) — see "Handing off to a team" below.
|
||||
|
||||
## Resuming the dev environment
|
||||
|
||||
```bash
|
||||
cd deploy
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
- Odoo: http://localhost:8069, database `communityos_dev`, login `admin` /
|
||||
password `admin`.
|
||||
- This dev database has been used for **all** testing across every phase —
|
||||
it has real posted accounting entries in it (from Interac transaction
|
||||
confirmations, membership invoices, etc.). That's *why* Phase 7's
|
||||
`tncsc_deployment` module can't be installed on it directly (Odoo
|
||||
correctly refuses to change a company's currency once journal entries
|
||||
exist) — it was instead verified on throwaway fresh databases
|
||||
(`tncsc_freshN`, all dropped after verification). A real TNCSC production
|
||||
database should be a genuinely fresh install, not this dev one.
|
||||
- **After any module code change**, the *running* `deploy-odoo-1` container
|
||||
has a stale in-memory registry until you either upgrade the module (`-u`)
|
||||
and restart the container, or just restart it:
|
||||
```bash
|
||||
docker compose restart odoo
|
||||
```
|
||||
This bit repeatedly during development — don't skip it when testing.
|
||||
|
||||
## Phase 8 — exactly where it was left off
|
||||
|
||||
Two files exist, committed as WIP (see the `wip:` commit), inert (not
|
||||
imported by anything, not installed, no manifest references them):
|
||||
|
||||
- `scripts/_migration_common.py` — shared JSON-RPC helpers (auth, execute_kw,
|
||||
CSV normalization, a `MigrationReport` class) for the migration scripts.
|
||||
Finished, not yet tested against anything.
|
||||
- `scripts/migrate_members.py` — first pass at the member migration script
|
||||
(idempotent upsert by email, dry-run flag, tier lookup by code, report
|
||||
CSV). Written but **not tested at all** — no live run against Odoo yet.
|
||||
|
||||
Still to do for Phase 8 (see the plan's Phase 8 section):
|
||||
- `scripts/migrate_students.py` — not started.
|
||||
- `scripts/migrate_opening_balances.py` — not started.
|
||||
- Test all three against synthetic/sample CSV data on a live instance,
|
||||
the same way `scripts/migrate_classroom.py` was tested (see that file
|
||||
and Phase 3's commit for the pattern: dry-run first, then a real run,
|
||||
then a second real run to prove idempotency).
|
||||
- Real TNCSC member/student data doesn't exist yet — these scripts were
|
||||
intentionally designed against a *documented, assumed* CSV schema (see
|
||||
each script's docstring) since the real export format isn't known yet.
|
||||
Confirm the assumed column names match the real export before relying on
|
||||
them, or adjust the scripts to match.
|
||||
|
||||
To pick this back up: either continue `migrate_members.py`'s testing and
|
||||
write the other two scripts, or decide the assumed CSV schemas are wrong
|
||||
and redesign first once real export samples exist.
|
||||
|
||||
## Phase 9 — not started
|
||||
|
||||
Two sub-parts with very different risk profiles:
|
||||
- **9-A (productize)**: static description pages, READMEs, CHANGELOGs,
|
||||
release tagging for the product modules. Safe to do autonomously.
|
||||
- **9-B (go-live)**: SSL, DNS cutover, firewall rules, backups on a **real
|
||||
production server**. This needs a human in the loop with actual server
|
||||
access/credentials — don't let an agent run this unattended.
|
||||
|
||||
## Gotchas discovered this build (Odoo 19 API drift + environment quirks)
|
||||
|
||||
This Odoo 19 build (`19.0-20260817`, essentially a nightly) diverges from
|
||||
older tutorials/docs in several places that cost real debugging time. Each
|
||||
is documented in detail in the commit message where it was found — this is
|
||||
just the index so you know to search `git log` for it instead of
|
||||
re-discovering it:
|
||||
|
||||
- `res.groups.category_id` → replaced by `privilege_id` →
|
||||
`res.groups.privilege.category_id` (Session 1-A commit).
|
||||
- Settings pages use a new `<app>/<block>/<setting>` structure inherited
|
||||
from `base.res_config_settings_view_form`, not the old raw-div layout
|
||||
(Session 1-A commit).
|
||||
- `ir.cron` dropped `numbercall` entirely (Session 1-B commit).
|
||||
- `event.event` has no `state` field anymore (replaced by a
|
||||
`stage_id`/`event.stage` kanban system) (Phase 2 commit).
|
||||
- Route `type='json'` should be `type='jsonrpc'` (`'json'` still works but
|
||||
is deprecated) (Phase 2 commit).
|
||||
- Search-view `<group>` elements for "Group By" sections no longer accept a
|
||||
`string` attribute, only `name` (Phase 3 / Phase 4 commits).
|
||||
- `res.groups.users` → renamed to `user_ids` (Phase 4 commit).
|
||||
- `res.config.settings` fields backed by `config_parameter=` only support
|
||||
boolean/integer/float/char/selection/many2one/datetime — **not**
|
||||
`Binary`. Using it on a Binary field breaks *every* Settings tab, not
|
||||
just the one you're adding, since they share one model (`fix:` commit
|
||||
after Phase 6).
|
||||
- Anonymous visitors hitting an `auth='user'` + `website=True` route in
|
||||
this specific build crash with a raw 500 instead of a clean redirect to
|
||||
`/web/login` (`Request._serve_db`'s `finally: self.env = None` runs
|
||||
before the website error handler tries to build the redirect). Worked
|
||||
around by switching those routes to `auth='public'` with a manual
|
||||
`if request.env.user._is_public(): return request.redirect(...)` check
|
||||
(`fix:` commit after Phase 6 — this is the fix to reach for if you add
|
||||
*new* member-only pages later, don't reintroduce `auth='user'` on a
|
||||
`website=True` route without it).
|
||||
- Setting `res.company.country_id` on a brand-new company auto-schedules
|
||||
this build's own chart-template installer via a precommit hook, which
|
||||
races an explicit `try_loading()` call made in the same install
|
||||
transaction and silently reverts it afterwards. `ir.cron` code execution
|
||||
is also sandboxed and forbids direct attribute assignment (`STORE_ATTR`)
|
||||
— use `.write(...)`. Both fully explained in the Phase 7 commit message,
|
||||
fix pattern is in `addons/tncsc_deployment/models/res_company.py`.
|
||||
|
||||
## Handing off to a team
|
||||
|
||||
1. **Push this repo to a real remote** (GitHub/GitLab/etc.) — right now it
|
||||
only exists locally.
|
||||
2. Point them at `CommunityOS_Implementation_Plan_for_Claude_Code.md` (the
|
||||
plan) and this file (current status).
|
||||
3. They'll need Docker Desktop (or equivalent) to run `deploy/docker-compose.yml`
|
||||
locally — no other local dependencies.
|
||||
4. If they're using Claude Code to continue: just point it at this repo and
|
||||
this file. It doesn't need conversation history — everything load-bearing
|
||||
is either in a commit message or in this document.
|
||||
@ -84,7 +84,8 @@ mode reload is enabled in `deploy/odoo.conf`).
|
||||
package, and a `static/description/index.html` listing page.
|
||||
|
||||
See `CommunityOS_Implementation_Plan_for_Claude_Code.md` for the full,
|
||||
phase-by-phase build plan.
|
||||
phase-by-phase build plan, and `HANDOFF.md` for exactly where the build
|
||||
currently stands, how to resume it, and known gotchas.
|
||||
|
||||
## CI
|
||||
|
||||
|
||||
94
scripts/_migration_common.py
Normal file
94
scripts/_migration_common.py
Normal file
@ -0,0 +1,94 @@
|
||||
"""Shared JSON-RPC helpers for the standalone migration scripts in this
|
||||
directory (migrate_members.py, migrate_students.py,
|
||||
migrate_opening_balances.py, migrate_classroom.py).
|
||||
|
||||
Deliberately stdlib-only (no external dependencies) so these scripts can run
|
||||
on any machine with Python 3, without needing the project's own virtualenv
|
||||
or the Odoo MCP server - they talk to a deployed Odoo instance purely over
|
||||
JSON-RPC.
|
||||
"""
|
||||
import csv
|
||||
import json
|
||||
import re
|
||||
import urllib.request
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def jsonrpc_call(url, service, method, args):
|
||||
payload = {
|
||||
'jsonrpc': '2.0',
|
||||
'method': 'call',
|
||||
'params': {'service': service, 'method': method, 'args': args},
|
||||
}
|
||||
request = urllib.request.Request(
|
||||
f'{url}/jsonrpc',
|
||||
data=json.dumps(payload).encode('utf-8'),
|
||||
headers={'Content-Type': 'application/json'},
|
||||
)
|
||||
with urllib.request.urlopen(request) as response:
|
||||
result = json.loads(response.read())
|
||||
if 'error' in result:
|
||||
raise RuntimeError(result['error'].get('data', {}).get('message') or result['error'])
|
||||
return result['result']
|
||||
|
||||
|
||||
def authenticate(url, db, username, password):
|
||||
uid = jsonrpc_call(url, 'common', 'authenticate', [db, username, password, {}])
|
||||
if not uid:
|
||||
raise RuntimeError('Authentication failed - check --url/--db/--username/--password')
|
||||
return uid
|
||||
|
||||
|
||||
def execute_kw(url, db, uid, password, model, method, args, kwargs=None):
|
||||
full_args = [db, uid, password, model, method, args]
|
||||
if kwargs is not None:
|
||||
return jsonrpc_call(url, 'object', 'execute_kw', full_args + [kwargs])
|
||||
return jsonrpc_call(url, 'object', 'execute_kw', full_args)
|
||||
|
||||
|
||||
def normalize_email(value):
|
||||
return (value or '').strip().lower() or None
|
||||
|
||||
|
||||
def normalize_phone(value):
|
||||
value = (value or '').strip()
|
||||
if not value:
|
||||
return None
|
||||
keep_plus = value.startswith('+')
|
||||
digits = re.sub(r'\D', '', value)
|
||||
return ('+' if keep_plus else '') + digits
|
||||
|
||||
|
||||
def normalize_date(value):
|
||||
"""Parse a handful of common export date formats into ISO (YYYY-MM-DD)."""
|
||||
value = (value or '').strip()
|
||||
if not value:
|
||||
return None
|
||||
for fmt in ('%Y-%m-%d', '%m/%d/%Y', '%d/%m/%Y', '%Y/%m/%d', '%B %d, %Y', '%b %d, %Y'):
|
||||
try:
|
||||
return datetime.strptime(value, fmt).date().isoformat()
|
||||
except ValueError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
class MigrationReport:
|
||||
"""Accumulates per-row outcomes and writes them to a report CSV."""
|
||||
|
||||
def __init__(self):
|
||||
self.rows = []
|
||||
|
||||
def add(self, key, action, reason=''):
|
||||
self.rows.append({'key': key, 'action': action, 'reason': reason})
|
||||
|
||||
def write(self, path):
|
||||
with open(path, 'w', newline='', encoding='utf-8') as handle:
|
||||
writer = csv.DictWriter(handle, fieldnames=['key', 'action', 'reason'])
|
||||
writer.writeheader()
|
||||
writer.writerows(self.rows)
|
||||
|
||||
def summary(self):
|
||||
counts = {}
|
||||
for row in self.rows:
|
||||
counts[row['action']] = counts.get(row['action'], 0) + 1
|
||||
return counts
|
||||
146
scripts/migrate_members.py
Normal file
146
scripts/migrate_members.py
Normal file
@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Migrate existing TNCSC members into Community OS Membership.
|
||||
|
||||
Input CSV columns (source system export - adjust the header names below to
|
||||
match the real export once available, the column *names* are what matter,
|
||||
not their order):
|
||||
|
||||
full_name,email,phone,tier_code,membership_start,membership_expiry,is_volunteer
|
||||
|
||||
- full_name: required.
|
||||
- email: required, used as the dedupe/match key.
|
||||
- phone: optional, normalized to digits (with a leading + kept if present).
|
||||
- tier_code: required, must match an existing community.membership.tier
|
||||
code (e.g. IND, FAM, STU, SEN, LIFE for the TNCSC deployment).
|
||||
- membership_start / membership_expiry: optional, several common date
|
||||
formats are accepted and normalized to ISO (YYYY-MM-DD).
|
||||
- is_volunteer: optional, "1"/"true"/"yes" (case-insensitive) = True.
|
||||
|
||||
Behaviour:
|
||||
- Rows missing full_name/email, with an unrecognized tier_code, or that
|
||||
duplicate an email already seen earlier in the SAME file are marked
|
||||
'failed' with a reason and skipped - they never reach Odoo.
|
||||
- Existing members are matched by email (case-insensitive) and updated;
|
||||
new ones are created. Re-running the same file is safe: it always
|
||||
converges on the same end state rather than creating duplicates.
|
||||
- A member with a tier and no explicit state that isn't already active
|
||||
is activated (generates a member ID) after being created/updated.
|
||||
|
||||
Usage:
|
||||
python scripts/migrate_members.py \\
|
||||
--url https://your-odoo-host --db communityos_dev \\
|
||||
--username admin --password admin \\
|
||||
--csv data/raw/members_export.csv \\
|
||||
--report data/raw/members_migration_report.csv \\
|
||||
[--dry-run]
|
||||
"""
|
||||
import argparse
|
||||
import csv
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from _migration_common import (
|
||||
MigrationReport, authenticate, execute_kw, normalize_date, normalize_email, normalize_phone,
|
||||
)
|
||||
|
||||
|
||||
def read_rows(csv_path):
|
||||
with open(csv_path, newline='', encoding='utf-8') as handle:
|
||||
yield from csv.DictReader(handle)
|
||||
|
||||
|
||||
def is_truthy(value):
|
||||
return (value or '').strip().lower() in ('1', 'true', 'yes', 'y')
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument('--url', required=True)
|
||||
parser.add_argument('--db', required=True)
|
||||
parser.add_argument('--username', required=True)
|
||||
parser.add_argument('--password', required=True)
|
||||
parser.add_argument('--csv', required=True, type=Path)
|
||||
parser.add_argument('--report', required=True, type=Path)
|
||||
parser.add_argument('--dry-run', action='store_true')
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.csv.exists():
|
||||
print(f"CSV file not found: {args.csv}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
uid = authenticate(args.url, args.db, args.username, args.password)
|
||||
report = MigrationReport()
|
||||
|
||||
def rpc(model, method, call_args, kwargs=None):
|
||||
return execute_kw(args.url, args.db, uid, args.password, model, method, call_args, kwargs)
|
||||
|
||||
tier_by_code = {}
|
||||
for tier in rpc('community.membership.tier', 'search_read', [[]], {'fields': ['code']}):
|
||||
tier_by_code[tier['code']] = tier['id']
|
||||
|
||||
seen_emails = set()
|
||||
|
||||
for row_number, row in enumerate(read_rows(args.csv), start=2):
|
||||
name = (row.get('full_name') or '').strip()
|
||||
email = normalize_email(row.get('email'))
|
||||
key = email or f'row {row_number}'
|
||||
|
||||
if not name or not email:
|
||||
report.add(key, 'failed', 'missing full_name or email')
|
||||
continue
|
||||
if email in seen_emails:
|
||||
report.add(key, 'failed', 'duplicate email in source file')
|
||||
continue
|
||||
seen_emails.add(email)
|
||||
|
||||
tier_code = (row.get('tier_code') or '').strip()
|
||||
tier_id = tier_by_code.get(tier_code)
|
||||
if tier_code and not tier_id:
|
||||
report.add(key, 'failed', f"unknown tier_code '{tier_code}'")
|
||||
continue
|
||||
|
||||
vals = {
|
||||
'name': name,
|
||||
'email': email,
|
||||
'phone': normalize_phone(row.get('phone')),
|
||||
'is_volunteer': is_truthy(row.get('is_volunteer')),
|
||||
}
|
||||
if tier_id:
|
||||
vals['membership_tier_id'] = tier_id
|
||||
start = normalize_date(row.get('membership_start'))
|
||||
if start:
|
||||
vals['membership_start'] = start
|
||||
expiry = normalize_date(row.get('membership_expiry'))
|
||||
if expiry:
|
||||
vals['membership_expiry'] = expiry
|
||||
|
||||
existing = rpc('res.partner', 'search', [[('email', '=', email)]], {'limit': 1})
|
||||
|
||||
if args.dry_run:
|
||||
report.add(key, 'would_update' if existing else 'would_create')
|
||||
continue
|
||||
|
||||
if existing:
|
||||
rpc('res.partner', 'write', [existing, vals])
|
||||
partner_id = existing[0]
|
||||
action = 'updated'
|
||||
else:
|
||||
partner_id = rpc('res.partner', 'create', [vals])
|
||||
action = 'created'
|
||||
|
||||
if tier_id:
|
||||
partner_state = rpc('res.partner', 'read', [[partner_id], ['membership_state']])[0]['membership_state']
|
||||
if partner_state not in ('active', 'renewal_due'):
|
||||
rpc('res.partner', 'action_activate_membership', [[partner_id]])
|
||||
|
||||
report.add(key, action)
|
||||
|
||||
report.write(args.report)
|
||||
summary = report.summary()
|
||||
print(f"Done. {summary}")
|
||||
print(f"Report written to {args.report}")
|
||||
return 1 if summary.get('failed') else 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Loading…
x
Reference in New Issue
Block a user