# Dine360 SaaS Multi-Tenant Platform: Production Scaling & Deployment Guide

This guide describes how to configure, deploy, secure, and scale the Dine360 SaaS platform to support 25 to 500+ concurrent restaurants.

---

## 1. High Availability Architecture

To ensure 99.9% uptime, the infrastructure must be divided into separate tiers with no single point of failure (SPOF).

```mermaid
graph LR
    User[Clients / POS] -->|HTTPS| DNS[Route53 / Cloudflare DNS]
    DNS -->|Anycast| ALB[Load Balancer Pool: Nginx / AWS ALB]
    ALB -->|Proxy Pass| WebPool[Odoo App instances cluster]
    WebPool -->|Session Cache| Redis[Redis Replication Group]
    WebPool -->|DB Pool| Bouncer[PgBouncer Poolers]
    Bouncer -->|Transactions| DBPrimary[(PostgreSQL Primary)]
    DBPrimary -->|Streaming replication| DBReplica[(PostgreSQL Hot Standby)]
    DBPrimary -->|Archived Logs| S3[(AWS S3 Backup Bucket)]
```

---

## 2. Server Sizing Recommendations

### 25 to 100 Restaurants (Starter Tier)
- **Odoo Servers**: 1x App Server (8 vCPU, 16GB RAM) running Odoo with `--workers=17`.
- **Database Server**: 1x PostgreSQL Server (8 vCPU, 32GB RAM).
- **Cache/Session**: Local Redis instance.

### 100 to 500+ Restaurants (Enterprise / High-Scale Tier)
- **App Instance Pool**: 3x App Servers (each 8 vCPU, 16GB RAM) running behind a Load Balancer (ALB).
- **PostgreSQL Database Cluster**:
  - 1x Primary Write Node (32 vCPU, 128GB RAM, NVMe storage).
  - 1x Replica Read-Only Node (16 vCPU, 64GB RAM) for heavy reports, API queries, and read scaling.
- **PgBouncer**: Dedicated PgBouncer container on database nodes configured in transaction pooling mode.
- **Cache/Session**: Managed AWS ElastiCache for Redis (Replicated/Sharded cluster).

---

## 3. Database Connection Pooling (PgBouncer)

With 500+ databases, Odoo's default connection model opens up to 500 * (workers + 2) connections, easily overwhelming PostgreSQL connection limits and exhausting system file descriptors.
- **Fix**: Use PgBouncer in **transaction pooling** mode:
  ```ini
  pool_mode = transaction
  max_client_conn = 10000
  default_pool_size = 50
  ```
- **Transaction Pooling Note**: In transaction mode, cursor-based operations (like Odoo's temporary tables or session locks) can sometimes fail if not handled. Odoo is compatible with transaction pooling from version 12 onwards, but long-lived locks should be avoided in custom modules.

---

## 4. Let's Encrypt Wildcard SSL Automation

For auto-onboarding subdomains (e.g. `*.dine360.com`), a Wildcard SSL certificate is required.

### Setup using Certbot (DNS-01 Challenge)
Since HTTP validation cannot verify wildcard domains, you must use DNS-01 validation.

1. **Install Certbot with DNS plugin** (e.g. Route53 plugin for AWS):
   ```bash
   sudo apt-get install certbot python3-certbot-dns-route53
   ```

2. **Acquire Wildcard Certificate**:
   ```bash
   certbot certonly --dns-route53 -d dine360.com -d *.dine360.com
   ```

3. **Cron for Automatic Renewal**:
   Add this to `/etc/crontab` to check renewals daily and reload Nginx:
   ```cron
   0 0 * * * root certbot renew --post-hook "systemctl reload nginx"
   ```

---

## 5. Monitoring Setup

To maintain system observability, implement the following stack:
1. **Node Exporter & Prometheus**: Collect CPU, Memory, Disk IOPS, and Network metrics of all host servers.
2. **postgres_exporter**: Track PostgreSQL active backends, transaction delays, lock waits, and cache hit rates.
3. **Grafana Dashboard**:
   - Create alerts for DB connection limits (>80%).
   - Create alerts for disk usage (>85%).
   - Monitor average query response time.
4. **Sentry Integration**: Configure Odoo to send trace logs and exceptions directly to Sentry for debugging tenant-specific issues without accessing logs directly.