103 lines
7.6 KiB
Markdown
103 lines
7.6 KiB
Markdown
# CrawlerX — Product Requirements & Implementation Plan
|
||
|
||
**Date:** Oct 17, 2026
|
||
**Company:** Metatroncube Solutions
|
||
**Positioning:** *"Semrush tells you what's wrong. CrawlerX fixes it — 10 sites for about a fifth of one Semrush seat."*
|
||
|
||
---
|
||
|
||
## Executive Summary & Financial Viability
|
||
|
||
- **Core Proposition:** A "10 sites for about $30" SEO tool is financially viable. At **₹2,499/month (≈$28) for 10 sites**, rented data plus our own crawler costs about $6 per account at typical usage and about $10 at full usage, yielding a **60–76% gross margin**.
|
||
- **Break-Even Target:** Roughly **380 paying accounts** with a ₹6.4 lakh/month team (reachable in 7–13 months post-launch at 30–60 paying accounts/month).
|
||
- **Lean Option (Recommended):** Running lean on the existing Metatroncube team (~₹3 lakh/month) cuts break-even to **under 200 accounts** with ₹15–20 lakh peak runway.
|
||
|
||
### Four Non-Negotiable Conditions
|
||
1. **Don't fight Semrush on raw data size:** Rent keyword, SERP, and backlink data from wholesale APIs (DataForSEO) and win on workflow: *Audit → AI Fix → Push to Live Site → Prove Result*.
|
||
2. **Usage caps are the business model:** Hard/soft caps with cached lookups to guarantee healthy margins (Pro margin 76% typical, 60% full).
|
||
3. **Agency & Freelancer First:** Target shops managing 5–40 client sites who feel per-site Semrush costs most acutely and churn far less than solo SMBs.
|
||
4. **Distribution & Value Proof:** Deliver visible ranking and audit improvements within the first 7 days. Dogfood on Metatroncube's own clients first.
|
||
|
||
---
|
||
|
||
## Market & Competitor Analysis
|
||
|
||
| Tool | Entry Price (Monthly) | Coverage | Weakness We Exploit |
|
||
| :--- | :--- | :--- | :--- |
|
||
| **Semrush** | $139.95 Pro; $249.95 Guru | 1 user, limited projects; white-label on Business only ($500/mo) | Prohibitive for Indian agencies/SMBs (₹11,750+/mo); extra seats $45–100 |
|
||
| **Semrush One** | $199 Starter | SEO + AI-search tracking | AI visibility treated as a luxury upsell |
|
||
| **Ubersuggest** | $29 Individual; $49 Business | 150 searches/day, 125 keywords/domain | Shallow data; basic reports; no fix execution engine |
|
||
| **SE Ranking** | $59 Essential; $109 Pro | 1 seat, 5 projects, 500 keywords | Western USD pricing; content & local add-ons cost extra |
|
||
| **Synup** | $49 (1 loc); $299 (10 loc) | Local listings, reviews, geo-grid | Multi-location local brand focus; little technical SEO |
|
||
| **CrawlerX** | **₹2,499 (≈$28) for 10 sites** | **Audit + AI fixes, rank tracking, keywords, local, AI visibility, white-label** | India-priced, complete audit-to-fix push loop, agency-first |
|
||
|
||
---
|
||
|
||
## Core Differentiators
|
||
|
||
1. **Fix, Not Just Find:** Generated code & copy (title, meta, schema JSON-LD, image alt, internal links, FAQ blocks) pushed directly to WordPress (plugin) or Shopify (app) after one-click approval, then automatically re-crawled to verify.
|
||
2. **Built-in AI Search Visibility:** Included by default (not a $199 upsell). Tracks mentions and citations across ChatGPT, Google Gemini, Perplexity, and Google AI Overviews, with actionable fixes.
|
||
3. **Local SEO Built-In:** Geo-grid ranking maps, Google Business Profile (GBP) health audit, and AI review response drafts.
|
||
4. **Built for Indian Agencies:** White-label reporting from the 10-site plan, client portal, WhatsApp report delivery, INR + UPI Autopay + GST compliant invoices, and Hindi/Tamil/regional keyword data.
|
||
5. **"Next 5 Actions" Weekly Briefing:** Plain-language, high-impact weekly priorities instead of overwhelming 400-row issue tables.
|
||
|
||
---
|
||
|
||
## 7 Core Product Modules & Phased Roadmap
|
||
|
||
```
|
||
MVP (Months 1–4) ────────► v1 (Months 5–7) ────────► v2 (Months 8–12)
|
||
• JS-rendering crawler • WP & Shopify Push • Bulk fixes & approval queue
|
||
• 120+ audit checks • AI Search Visibility • Competitor link gap
|
||
• AI fix generator (copy) • Geo-grid Local Map • Public API & MCP Server
|
||
• India city-level ranks • Regional keywords • Scheduled WhatsApp reports
|
||
• Razorpay UPI Autopay • Client portal & PDF • Own cached keyword index
|
||
```
|
||
|
||
### Module Breakdown
|
||
|
||
| Module | MVP (Months 0–4) | v1 (Months 5–7) | v2 (Months 8–12) |
|
||
| :--- | :--- | :--- | :--- |
|
||
| **1. Site Audit (Own Crawler)** | JS-rendering crawl, 120+ checks, health score, weekly re-crawl | Log-file upload, crawl diff vs last crawl | Scheduled crawls by page type, regression alerts |
|
||
| **2. AI Fix Engine** | Generated fixes (titles, metas, alt text, schema JSON-LD); copy to clipboard | 1-Click push to WordPress (plugin) & Shopify (app); re-crawl verification | Bulk fixes, agency approval queue, internal link engine |
|
||
| **3. Rank Tracking** | Google desktop + mobile, India city-level, top-20 depth, 6-day refresh | Competitor rank share, SERP features | Bing, YouTube |
|
||
| **4. Keyword Research** | Volume, difficulty, intent, CPC for India + 10 countries (cached) | Regional languages (Hindi, Tamil, Telugu, Marathi, Bengali), clusters | Proprietary India keyword database built from cached lookups |
|
||
| **5. Local SEO** | GBP connection, profile health check | Geo-grid rank map, review monitoring + AI reply drafts | Citation (NAP) audit for Indian directories (Justdial, Sulekha, IndiaMART) |
|
||
| **6. AI Search Visibility** | — | 10–20 tracked prompts across ChatGPT, Gemini, Perplexity, AI Overviews; mention share | llms.txt & AI crawler access checks; "why AI skipped you" fixes |
|
||
| **7. Agency Layer** | White-label PDF report, client sharing link | Client portal, WhatsApp/email scheduled reports, team seats | Public API, MCP Server for Claude/ChatGPT/Hermes |
|
||
|
||
---
|
||
|
||
## Pricing & Unit Economics
|
||
|
||
*All prices in INR exclude 18% GST. USD converted at ~₹88.*
|
||
|
||
| Plan | Price / Month | Sites | Pages / Audit | Tracked Keywords | Keyword Lookups / Mo | AI Visibility Checks / Mo | Data Cost (Full Use) | Gross Margin (Typical / Full) |
|
||
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
||
| **Free** | ₹0 | 1 | 100 | 10 | 30 | — | ≈$0.30 | Lead Magnet |
|
||
| **Starter** | ₹999 (≈$11) | 3 | 1,000 | 100 | 150 | 40 | $3.11 | 82% / 69% |
|
||
| **Pro** | ₹2,499 (≈$28) | 10 | 3,000 | 300 | 450 | 120 | $10.30 | 76% / 60% |
|
||
| **Agency** | ₹7,499 (≈$85) | 30 | 5,000 | 1,000 | 1,200 | 400 | $33.57 | 72% / 57% |
|
||
|
||
- **International Pricing:** $15 / $29 / $79 via Stripe with annual billing discount (2 months free).
|
||
- **Add-on Packs:** ₹299 for +100 tracked keywords, ₹499 for +200 AI checks.
|
||
- **Payment Rails:** Razorpay for domestic INR payments (UPI Autopay, NetBanking, Cards, GST invoicing) + Stripe for international USD.
|
||
|
||
---
|
||
|
||
## Technical Stack & Architecture
|
||
|
||
- **Frontend:** TanStack Start / Router / Query / Form + React + Tailwind CSS + DaisyUI.
|
||
- **Backend & Compute:** Cloudflare Workers + Node/FastAPI worker pool for heavy crawls.
|
||
- **Database & Storage:** Cloudflare D1 / PostgreSQL (dual-dialect Drizzle ORM) + Cloudflare R2 for crawl snapshots and PDF storage.
|
||
- **Data Integrations:** DataForSEO wholesale API (metered through unified quota service with redis cache), Google APIs (Search Console, Business Profile, PageSpeed).
|
||
- **Cost Controls:** Central metering layer checking plan quota before external provider calls, with multi-tenant caching for SERP/keyword queries.
|
||
|
||
---
|
||
|
||
## Decision Gates & Milestones
|
||
|
||
- **Gate 1 (End of Month 4 - MVP):** 3 Metatroncube retainer clients dogfooded with measurable gains (audit score +20, 10+ ranking improvements).
|
||
- **Gate 2 (End of Month 6):** 15 of 20 Indian agency design partners active weekly; at least 10 converting to paid list price.
|
||
- **Gate 3 (End of Month 12):** 200+ paying accounts, monthly logo churn < 6%, data cost < 25% of ARPU.
|