3 Commits

Author SHA1 Message Date
ead5d75309 feat(bikegear): add package.json to local scraper folder
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-06 00:33:40 +05:30
ac0c37670b feat(bikegear): add fully standalone single-file scraper for sharing
Self-contained — all brands and scraper logic inlined, no project imports.
Only needs: npm install axios, then node bikegear-scraper.js.
Supports resume on Ctrl+C, saves bikegear_output.json in server-compatible format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-05 23:58:58 +05:30
f98febc5c5 feat(bikegear): add local scraper with resume support for residential IP scraping
Standalone script for running bikegear scrape from a local machine (residential
IP bypasses Cloudflare). Saves progress after every brand so Ctrl+C resumes
cleanly. Output is written to data/sources/bikegear/01_products_aggregated.json
in the exact same format the server pipeline expects — copy the file to the
server and the pipeline uses it as a 24h cache, skipping straight to stage 2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-05 23:50:13 +05:30