MOHAN f98febc5c5 feat(bikegear): add local scraper with resume support for residential IP scraping ...

Standalone script for running bikegear scrape from a local machine (residential
IP bypasses Cloudflare). Saves progress after every brand so Ctrl+C resumes
cleanly. Output is written to data/sources/bikegear/01_products_aggregated.json
in the exact same format the server pipeline expects — copy the file to the
server and the pipeline uses it as a 24h cache, skipping straight to stage 2.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-08-05 23:50:13 +05:30

bikegear-local-scraper

feat(bikegear): add local scraper with resume support for residential IP scraping

2026-08-05 23:50:13 +05:30

routes

Add job cancellation — backend pipeline + cancel API route

2026-06-04 16:57:35 +05:30

scripts

fix(bikegear): use warmed referrer navigation for pagination

2026-08-02 16:19:01 +05:30

src

feat(bikegear): add local scraper with resume support for residential IP scraping

2026-08-05 23:50:13 +05:30

.gitignore

feat(bikegear): add local scraper with resume support for residential IP scraping

2026-08-05 23:50:13 +05:30

auth.js

first commit

2026-04-13 17:31:26 +05:30

fulfillmentService.js

first commit

2026-04-13 17:31:26 +05:30

logger.js

first commit

2026-04-13 17:31:26 +05:30

package-lock.json

Update package-lock.json

2026-06-25 18:57:28 +05:30

package.json

chore(bikegear): add scrape-only cache command

2026-07-31 01:19:46 +05:30

server.js

Fix typo in health check service name

2026-04-14 14:23:00 +05:30

tokenStore.js

first commit

2026-04-13 17:31:26 +05:30

Description

No description provided

543 KiB

Languages

JavaScript 100%