JSON-LD on bikegear.in has incorrect/all-caps brand names (e.g. "ACERBIS Hemets",
full page titles as brand). Our BRANDS list has the correct clean names.
Changed priority: brandName (our list) > jsonLd.brand.name > fallback.
Applied to both standalone local scraper and server scraper.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After the main scrape loop finishes, collects all products that had scrapeError
and retries them sequentially up to 3 times with a 2s pause between passes.
Updates the output in-place so recovered products count as successes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Self-contained — all brands and scraper logic inlined, no project imports.
Only needs: npm install axios, then node bikegear-scraper.js.
Supports resume on Ctrl+C, saves bikegear_output.json in server-compatible format.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Standalone script for running bikegear scrape from a local machine (residential
IP bypasses Cloudflare). Saves progress after every brand so Ctrl+C resumes
cleanly. Output is written to data/sources/bikegear/01_products_aggregated.json
in the exact same format the server pipeline expects — copy the file to the
server and the pipeline uses it as a 24h cache, skipping straight to stage 2.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
