Commit Graph

  • b6115cc637 fix(bikegear): use warmed referrer navigation for pagination main MOHAN 2026-08-02 16:19:01 +05:30
  • ed877e3a78 fix(bikegear): click through pagination requests MOHAN 2026-08-02 16:15:40 +05:30
  • 500537f8ab fix(bikegear): use real clicks for product detail navigation MOHAN 2026-08-02 16:10:05 +05:30
  • d89db25aee fix(bikegear): click through listing pages for product details MOHAN 2026-08-02 16:06:29 +05:30
  • b460853806 fix(bikegear): preserve listing referrer for paged and detail requests MOHAN 2026-08-02 15:59:15 +05:30
  • 57b174bb3c fix(bikegear): support sticky Webshare residential sessions MOHAN 2026-08-01 19:00:07 +05:30
  • aef5000539 fix(bikegear): serialize browser fetches and slow 429 retries MOHAN 2026-08-01 18:52:58 +05:30
  • d4b12fdd69 fix(bikegear): pass proxy test when product listing is readable MOHAN 2026-08-01 18:43:26 +05:30
  • 91356cd6ac fix(bikegear): avoid false Cloudflare challenge in proxy test MOHAN 2026-08-01 18:39:24 +05:30
  • 78bc7c5140 chore(bikegear): add scrape-only cache command MOHAN 2026-07-31 01:19:46 +05:30
  • 6891ebf4bf chore(bikegear): add Playwright proxy diagnostic MOHAN 2026-07-31 01:13:19 +05:30
  • 2b8cf2c6fc fix(bikegear): support Webshare proxy config and cache-only mode MOHAN 2026-07-31 01:06:37 +05:30
  • 19fd7f16ea fix(bikegear): add proxy support to Playwright browser context MOHAN 2026-07-29 07:34:32 +05:30
  • 76bd69a92f fix(bikegear): switch to persistent browser profile + stealth script to beat Cloudflare detection MOHAN 2026-07-29 07:25:35 +05:30
  • e4b3292533 fix(bikegear): always delay between brands and tighten anti-ban settings MOHAN 2026-07-24 17:02:57 +05:30
  • 46d018c56e fix(bikegear): recycle browser context every 20 brands to prevent memory OOM restart MOHAN 2026-07-24 16:27:00 +05:30
  • a8adf00732 fix(bikegear): reduce browser concurrency to 1 and add 1.5s delay between requests MOHAN 2026-07-22 18:13:29 +05:30
  • 0fbe75dd3d fix(bikegear): use Playwright browser mode to bypass Cloudflare on VPS IPs MOHAN 2026-07-22 18:05:24 +05:30
  • 846e467b85 fix(bikegear): route scraper through residential proxy to bypass Cloudflare 403 on server IPs MOHAN 2026-07-14 01:17:29 +05:30
  • bf207e48a1 fix(bikegear): fix 2 scraper issues found during test run MOHAN 2026-07-13 18:18:35 +05:30
  • 0c66b4b7ee feat(bikegear): expand brand list to all 130 brands from bikegear.in, pipeline test passed 100% MOHAN 2026-07-13 18:03:39 +05:30
  • c2e037310d feat: add bikegear.in import source (Alpinestars as first brand) MOHAN 2026-07-13 17:27:06 +05:30
  • 4b5b7cab9f feat: expose brands list per source via /pipeline/sources API MOHAN 2026-07-07 13:58:05 +05:30
  • 07fff6ac5a feat(motousher): expand brand list from 12 to 44 brands (2026-07-07) MOHAN 2026-07-07 13:47:27 +05:30
  • 9c70fcb86c fix: don't cache or process all-failed Yuasa scrape results MOHAN 2026-06-27 19:08:20 +05:30
  • 54b2ad381c fix: add Sec-Fetch browser headers to Yuasa scraper to bypass 403 MOHAN 2026-06-27 19:00:55 +05:30
  • 95e100c9b4 feat: add Yuasa Batteries as new import pipeline source MOHAN 2026-06-27 18:57:09 +05:30
  • ccc1179fa2 Fix retrorides 409 on production — add full Sec-Fetch browser headers MOHAN 2026-06-25 19:46:22 +05:30
  • 6938c113ae Fix retrorides 409 errors — treat as rate-limit, drop concurrency to 1 MOHAN 2026-06-25 19:36:59 +05:30
  • f48bc39751 Update package-lock.json MOHAN 2026-06-25 18:57:28 +05:30
  • 4b2288187c Add Retro Rides source — sitemap scraper + converter + pipeline integration MOHAN 2026-06-25 18:37:47 +05:30
  • 08f21d9bc9 Add job cancellation — backend pipeline + cancel API route MOHAN 2026-06-04 16:57:35 +05:30
  • 4e536f08b3 Add 24h cache to motousher and dirtstreet fetchWebsiteData MOHAN 2026-06-04 16:44:38 +05:30
  • 6eac0b92ed Fix converter reading nested scraped data and brand priority MOHAN 2026-06-04 16:39:54 +05:30
  • c0132ab0aa Add motousher and dirtstreet as active import pipeline sources MOHAN 2026-06-04 12:25:31 +05:30
  • b8d9478afa Add test_source scrapers for motousher.com and dirtstreet.in MOHAN 2026-06-04 12:17:23 +05:30
  • 1d254a9009 Install Playwright browser for backend MOHAN 2026-05-15 00:25:40 +05:30
  • 68949f124e Add multi-source import pipeline MOHAN 2026-05-14 23:57:27 +05:30
  • bef07eff10 Refactor code structure for improved readability and maintainability MOHAN 2026-05-14 23:56:13 +05:30
  • 2320d1d5c3 Fix typo in health check service name MOHAN 2026-04-14 14:23:00 +05:30
  • 33ad269821 Fix health check service name in response MOHAN 2026-04-14 14:22:25 +05:30
  • 9480832478 Add concurrency handling and logging enhancements to KYT pipeline MOHAN 2026-04-14 13:26:56 +05:30
  • e87bd907ea first commit MOHAN 2026-04-13 17:31:26 +05:30