Automated web crawler for up to 1,000 pages. JS rendering, PDF extraction, scheduled re-crawling.