Scoutly

Scoutly is a fast and lightweight website crawler and SEO analyzer that I started in November 2025 to practice Rust and rewrote in Go in 2026. It is inspired by Scrutiny.

Scoutly crawls same-origin HTML pages with configurable depth, page, concurrency, and request-rate limits, respects robots.txt, and discovers pages from XML sitemaps. It checks discovered links and images, and flags common SEO problems such as titles, meta descriptions, H1 headings, and missing image alt text. Results can be browsed in a keyboard-driven terminal UI or written as text and JSON reports, and the crawler is also available as an importable Go library.