Give it a sitemap URL or drop a sitemap XML file. Every URL inside gets a live HTTP status.
Lots of 403s? That is almost always bot protection reacting to how the
request looks, not to the URL. Leave the three boxes above checked: a 403 or 405 is automatically
retried as GET with a full browser header set, then again with a Referer
and any cookies the host set. Each blocked row names the WAF it detected. Also try concurrency
2–3 with a 300–1000 ms interval — burst rate alone triggers some rules.
429 Too Many Requests? Retries honor the server's Retry-After, and
every 429 widens the interval automatically for the rest of the run.
Count URLs reads the sitemap (and every child sitemap, if it is an index) and tells you how many URLs are in it — without sending a single request to those URLs.
Pick a bucket above (or type a filter), change any setting, then Re-check these to re-run only those URLs — results are merged into the table in place, and each re-checked row shows what it returned last time.
| # | Status | URL | Time |
|---|