CrowdAuditBot

CrowdAuditBot fetches pages for CrowdAudit website audits. It only visits a site when a CrowdAudit user starts an audit of it; it does not crawl the web on its own.

• Reads robots.txt and the sitemap, then follows links from the homepage to suggest pages to audit.
• Renders the chosen pages (a few per audit) in a real browser on desktop and mobile, and checks links on them.

Pages disallowed for CrowdAuditBot in robots.txt are not suggested for audits. To opt out, add User-agent: CrowdAuditBot / Disallow: / to your robots.txt, or email hello@crowdaudit.online.