Q4RLBot#

Q4RLBot is a web crawler operated by Q4 Reach Labs GmbH, Munich, Germany.

Purpose#

It reads publicly available pages of online shops for market research. It does not log in, fill in forms or place orders.

How to recognise it#

  • The User-Agent contains Q4RLBot and a link to this page.
  • Requests are signed with Web Bot Auth (HTTP Message Signatures). The public key is published at /.well-known/http-message-signatures-directory on this site.

robots.txt#

Q4RLBot follows robots.txt, including Crawl-delay. It obeys the rules for Q4RLBot and, where there are none, the rules for *. To keep it out of a site:

User-agent: Q4RLBot
Disallow: /

Request rate#

Without a Crawl-delay, it sends at most one request every 5 seconds to a site. When a server answers with HTTP 429, it pauses its requests to that site.

Contact#

impressum.q4rl.2024@q4rl.com

made for 🫵 with ❤️ and 🩸