Wednesday, August 12, 2026 For AI agents: /llms.txt
Poliety
Building the business layer of the agent internet.

PolietyBot

The crawler behind Poliety Concierge: what it fetches, why, and how to control it.

What it is

PolietyBot fetches public web pages to build source-grounded knowledge bases for Poliety Concierge customers. It crawls a domain only after the domain's owner has verified control of it and asked for the crawl. It is not a search engine spider and does not roam the open web.

How it behaves

  • User-agent: PolietyBot/1.0 (+https://poliety.com/bot)
  • Respects robots.txt by default. A customer may override it only for a domain they have verified they own.
  • Paces requests per host and honors a bounded page budget per crawl.
  • Fetches over HTTPS where available, follows a limited number of redirects, and never executes page JavaScript.
  • Stores fetched content with a content hash so customers can see exactly which version of a page a fact came from.

How to block it

Add this to your robots.txt:

User-agent: PolietyBot
Disallow: /

Status

Concierge is in build and PolietyBot is not yet crawling anyone. This page exists ahead of the first crawl so the user-agent string always resolves to an explanation.

Contact

Questions or reports about crawler behavior: support@nuralyn.com. Security reports: see /.well-known/security.txt.