# bot-policy.com

> Archive of robots.txt and llms.txt rules for AI and search crawlers.

bot-policy.com records what websites tell bots. It never scrapes HTML page content. The only outbound fetches are HTTP GET to `https://{domain}/robots.txt`, `/llms.txt` and `/llms-full.txt`.

Public lookup is one domain at a time. If a host is missing from the register it is added and those three files are fetched. Historical snapshots and daily counts require an API key (`bp_…`).

## Pages

- [Home](https://bot-policy.com/): look up a domain
- [API guide](https://bot-policy.com/docs): public and authenticated endpoints
- [Full site notes](https://bot-policy.com/llms-full.txt): longer description of the archive and API

## Optional

- [robots.txt](https://bot-policy.com/robots.txt)
- [Open Graph image](https://bot-policy.com/og.png)