EWMuseumBot

Found this in your logs? Thanks for looking before blocking.

What it is. The preservation crawler of the E-Wrestling Museum, an archive of fantasy wrestling (e-feds, play-by-mail leagues, roleplay federations). Operated by a person, for preservation — not a search engine, ad harvester, or training-data collector.

The rules. Active communities are collected and held, never published. Only closed promotions are exhibited, with private people's real names redacted.

Where it goes, and why

EWMuseumBot visits the public pages of community sites — forums, league sites, federation homepages — for three reasons, and no others:

1. To preserve. A closed federation's site that is still online today may not be tomorrow. The museum copies it while it can, and holds the copy.
2. To list what's coming. Once a week it looks at a living federation's own public pages for upcoming shows. Those pages are never served by the museum; a show is only listed, by date, in the museum's calendar of upcoming shows, with a link back to the federation's own page.
3. To find where the hobby lives. A census of which communities exist and where, so the museum knows whom to ask and what to save.

How it behaves. Identifies as EWMuseumBot/1.0 (+https://crawler.ewmuseum.org/), obeys robots.txt (both * and its own name; a group naming EWMuseumBot wins over *), keeps to one stream per host — one page at a time — and waits eight to twenty seconds between requests on a forum, longer if your Crawl-delay asks. It never logs in or posts, and it stops at a refusal: a block, a rate limit or a challenge page ends the visit for that site, and nothing is retried around it.

To block it:

User-agent: EWMuseumBot
Disallow: /

Checked before every visit; honored immediately, no retry under another name. And if your promotion has history you'd rather hand over whole someday, the museum prefers records given from your own hands over crawls — this page will carry the way in when the doors open.

The archive-research identity

Collection work against public web archives — the Wayback Machine and its CDX index, Common Crawl, Arquivo.pt — and only against them, never against a community site, identifies as:

ewrestling-museum-collection-analyst/1.0 (+https://crawler.ewmuseum.org/; contact: crawler@ewmuseum.org)

Serial, rate-limited, backs off on load signals.

Questions

Both identities answer to the same person and the same address, crawler@ewmuseum.org. Write if the crawler has bothered you, if you want a page left alone, or if you want to talk.