ShortlistTraceBot

ShortlistTraceBot fetches a small number of public website pages when an authorized user requests an audit.

User agent

ShortlistTraceBot/0.1 (+https://shortlisttrace.com/bot)

What it does

The crawler reads publicly accessible HTML and sitemap information to identify business services, audiences, trust signals and page structure. It is not intended for bulk indexing or continuous crawling.

Crawl limits

Audits use a deliberately small crawl budget. The current crawler typically inspects no more than a limited set of representative pages per audit and applies request timeouts and response-size limits.

Robots and access controls

ShortlistTrace does not attempt to bypass authentication, paywalls or access controls. Sites can control crawler access through standard robots.txt directives. We may also block crawling where technical or security checks indicate a private or unsafe destination.

Why a site may see requests

A request normally means a ShortlistTrace user submitted that company domain for an audit or recheck.

Contact

If you operate a site and have a crawler question, use the ShortlistTrace support contact and include the affected domain and approximate request time.