AI Index

GoTryAI-Collector

GoTryAI-Collector reads the pages AI providers and sellers publish about their own products (pricing, model and documentation pages) so ai.gotry.io can show their facts with a link back to each page.

  • User agent: “Mozilla/5.0 (compatible; GoTryAI-Collector/1.0; +https://ai.gotry.io/bot)”.
  • It honors robots.txt (RFC 9309), including Crawl-delay, and fetches a fixed list of pages (a few hundred) about once a day.
  • At most three requests at a time per host (two for pages we render), each started at least one second after the last; one at a time where robots.txt sets a Crawl-delay or the server answers 429. Unchanged pages are skipped where the server supports conditional requests.
  • It records facts only (prices, specs, dates) and links every value to the page it came from.

To stop it, disallow “GoTryAI-Collector” in your robots.txt; the next daily run respects it.

To ask us to stop, correct or remove anything, email contact@gotry.io. Terms, notices and privacy