About our crawler

PostRequest keeps a directory of food and beverage suppliers so small manufacturers can request quotes. To keep it accurate, a small crawler called PostRequestBot visits supplier websites. It identifies itself with this User-Agent:

PostRequestBot/1.0 (+https://postrequest.io/about-our-crawler; supplier directory)

What we collect

• Public business facts: company name, website, city and state, and the kinds of products or services you offer.
• A role-based business email that your own website publishes for enquiries, such as info@ or sales@.
• We don't collect named individuals' email addresses, personal or free-mail addresses, phone numbers, or anything behind a login. We don't decode addresses a site has chosen to hide.
• Some listings start from public government datasets. Contact details still come only from the company's own website.

How we crawl

• We honor robots.txt. We read the rules for * and for PostRequestBot, never fetch a disallowed path, and respect a Crawl-delay.
• We visit only a few pages per site (the home page and a contact or about page), with a pause between requests to the same site.
• If your site answers with an error or tells us no (for example a 403), we stop and don't retry. We never change our User-Agent to get around a block.

How to opt out

• To stop the crawler, add this to your robots.txt:

  User-agent: PostRequestBot
  Disallow: /

• To have your company removed from the directory, or to stop all email from us, write to info@postrequest.io from your company domain and tell us which listing it is.
• Every email we send to suppliers includes a one-click unsubscribe link.