MachineryOneBot

MachineryOneBot is the automated reader that Machinery One uses to read the public used-equipment pages on Canadian dealers' own websites. This page is for the people who run those websites.

How to recognize it

Every request carries this user-agent:

MachineryOneBot/0.1 (+https://www.machineryone.ca/bot)

It is a plain request for pages that any visitor can open. It does not log in, fill in forms, solve challenges or use anyone's account.

What it reads, and what it does not

• The dealer's public used-inventory pages and the individual unit pages linked from them, to read year, make, model, hours or kilometres, location, asking price and the unit's own address.
• Sitemaps, where a dealer publishes one.
• It does not copy photos. It does not read customer data, parts catalogues or anything behind a login.
• It only reads dealers' own websites. Marketplaces and auction sites are not read.

How politely it behaves

• robots.txt first. It fetches your robots.txt and obeys the group named MachineryOneBot, or the * group if there is no specific one. Disallow rules, including the * and $ wildcards, are respected.
• One request every 3 seconds to any one website.
• It backs off. If your site answers 403 or 429, it stops asking and does not try another way in. If your site returns a server error it waits and tries a limited number of times.
• Schedule. A quick daily check that listed units still exist, and a fuller read once a week, all in the early morning, Pacific time. A daily pass also reads some unit pages for the dealer's description, which is used for search only.
• Some sites build their pages with JavaScript. For those, it loads the page in a headless browser that sends the same user-agent.

If your website blocks it

Many dealer websites use bot protection, and some of it blocks automated readers by default. If a dealer would like to be listed and the reader is being turned away, there are three ways to fix it:

1. Allow the user-agent. In your firewall, CDN or website vendor's bot settings, add an allow or skip rule for the user-agent containing MachineryOneBot. Our requests come from cloud servers whose addresses change, so an allow rule based on IP address will not hold. We know a user-agent can be imitated, so if your vendor will not allow a rule on it alone, use option 2.
2. Send us a feed. A CSV, XML or JSON file (or a link to one) with your used inventory works the same way. This is often the easiest route. We need: stock number, year, make, model, category, asking price in CAD, hours or kilometres, city and province, used or new, available or sold, and the address of the unit's own page.
3. Point us to a sitemap or inventory API that you are happy for us to read.

Please email hello@machineryone.ca so we know which dealer you are and can test it with you.

If you do not want it on your site

Add this to your robots.txt and it will stop reading your site at its next visit:

User-agent: MachineryOneBot
Disallow: /

You can also ask for any or all of your listings to be removed at any time through the dealers page or by email. We take them down within 24 hours.

What happens to what it reads

Machinery One is a free search site. It shows the unit's year, make, model, hours or kilometres, location, your name and logo, and your asking price in Canadian dollars. Every "View at dealer" button sends the buyer to your own page. We do not sell machines or charge dealers. More on how listings are chosen, and our privacy policy.

Contact

Chris Ireton, Machinery One, 2373 Selkirk Drive, Kelowna, BC V1V 2R5. hello@machineryone.ca.