Our web crawler for indexing the open web
Mozilla/5.0 (compatible; inetdex-bot/1.0; +https://inetdex.com/bot.html)
robots.txt
To block our bot, use the following in your robots.txt file:
User-agent: inetdex-bot Disallow: /