Tweet
Hello,
Some of our users have been complaining because inside their Majestic SEO tool, they get the anti-bot challenge i360 page instead of their content.
This is also an issue for their SEO metrics that come from netlinking (e.g., TrustFlow, CitationFlow…) unlike SEMRush or Ahrefs. It is actually impossible to safely whitelist the MJ12bot IPs because they use what they call "community IP”:
So, if I understand well, they use a large amount of random servers instead of their own IPs for funds concerns :
The thing that makes it even more complicated for MJ12bot is that most of their community servers are listed in one or several RBL and few reports online are saying that this crawler can sometime overload the servers.
My concern is that many of other firewalls don't experiment this issue (for instance the Cloudflare anti-bot protection).
So, my users complain. "Why only Imunify 360 experience the One Moment Please issue and not other web hosting providers ?".
Because user-agent can be faked and the community IPs used by MJ12bot are infinite, it's clear that making a hole based on user-agent and/or their hundreds of dirty IP addresses from anywhere is not on the schedule.
My thought is that MJ12bot takes some time to improve its crawler to comply with the human verification process of i360 but it seems that the communication with them is difficult :
Source: MJ12bot[.]com
So, I threw this bottle into the sea to take my chance to meet other i360 anti-bot users who also want to make Majestic compatible with our lovely Imunify 360.
Some of our users have been complaining because inside their Majestic SEO tool, they get the anti-bot challenge i360 page instead of their content.
This is also an issue for their SEO metrics that come from netlinking (e.g., TrustFlow, CitationFlow…) unlike SEMRush or Ahrefs. It is actually impossible to safely whitelist the MJ12bot IPs because they use what they call "community IP”:
we do not use any consecutive IP blocks as we are a community based distributed crawler. (...)
As a community project unfortunately we don't have the ability to restrict our bots to a limited number of IP addresses, as some of our better funded counterparts do.(...)
My concern is that many of other firewalls don't experiment this issue (for instance the Cloudflare anti-bot protection).
So, my users complain. "Why only Imunify 360 experience the One Moment Please issue and not other web hosting providers ?".
Because user-agent can be faked and the community IPs used by MJ12bot are infinite, it's clear that making a hole based on user-agent and/or their hundreds of dirty IP addresses from anywhere is not on the schedule.
My thought is that MJ12bot takes some time to improve its crawler to comply with the human verification process of i360 but it seems that the communication with them is difficult :
Some ISPs and badly configured firewalls may stop MJ12Bot from crawling your website. This is usually because the ISP or Firewall does not understand that in doing so, they are blocking genuine visitors to your website at a later date. Some also do this to minimize bandwidth. In these instances, some ISPs can remove the block for all their users when they understand the purpose of the bot. If your ISP will not allow our bot, we recommend that you consider moving ISPs.
So, I threw this bottle into the sea to take my chance to meet other i360 anti-bot users who also want to make Majestic compatible with our lovely Imunify 360.
Comment