Announcement

Collapse
No announcement yet.

Ahrefs bot crawling issue

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • mchernyavsky
    replied
    Hello KJ-K,

    The Ahref bot subnets are whitelisted and maintained:

    https://help.ahrefs.com/getting-started-with-ahrefs/about-ahrefs/what-is-the-list-of-your-ip-ranges

    Was it possible to identify the corresponding access log entries and compare IPs to the list?

    Leave a comment:


  • KJ-K
    replied
    Same issue today with Google bot. Received 500 error for our main domain and one of our subdomains.

    With high certainty there was no website downtime recently.

    Leave a comment:


  • KJ-K
    started a topic Ahrefs bot crawling issue

    Ahrefs bot crawling issue

    Hello,

    Received an email from Ahrefs that it has identified pretty much every webpage of our website with status 500. From the logs we can see with certainty that we didn't have any website downtime recently.

    Never had issues before with Ahrefs, Semrush, Majestic, Google, Bing, etc. So far we had been seeing this situation happening with Yandex and Imunify360 team even posted a notice to verify that. Yandex had been emailing us for about a year now that once every week their bot was getting into a 500 error on our website.

    Now it's Ahrefs turn. Most likely its the same issue that's prevent both bots.

    Just wanted to report that.
Working...
X