0

My website is built by Joomla3.6+k2 2.7. Today I find out that Google Webmaster indicates that "Googlebot found an increase in server errors". On the "Crawl Error" page, I see something like the picture below:

enter image description here

Clicking one of the links I see this:

enter image description here

it is indicated in the picture that the tag list page was linked from http://www.weiwei-tv.com/index.php/component/k2/author/842-admin and http://www.weiwei-tv.com/index.php/component/k2/author/842-admin?start=10, These are k2 author pages, but I never enable author view in any occasion , so how does googlebot get to the author page? Also is there anyway to find out the "linked from" information for these two links? I asked this question on "webmaster" but they said it was too specific.

Also a side question, the links in the first picture are actually good, why googlebot encounters 500 error when crawling them? I copy two of the tag list pages in the first picture below: http://www.weiwei-tv.com/indexindex.php/component/k2/tag/%E8%B5%B5%E8%96%87

http://www.weiwei-tv.com/index.php/component/k2/tag/%E4%B8%AD%E5%9B%BD%E7%BA%BD%E7%BA%A6%E7%94%B5%E5%BD%B1%E8%8A%82

Improve this question

1 Answer 1

Reset to default
1

GoogleBot is a very aggressive bot. We have the exact same issue on K2 based websites, and we have determined that this is mainly caused by Google knowing some patterns (from other K2 websites) and so it tries to index pages that it knows it may exist on your website, such as author pages, despite the fact that you don't have any links to the author page.

I can give you another example. We have a K2 based page that has items in it, but none of these items are linked and nor there are links to these items anywhere from the website, but GoogleBot has indexed those items anyway. How? Again, pattern (it might be something else, but we think it's pattern).

As for your 500 links, it seems that your website experienced a hiccup end of June - beginning of July. I don't think it's anything to worry about.

I don't know how big your website is, but, on large websites, or websites with many extensions, forcing Google to index the pages that you want (and only the pages that you want) is a losing battle.

Improve this answer
2
  • Your answer sounds convincing to me, thanks. But google can figure out the author's link as "author/842-admin"? I mean how did google get this number "842", do they try like 1 to 1000?
    – shenkwen
    Commented Aug 17, 2016 at 16:23
  • 1
    I really can't answer this question - but you can try to find an answer from Google. Here's how: go to your WebmasterTools, then click on Search Traffic -> Internal Links, and then try to search for the internal links linking to a specific page (there is a search input field at the top).
    – itoctopus
    Commented Aug 17, 2016 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.