1

Check the URL - no errors
https://www.dach-spengler.eu/sitemap.xml
https://www.dach-spengler.eu/robots.txt

In the google search console, I got the error:

Crawl-Anomaly

Submitted Sitemaps

  • Status Successful!
  • Found URLs: 515
  • Sitemap was processed without error
  • View index coverage (Link disabled)

Coverage

  • Excluded: 1 (Crawl anomaly)
  • Examples:
  • Test Robots.txt blocking:
    • Latest version seen on 12/06/2019, 21:40 OK (200)

Robots.txt

Sitemap: https://www.dach-spengler.eu/sitemap.xml
User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /component/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
User-agent: AimySitemapCrawler
Disallow: /images/sampledata/
Disallow: *?tmpl=component&print=1*

Inspect URL

Retrieving data from Google Index…

Result:

Coverage – Indexed, though blocked by robots.txt

Discovery

  • Sitemaps: N/A
  • Referring page: None detected

URL might be known from other sources that are currently not reported

Crawl

  • Last crawl: 4 Jun 2019
  • Crawled as: Googlebot desktop
  • Crawl allowed?: No: blocked by robots.txt
  • Page fetch: Failed: Blocked by robots.txt

Indexing

  • Indexing allowed?: Yes User-declared
  • canonical N/A
  • Google-selected: canonical: Inspected URL

Webseite

  • CMS Joomla 3.9.6
  • Akeeba Admin Tools Professional 5.3.2
  • Template: yoo_micasa
  • Server: managed server, 7.2 php, allow_url_include

Actual result

Error: Crawl-Anomaly

  • I just watched youtu.be/EUAMLEYAuno to learn more about the error that you received. Basically, the error indicates that a url that Google visited returned a 4## or 5## error code (the page didn't load properly). The video talks about how to dig into the report and identify the flawed url then potentially drill down into that page's resources via the browser's developer tools. We are going to need more specific details to help you to resolve the error. Please update your question when you can. – mickmackusa Jun 12 at 11:33
  • thanks! I tried to describe something in more detail. I watched the video, but did not help me. – cemfirat Jun 13 at 10:25
  • So Google is complaining because it can crawl to links that it has found. Simple as that, right? Seems like one of those "can't have your cake and eat it too" situations. – mickmackusa Jun 13 at 10:28
  • Sorry, I do not understand the solution. – cemfirat Jun 13 at 10:32
  • Stop Disallowing the Google bot from crawling if you are bothered by its inability to crawl your site. – mickmackusa Jun 13 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.