0

I use Joomla 4.x, then I inspect the google search console and it only has a index.php crawled but any links shown in index.html are ignored, not followed.

this is the original robot.txt right from Joomla installation

# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

I want google to crawl my website other than disallow URL, thanks

Improve this question

1 Answer 1

Reset to default
1

In System > Global Configuration > Site make sure that under Metadata > Robots is set to "index,follow" in the dropdown.

Improve this answer
3
  • I did set to INDEX FOLLOW, but doesn't work, is it related to .htacces ?, but I know nothing about .htacces
    – dhany saguntoro
    Dec 7, 2022 at 23:06
  • @dhanysaguntoro If you think it might be related to .htaccess then you should update your question with the contents of your .htaccess file. However, there is nothing normally in .htaccess that would affect crawling.
    – MrWhite
    Apr 28 at 10:51
  • I created a new Folder inside Joomla Installation, the Files within this new folder are indexed normally by Google, and the files I mention here are PHP FILES with SQLite as a database, while Joomla uses MySQL, is it possible that the Problem is with MySQL server?
    – dhany saguntoro
    Apr 30 at 1:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.