1

While trying to generate a sitemap with a crawler, it is finding some really weird URLs which is not giving a 404 and displaying incorrect content on the wrong section.

So for example, I have a Menu called fiction with sub item used-fiction/offers. The fiction section has it's own template which makes it look different from Autobiography section but somehow the crawler is finding the URL below and the loading content of the id=124 under the fiction template which is completely wrong. There are no menu items pointing to that article that is in the Fiction Menu. That article ID has a menu item in 'Autobiography' Menu which is working fine.

/fiction/used-fiction/offers?id=124:a-long-walk-to-freedom&catid=2

I wondering if there is a way I could identify where these links might be in content table but checking in the database, I can't find anything that is remotely like this. Any suggestions would be great!

2

I would take a dump of the database, then use notepad++ or similar to search the sql file for "id=124:a-long-walk-to-freedom". If something was found it should be relatively easy to fix.

This is assuming that during your previous search attempt you forgot to compensate for SEF rewriting parts of the link or forgot to search module content. Searching the entire database with a text editor for a shorter part of the string should cover it.

If even that fails the link might also come from the language files.

If you have a server access log, you could simply search that to see where the crawler follows the link from.

  • Apologies for the late reply. I tried the database dump approach which showed no such ids. But what I did do was to use those ids and catids to search in the introtext column in the com_content and I found quite a few non-sef URL (index.php?...) with those ids/catids. The reason for it mostly was we had migrated a huge number of content from an older Joomla site and the problem pages had internal links that were not cleaned up properly. Thank you for reminding me about the access log as well as it will be a great help in determining those problem urls. – sifu Feb 12 '15 at 23:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.