Question: How can I track down the source of excessive resource usage on a Joomla site and fix it? Any tips or suggestions?
Details: My client's host keeps advising of excessive resource usage on his account. I have scanned the Joomla installation (3.6.5 - I will be updating to 3.7 shortly) using MyJoomla and the site does not appear hacked. There are very few extensions installed and any that are or very reputable, i.e K2, Akeeba etc and all are upto date as is the Joomla installation. I have disabled the none core extensions and the usage was still high (apparently).
The logs that were sent from the host all seem to be files which are not related to Joomla and that I cannot find in the clients account at all. For example:
/etc/httpd/domlogs/domain.com:188.8.131.52 - - [28/Apr/2017:18:24:25 -0700] "POST //plus/myjs.php HTTP/1.1" 200 - "http://www.domain.com//plus/myjs.php" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)"
Edit: I have just checked the access logs also and the highest number of hits come from the following:
The following are the most prevalent IP's visiting the site in April, the numbers show the number of visits from that IP:
2380 panscient.com 2617 MobileSafari/602.1 CFNetwork/811.4.18 Darwin/16.5.0 2825 MobileSafari/602.1 CFNetwork/808.3 Darwin/16.3.0 2951 Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 3418 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0 3529 Safari/12602.4.8 CFNetwork/807.2.14 Darwin/16.4.0 (x86_64) 3627 Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Version/10.0 Mobile/14D27 Safari/602.1 3699 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 4184 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 4204 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 4211 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 4554 Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/) 4629 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 4807 Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) 6054 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 6327 Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko 8363 Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/) 9994 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1 10027 Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html) 14472 Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, email@example.com) 133603 Java/1.8.0_121 258151 Mozilla/5.0
These were sorted using the mac terminal, the most concerning being the last 2 lines where it isn't actually clear who/what has been visiting wither 258251 from Mozialla/5.0. I'm going to look at block DotBot and the SEMRush bot but these are nothing in comparison to the last 2.
The host is not very good with responding to support requests, I do not even think they read the replies throughly to be honest. Nor do they keep their server software up to date, (still running php 5.6.28) and have ignored any questions I have put to them about why this isn't running 5.6.30. If they cannot even update PHP what else may not be up to date?
Anyway, has anyone else experienced anything like this before? I'm running out of ideas and feel this may be related to the host server and nothing at all to do with the Joomla installation.
Your comments/experiences with this sort of thing would be most appreciated. Anything at all really, even if it is host related and nothing to do with Joomla.