3

We have used Akeeba for some time now, however due to the size of backups many issues come up from using remote backups (S3).

Is it less error prone when using a remote ftp backup rather then S3? From what we notice, due to the splitting of files and the typical php timeout limit of 30 seconds, even optimizing settings the backups the backup can fail roughly 20% of the time, and increase depending on the size of the site.

Also we have tried variations in the settings for the S3 backup, 20% failure rate is about as low as we can get it on any server environment (with 500mb+ sites, lower sites had less of a problem). We recently stopped using Akeeba due to this in favor of automated WHM backups but have a situation where we need to use it again. the version was in the 3.7 range when we last used it actively (it was around 80 Joomla sites with varied versions).

So is the process Akeeba uses for backups safer/faster with FTP/SFTP? The reason I ask this question is the 20% failure rate in my opinion has nothing to do with Akeeba, but packet loss and such that leads to it failing, when creating automated backups a failure could be right when its needed most. So my thoughts are the connection through FTP/SFTP could be more reliable then HTTP.

Improve this question
6
  • How big are your S3 file chunks? We run ours at 20-40 meg and so far we haven't had many failures. I don't test every day's backups, but the times I've used it to move a site to another server it's always worked pulling back from our S3 account.
    – Brian Peat
    May 7, 2014 at 20:52
  • We have tried many, at 40 megs we can have sites with 20+ chucks. Also command line backups fail much easier, which is the "automatic" part. Typically we find that the safest way from a percentage standpoint 10megs fail the least (80+ parts) but still fails at times. If we can lower the failure rate to 5% or lower, that is currently our goal. Although I say 20% that is based on manual backups, 90% failure rate on command line is the norm for larger sites. We also use Watchful and have yet to get a successful backup from it.
    – Jordan Ramstad
    May 7, 2014 at 21:34
  • We have found that when a sites backup to Amazon S3 fails it is almost exclusively due to the hosting environment limiting the bandwidth out of the server. In every case where we moved clients off traditional hosts experiencing this problem to our EC2 instances the problem has been resolved. By the way Akeeba can often report successful transfers to S3 and leave entire backups on the webserver. Also 20% is not a high failure rate.
    – Craig
    May 8, 2014 at 0:01
  • Yeah, this is really interesting. The only time I had backups fail with watchful is the same way they failed with Akeeba-I had added a second db and forgotten to change the IP when we changed hosts. But Watchful works 100% for us. When I say 100% I mean in the backing up and moving of files. I don't always check the results to see if they work but again, I have used some of those backups to move sites so they have worked so far for us. You may want to talk to your web host about this.
    – Brian Peat
    May 8, 2014 at 0:56
  • Can i ask who you use for hosting? I shift lots of backups to Dropbox and have never had failure rates that i have seen. Usually use 50 meg chunks.
    – David Fritsch
    May 8, 2014 at 1:02

2 Answers 2

Reset to default
3

While restructuring our company website we had the same experience.

Previously we were using akeeba for backing up our websites. This time we decided to remove akeeba backup system from our joomla based website.

We wrote simple phing scripts to build backups and dump them to protected Amazon S3 service. This improved our backup process and it takes now 60-80 seconds, around 4x faster than akeeba backup.

You can use these scripts to improve backup and its transfer to s3. https://github.com/readybytes/joomla-scripts/tree/master/backup

Improve this answer
2

With the remote backup you want to switch to breaking the archive up into chunks. You need to experiment with the server set up you have. Basically php and FTP timeout and so you loose all the upload if not finished. start with 100MB and then go up or down if it works or not. I think we got up to 300-500MB on last site we did this to FTP back to an office server from remote. As most of our sites are 1-2GB with all the large images they use in promotion galleries.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.