The problem

I have close to a dozen sites that I personally handle.  That includes feature requests, performance, and backups.  To perform backups, I pretty much set aside a decent chunk of time to manually export the WordPress databases, then I FTP down the corresponding files.

This is not exactly the fastest process, but I’ve certainly reinforced anything and everything there is to know about setting up a WordPress-based site.  The larger downside is that it requires my active time – it’s not something that just automatically happens.

Well, I’ve been thinking of writing my own software to do this.  But time management can be a challenge.  Instead, I researched backup solutions and went with BackupBuddy – it’s a plugin that supposedly backs up database and files on a schedule, then sends them out to cloud storage like Drive, Dropbox, etc.

The attempt

In general, it seems to work fine.  That is, except for two very key pieces:

1. The cron job that runs the schedule only activates if someone is visiting some portion of the site (front or admin ends).
2. Large file backups are not reliable.  It sounds like there is an issue between the server and possibly the software.  More on this, next.

So, I have two sites I’m backing up.  One has about 40MB of data.  The other has about 4GB of data.  Both sites initially failed.  I contacted support from iThemes and got this instructional response:

you’ll need to please try the Alternative Zip System with the following settings:

BackupBuddy -> Settings -> Advanced Settings/Troubleshooting -> Zip
Select Alternative Zip System
Set Zip Build Strategy to Multi-burst/Multi-Step (please make sure this is Multi-Step)
Set Maximum time per chunk to 90 (seconds) (default is 30)
Set Maximum content size for a single burst to 20 (MB) (default is 100) – leave Min at 10
Save and try those settings.

Depending on the complexity and/or size of the site you may need further assistance.

So, I made these changes and the lesser-data site succeeded; however, the greater-data site still failed.  After further support discussion, it turns out that the failure is caused because there is at least one file that is more than 20MB in size – this is the maximum I had just changed.  iThemes blamed it on the server speeds – although, that feels really lame that there is not a method on the BackupBuddy side to slow down the process or break up the compressed files into smaller pieces.

The next day

I tried BackupBuddy on another one of my sites.  It does have a lot of large files, so I ended up ignoring the offending folder.  That seems to have worked, maybe.  But I got another odd error:

Backup halted. Maximum number of attempts made attempting to access locked fileoptions file. This may be caused by something causing backup steps to run out of order or file permission issues on the temporary directory holding the file `/<mysite>/wp-content/uploads/pb_backupbuddy/fileoptions/vgnub51tbw.txt`. Verify correct permissions.

Contacting iThemes again, I received the following response:

The problem seems to be something in regards to the sending of your backup file to your Google Drive remote destination. The log doesn’t specifically say why it didn’t send, but from my experience I can take a guess that your server didn’t have enough memory during the attempt.

You could try setting the Google Drive “Send per burst” lower to work within your server’s restrictions. By default it’s set to 25, but you could try 20 and even 15 or 10 if 20 still doesn’t work. To get to the remote destination “Send per burst” option you have to go here:
wp-admin -> BackupBuddy -> Remote Destinations ->
Then to the right of the remote destination that you are wanting to work with is a gear icon. Just click it. Once it opens, click on “Advanced Options”. You should see the option “Send per burst”.

You might also try excluding items from your backup to make the backup file smaller:
wp-admin -> BackupBuddy -> Server Tools -> Site Size Maps -> Display Directory Size Listing
Then you can exclude them here:
wp-admin -> BackupBuddy -> Settings -> General Settings -> File & Directory Defaults ->
Things you might be able to exclude are any directories in the root of your install like a /stats/, /logs/, /errors/, or any directory that your host may have placed there or anything else that is not part of a standard WordPress install. Also any /cache/ directories that might be in your /wp-content/ directory as they are not usually needed in a backup because they will be rebuilt on a restore/migration. Also, if you’ve ever used any other backup plugin, please make sure that you are not making backups of other backups as that is somewhat redundant and the size of the backup will get incrementally larger with each backup attempt.

It does appear that the backup completed, but failed deploying.  So, I redeployed with the new “Send per burst” setting and it appears to have worked.  Crossing fingers that the full backup completely works on the next scheduled time.