Follow us on:
Backing Up Your Databases
It may go without saying, but database backups are just as vital, if not more so, than the design of your custom database application. With the introduction in FileMaker 12 of the new progressive backup feature, we decided to take an in-depth look at backup techniques/best practices, as well as the specific backup features offered by both 4D and FileMaker Server.
4D and FileMaker Backup Basics
Both FileMaker and 4th Dimension are similar with regard to how your database is backed up. The server application has a built in backup function that briefly halts all database transactions while the database is backed up to a location specified by the database administrator. During this brief time period, people using the application are prevented from saving data changes. For this reason, you really want to keep the backup running quickly if you are running it during the day, which is especially common with FileMaker.
This often means that the backup will be stored on a hard drive on the same computer on which the database runs, and is thus vulnerable to any problem that might affect the server. So, a critical piece of getting your database backed up properly is to have a secondary mechanism that backs up your backup to a different location.
Tape/External Hard Drive:
In the past, secondary backups were often done with a tape backup system, with tapes being rotated offsite in case of a disaster like an earthquake or fire. With the falling price of hard drives in the last decade, it has become common to back up to external hard drives. We have a number of clients that rotate a hard drive offsite for disaster protection.
Internet-based backup has come of age lately, allowing you to more painlessly store backups offsite (assuming you trust the internet backup vendor). Options for Internet backup include Crashplan Pro, Carbonite, Dropbox, Google Drive, and iCloud, to name only a few. How to configure each of these is beyond the scope of this article, but as an example, Dropbox can be used to easily get your database backup offsite – simply configure your backup to save the backup files to the Dropbox folder. Dropbox will take care of the rest.
One very important note with regard to using one of these Internet backup options is that they may not work if there is no user logged into your server. If you restart your server (Mac or Windows) your database will be accessible as soon as the restart is complete, but if you don’t log in after the reboot, applications like Dropbox may not function as expected. Dropbox is a good example of this issue in that your Dropbox folder will not be kept synchronized if the account under which the folder was created is not currently logged into the operating system. You’ll want to verify this functionality with any Internet-based backup tool you plan to use.
Keep in mind that regardless of how you run your secondary backup, you should not back up production files being actively served by FileMaker Server or 4D Server. The folder containing your live database files should be exempted from the secondary backup.
Database Backup in FileMaker Server
Over the years, FileMaker Server has made significant improvements to its backup tools. In version 7, we first gained the ability to schedule automatic backups. Version 9 gave us the backup schedule assistant, allowing easy creation and modification of backup schedules, as well as the option to verify backup integrity after performing backups, a very valuable feature that helps detect file corruption as soon as it occurs. In FileMaker Server 11, we had the option to save a clone of the file during backup, assisting in preserving data architecture. Now with the release of FileMaker 12, we have some major improvements to the standard scheduled backups as well well as an entirely new kind of backup process, progressive backup.
In the past, FileMaker Server scheduled backups would copy all files as defined by the schedule to the designated backup folder location. For solutions with large files, this process could take a long time and use up quite a bit of the hard disk space on the machine. Now in FileMaker Server 12 schedule backups will only copy files that were changed. The files that were not changed will be hard-linked to an existing previous copy. This will minimize the amount of disk space used by backups and also increase the speed of the backups. One important implication about this new process is that a backup file must never be opened from its backup folder, but copied to a different location and opened from there.
The progressive backup method improves application performance by backing up only the modified data in files. This backup runs significantly faster than the scheduled backups, since the entire file isn’t being backed up. In addition, the data in the progressive backup will never be more out of date than the backup interval you choose (10 minutes for example). Together with scheduled backups, FileMaker Server 12 delivers the tools to create a complete backup strategy.
You enable progressive backups, select the save path, and set the save interval from the "Folders" tab on the Configuration: Database Server settings page.
Progressive Backups vs. Scheduled Backups
The table below shows the differences between scheduled backups and progressive backups and how the two work together to form a complete backup system.
|Scheduled (since FMS7)||Progressive (NEW in FMS12)|
|Ability to define which databases will be backed up, and how often. No limit to how often databases are backed up.||Not database specific. Cannot specify start and end time. If enabled, progressive backups will run continuously at designated interval (anywhere between 1-60 minutes).|
|Creates full copies of databases that have changed and hard links to databases that have not changed. Increased speed of backups with new container management tools.||Improves performance and runs more quickly by only backing up file changes (after initial full backup). Increases stability by running its own process.|
|Backups are saved to the local hard drive, but cannot be saved to the root of the hard drive.||Must choose to backup outside of FMS backup folder. Best practice is to save to an external hard drive or cloud backup.|
|Unlimited number of backups to keep (limited only by available disk space).||Maximum of 2 full backups kept.|
Our typical configuration includes both progressive and scheduled backups. Progressive backups are enabled at an interval of 10 minutes. We schedule nightly backups that keep 7 days of archives, weekly backups that keep 4 weeks of archives, and hourly backups that run every two hours from 8:00 am to 5:00 pm and keep the most recent 4 archives. For systems with large amounts of data, the backup every two hours during the day can cause noticeable delays for users during the day. In these cases, progressive backup now gives us an reasonable alternative to these backups scheduled during the middle of the day.
More information on FileMaker Server backups: http://help.filemaker.com/app/answers/detail/a_id/7464/~/creating-backups-with-filemaker-server
Database Backup in 4th Dimension
Both 4D Server and the stand-alone 4D application have a built-in backup function (this is in contrast to FileMaker, where only the server application has the built-in backup function).
4D is fundamentally different in terms of backup from FileMaker in one important respect: the existence of a transaction log file. For every single data change made to the 4D data file, 4D writes out an entry in a separate log file. If the database becomes lost or corrupted for any reason, the most recent backup is restored and then the transaction log is applied to that backup to bring the database up to the most recent possible point before the recovery became necessary.
For this reason, in 4D we generally do not have the hourly (or more frequent) backups common to FileMaker Server configurations.
Our standard configuration for our 4D customers is to have a backup running each night, with an archive of 7 to 14 days worth of backups, and a log file being maintained throughout the day. Ideally, the log file and backups are stored on a separate drive from the production database. 4D provides additional backup flexibility by allowing backups to be controlled programmatically as well. Developers can essentially write their own backup system for unique situations requiring this level of control.
You can find detailed backup configuration documentation for 4D on the following web pages:
4Dv11 Backup: ftp://ftp-public.4d.fr/Documents/Products_Documentation/LastVersions/Line_11/VIntl/PDF_Format/4D_v11_Design_Ref.pdf
4Dv12 Backup: http://doc.4d.com/4D-Design-Reference-13/Backup-and-restoring-of-the-application.200-832499.en.html
Keep Up to Date with FileMaker and 4th Dimension
Who We Are
- FileMaker Pro & 4th Dimension upgrade and development
- MS Access to FileMaker conversions
- FileMaker Pro & 4th Dimension integration with other data environments
- Web/Database integration
- FileMaker Instant Web Publishing
Seattle - 425.481.8175 | Austin - 512.763.7864
Mailing Address: 17115 NE 163rd Place, Woodinville, WA 98072