Backup Delivery via SSH

Posted by admin on June 5, 2007 under Tech Tips | Be the First to Comment

If you’re not going to use tapes, CD’s, DVD’s, or other form of attached media for storing your backups, you’re more than likely going to use some form of a remote network storage repository. There are many ways to ship your *nix backups across a network to a remote file system. Using SSH (and its related tools) is among the most popular methods for this delivery process as it can be relatively fast, free, secure, and very flexible.

In the following examples, I’ll show you three ways to ship an archived folder to a remote SSH server.

Method 1: Secure Copy
Using ‘scp’, (secure copy), one can take any existing file and deliver it to an SSH server. This means that you can create a backup, store it temporarily to your “local” file system, and copy the file across the network.

In this example, one backs up a folder in their home directory called “myfiles” using tar and gzip compression, and then copies the resulting archive using scp to a folder called /archives on a remote SSH server.

$ tar -czvpf myfiles.tar.gz ~/myfiles
$ scp myfiles.tar.gz user@sshserver:/archives/
$ rm myfiles.tar.gz

Cool stuff, but the downside is two-fold:
(1) If your backup is larger than the available space on your local file system, this method obviously won’t work;
(2) If your backup is large, the entire process takes a little longer than you might find convenient, since you have to first create the backup, and then copy it across the network.

A better solution would be to start sending the backup during the file creation process, which leads us to to the next two methods.

Method 2: Concatenate to SSH
SSH can read from STDIN and print results to STDOUT, which means one can concatenate any type of “input” to a remote SSH server. For example, you could redirect the output of ‘tar’ using the following syntax:

$ tar czpvf - ~/myfiles | ssh user@sshserver "cat > /archives/myfiles.tar.gz"

As you can see, with a single command, you can both create and deliver the backup at the same time. The backup process does not take up any space on the local file system. Wicked cool!

There is however yet another way to accomplish this task as shown in the next section.

Method 3: Write to an SSH File System (SSHFS)
For those of you not familiar with SSHFS, this is a file system client based on SFTP and FUSE. This client allows you to mount any remote SSH server to a local empty directory, just as you would with other devices like CD/ROM’s, floppies, usb sticks, etc. What’s also great about this client is that it requires no server side modification. It’s resource friendly, and sending data is just as fast as any other SSH file transfer.

In Ubuntu 7.04, the fuse kernel module and utilities are installed by default, and sshfs is available in the repositories.

Once you have sshfs installed and working, the following example mounts the remote “/archives” directory to the local “~/temp-mount” folder, and then places the backup directly in the mounted file system. The file is transported across the network during the write process.

$ mkdir ~/temp-mount
$ sshfs user@sshserver:/archives ~/temp-mount
$ tar -czvpf ~/temp-mount/myfiles.tar.gz ~/myfiles

To unmount the directory,

$ fusermount -u ~/mnt

Conclusion:
As you can see, using SSH for the delivery of your backups can make your life a whole lot easier. A suggested practice would be to use DSA/RSA public key authentication for making SSH connections. This way, you don’t have rely on passwords every time the SSH client is used, which makes sense when applying any of the above examples to an automated process such as Crontab, or At.