Duplicity is a space- and bandwidth-efficient backup software utility built atop the rsync and GnuPG technologies. It provides a relatively simple way to create full or incremental backups, either from a command line or using a variety of graphical user interfaces. By default, it uses password-based (symmetric) encryption to protect data at rest, and is compatible with numerous storage backends including Free Software and even technocapitalist SAASS offerings.
We recommend the use of
duplicity(1) as a simple and effective means of being prepared for disaster recovery.
- 1 Installing
- 2 Using
- 3 Hardening
- 4 Provisioning
- 5 See also
GNU/Linux users may already have
duplicity installed on their systems. If you do not, on Debian-derived GNU/Linux distributions, install
sudo apt install duplicity
This section describes several basic uses of
Automated symmetrically encrypted backup via rsync over Tor¶
In this configuration,
cron is used to schedule an unattended backup of a given important directory, which will be encrypted locally before being stored on the destination server. The remote host is accessed as a Tor Onion service exposing an
rsync daemon. When the configuration is complete, a single cron job of the following form will automatically back up the important directory:
PASSPHRASE="$(/bin/cat $HOME/.duplicity.secret)" RSYNC_CONNECT_PROG='/bin/nc -x 127.0.0.1:9050 \%H 873' /usr/bin/duplicity --gpg-options="--cipher-algo=AES256" --rsync-options="--password-file=$HOME/.rsync.secret --partial --compress" /path/to/important/directory rsync://email@example.com::backups/ >/dev/null 2>&1
Defining a backup schedule is the responsibility of the data owner, as is determining how many backups to keep and for how long. In this configuration, the following backup policy is assumed, but you should change this to suit your needs by editing the relevant cron job schedules and
duplicity command options:
- Weekly incremental backups.
- Bi-monthly full backups.
- Two years worth of full backups are retained (12 full backups maximum).
$HOME/.duplicity.secretbe a file containing your backup archive password. Make sure this file is
backup_userbe the user that will be running the backups on the local machine.
$HOME/.rsync.secretbe your rsync user account’s login password. Make sure this file is
rsync_userbe the rsync account name that the
backup_userwill use to login to the rsync server.
abcdef0123456789.onionbe your remote rsync Onion service. It is assumed that you have already configured your Tor client to authenticate succesfully, and that an rsync daemon is listening.
/path/to/important/directorybe the filesystem path containing the data you want to backup.
backups/be the exported rsync module that you are backing up to.
Do this to set up automated symmetrically encrypted backups using
duplicity that are stored on a (hopefully off-site)
rsync Tor Onion service.
- Terminal in to the server and become the user who will be performing the backup, say
ssh the.server.local sudo su --shell /bin/bash --login backup_user
- Generate a password to use to encrypt the backup archive, and write this password to a file (for example,
.duplicity.secret). This will be used to ensure the remote host never receives a plaintext copy of any filesystem data. Be certain to record this password somewhere safe (such as a password/secret manager application), as you will need it to restore the (encrypted!) backup archive.
# Collect 45 random bytes (from `/dev/random`?) and base64 encode them. gpg --armor --gen-random 2 45 > .duplicity.secret # Ensure the password file is not readable by any other users. chmod 600 .duplicity.secret
- Obtain the password for your rsync user and write it to a file, as well.
vim .rsync.secret # Put the password to your rsync user account in this file. # Ensure the password file is not readable by any other users. chmod 600 .rsync.secret
- Create or edit a
crontab(5)file for the user that will be running the backups:
- Add the following entry in the backup user’s cron table file. Note that any literal percent signs (
%) must be escaped, as they have special meaning to most implementations of the
# For more information see the manual pages of crontab(5) and cron(8) # Cron entry fields: # # ┌───────────── minute (0 - 59) # │ ┌───────────── hour (0 - 23) # │ │ ┌───────────── day of month (1 - 31) # │ │ │ ┌───────────── month (1 - 12) # │ │ │ │ ┌───────────── day of week (0 - 6) (Sunday to Saturday; # │ │ │ │ │ 7 is also Sunday on some systems) # │ │ │ │ │ # * * * * * command to execute # Run (incremental) backups to a remote Onion at 3:05am every Tuesday. 5 3 * * 2 PASSPHRASE="$(/bin/cat $HOME/.duplicity.secret)" RSYNC_CONNECT_PROG='/bin/nc -x 127.0.0.1:9050 \%H 873' /usr/bin/duplicity --full-if-older-than 2M --gpg-options="--cipher-algo=AES256" --rsync-options="--password-file=$HOME/.rsync.secret --partial --compress" /path/to/important/directory rsync://firstname.lastname@example.org::backups/ >/dev/null 2>&1 # Remove the 13th (and older) full backups, i.e., retain the most recent 12 full backups and their incremental diffs. 5 4 * * 2 RSYNC_CONNECT_PROG='/bin/nc -x 127.0.0.1:9050 \%H 873' /usr/bin/duplicity remove-all-but-n-full 12 --force --rsync-options="--password-file=$HOME/.rsync.secret --partial --compress" rsync://email@example.com::backups/ >/dev/null 2>&1
A breakdown of the command invocations:
PASSPHRASE="$(/bin/cat/ $HOME/.duplicity.secret)"– Save the contents of the file
$HOME/.duplicity.secretas the environment variable
PASSPHRASE. This variable is checked by
duplicity(1), which will pass it to
gpgto use as the encryption password. This is only required when encrypting (backing up).
RSYNC_CONNECT_PROG='/bin/nc -x 127.0.0.1:9050 \%H 873'– Set the environment variable
RSYNC_CONNECT_PROGto a netcat (
nc) proxy (
-x) command. The sequence
%His replaced by
rsyncitself with the address of the remote host. It is backslash-escaped because
%characters are treated specially in
duplicity(1), with the following options and arguments:
--full-if-older-than 2M– If the last full backup is older than 2 months, do a full backup instead of an incremental one.
--gpg-options"--cipher-algo=AES256"– Instruct the
gpgexecutable to use the AES256 cipher algorithm. This arguably improves the security of the backup archive by using a more modern cipher (
AES256) than the
--rsync-options="--password-file="$HOME/.rsync.secret" --partial --compress"– Instruct
rsync, when invoked, to read its user account’s password from the file
$HOME/.rsync.secret, to record partial transfers, and to compress the datastream before being sent to the network. The
--password-fileis needed to allow for non-interactive execution. Meanwhile, partial transfers ensure that interruptions caused by, e.g., network issues, can be resumed on the next run, and compression speeds the transfer by minimizing the amount of data that needs to be emitted on the network link.
/path/to/important/directory– The duplicity backup source directory.
@abcdef0123456789.onion::backups/– The duplicity target URL, i.e., the location where the backup archive will be sent.
- In the second command, invoke
/usr/bin/duplicityagain, but with these alternate instructions:
remove-all-but-n-full 12– Remove all old backups except the most recent
--force– Actually do the removal; this is required by
duplicityin order to ensure you mean what you say.
>/dev/null 2>&1– Redirect the command’s
/dev/null, and redirect the command’s
STDOUT(which is going to
If you need to use
sudo(1) to run the backup command as another user, say,
other_user, you can pass the
RSYNC_CONNECT_PROG environment variable through the
sudo invocation like so:
sudo RSYNC_CONNECT_PROG='nc -x 127.0.0.1:9050 %H 873' -u other_user duplicity --rsync-options="--password-file=/path/to/password/file" rsync://firstname.lastname@example.org::module_name/ /path/to/restoration/directory
To restore a Duplicity backup archive, re-run the same command as that used to create the archive but swap the positions of the source URL and the target directory. For example:
- If the backup command was…
duplicity --rsync-options="--password-file=/path/to/password/file" /path/to/important/directory rsync://email@example.com::module_name/
- …then the restore command is:
duplicity --rsync-options="--password-file=/path/to/password/file" rsync://firstname.lastname@example.org::module_name/ /path/to/important/directory
Use strong GPG ciphers¶
Duplicity makes use of a locally installed
gpg to perform encryption and decryption on its behalf. By default, most versions of
gpg use the
CAST5 cipher, despite more modern ciphers being available. View the available cipher list by invoking
gpg --version. A good cipher is
duplicity(1) to use this cipher when calling
gpg by passing
The AnarchoTech NYC collective provides an Ansible role for automating the setup of Duplicity backup jobs that run on a Raspberry Pi. It can be installed in your local
$ANSIBLE_ROLES_PATH (see Ansible Configuration Settings) for use with an Ansible project with:
ansible-galaxy install https://github.com/AnarchoTechNYC/ansible-role-duplicity/archive/master.tar.gz
- ArchWiki: Duplicity
- The Arch Linux community’s own documentation for use of
duplicity, including an example backup script.