leponceau.org

Programming And Stuff, You Know The Thing…

Subversion (svn) repository live replication via rsync/ssh

Posted at — Feb 8, 2019

Disclaimer

As always, this is a description of what I am doing. Use with care and know what you are doing.

Basic Replication


## I'm using Debian 8 and Subversion repository format 3.

$ cat /etc/debian_version 
8.11
$ cat repos/format 
3

## on root@<destination-server>
## (util-linux is for flock command, subversion for svnadmin verify)

$ apt install rsync subversion util-linux
$ useradd -m svnbackup
$ su - svnbackup
$ chmod 700 .

## empty password, so I can hook it up to cron:

$ ssh-keygen

## take note of your SSH public key:

$ cat .ssh/id_rsa.pub

## on source server, where my repository resides:

$ ssh root@<repository-server>
$ dpkg -S rrsync
rsync: /usr/share/doc/rsync/scripts/rrsync.gz
$ su - svn
$ gzip -dc /usr/share/doc/rsync/scripts/rrsync.gz > rrsync
$ chmod 755 rrsync

## now, using rrsync, add read-only rsync access to the svn repository via SSH:
## (replace the ssh-rsa ... part with your generated public key
## in svnbackup@<destination-server>:/home/svnbackup/.ssh/id_rsa.pub)

$ echo 'command="./rrsync -ro /home/svn/repos",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAA...' >> ~svn/.ssh/authorized_keys

## root@<destination-server>:~

## create run.sh script:

$ cat run.sh 
#!/bin/bash
if [[ -n $DEBUG ]]; then set -x; fi
set -eE
# never remove the lock file!
exec 9>>$0.lock
flock -w 1800 9
opts="-c"
if [[ "$1" == "fast" ]]; then
    opts="-i"
    echo "replicating to $(hostname)"
fi
rsync -a $cksum --del --backup --backup-dir=$HOME/repos.bak --suffix=.$(date +%Y%m%d-%H%M%S) svn@<repository-server>: $HOME/repos/
if [[ "$1" != "fast" ]]; then
    if ! svnadmin verify $HOME/repos >& /dev/null; then
        echo "svnadmin verify failed" >&2
        exit 2
    fi
fi

$ chmod 755 run.sh
$ crontab -e

## add the following lines to your crontab:

MAILTO=<email-address-for-error-reports>
@daily ./run.sh

The rsync command used for backup doesn’t have write access on the repository server. Also, it will never delete existing backup files, instead it will move them into the repos.bak directory, appending a timestamp to the filename. If you use a different repository format than 3, you might get into disk space troubles by keeping everything. That’s why I chose that specific, somewhat outdated format: because new commits for the most part simply result in few files added to the repository structure.

Live Replication

The ansatz used here has one huge advantage: the two servers don’t have write access to each other. In order to keep that safety advantage in a live replication scenario, one could simply add a post-commit hook to the repository that would start the run.sh fast script on svnbackup@destination-host via a SSH command:

## svn@<repository-server>:~

$ cat repos/hooks/post-commit
#!/bin/bash
set -Ee
ssh svnbackup@<destination-host>

$ chmod 750 repos/hooks/post-commit

## if you don't already have a SSH key-pair on the svn@repository-server
## account, do:
## (again, no password, take not of the public key)

$ ssh-keygen
$ cat .ssh/id_rsa.pub

## svnbackup@<destination-server>:~
## (again, replace the ssh-rsa ... part with the public key just created)
$ echo 'command="./run.sh fast",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1y....' >> .ssh/authorized_keys

That should do it. Cron now makes a daily backup incl. a verification down to each byte, then runs svnadmin verify. And a fast replication is done after each commit to the main repository.