Unison : synchronisation de données
Bonjour à tous !
3 mots sur un outil qui permet de synchroniser deux espaces de données : Unison.
Son intérêt peut paraître très relatif, notamment en comparaison de protocoles plus utilisés et plus aboutis comme RSync, toujours est-il qu'il est capable de créer et de maintenir 2 réplications de données, qu'elles soient locales (ou montées) ou distantes accessibles en SSH, et de gérer les modifications source vers destination ou bien destination vers source.
Pourquoi ne pas faire un détour ?
Installation from src
Comme d'habitude, partons des sources. A l'heure où j'écris ces lignes, nous en sommes à la 2.40.61, donc l'installation se basera sur cette version. Attention à corriger lors de la parution d'une version supérieure.$ cd /usr/local/src/ & wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.40.61.tar.gz $ tar zxf unison-2.40.61.tar.gz $ cd unison-2.40.61 $ make mkProjectInfo $ makeLe make install a planté sur la copie du binaire sous ~/bin/ (pour la simple et bonne raison que ~/bin n'existe pas). Pour résoudre le problème, il faut modifier le Makefile, et notamment la ligne 82. En commenté, le répertoire qui doit contenir les binaires est mis à ~/bin par défaut. En le mettant à /usr/local/bin, c'est tout de suite plus propre...
#INSTALLDIR = $(HOME)/bin/ INSTALLDIR = /usr/local/binLe / manquant implique une autre modification, ligne 95 :
# cp $(NAME)$(EXEC_EXT) $(INSTALLDIR)$(NAME)-$(MAJORVERSION)$(EXEC_EXT) cp $(NAME)$(EXEC_EXT) $(INSTALLDIR)/$(NAME)-$(MAJORVERSION)$(EXEC_EXT)
$ make install mv /usr/local/src/unison-2.40.61 /tmp/unison-7838 cp unison /usr/local/bin/unison cp unison /usr/local/bin/unisonunison-2.40Nous avons fini l'installation. Pour avoir une idée des options disponible, il suffit de faire appel au traditionnel -help.
$ /usr/local/bin/unison -help
Usage: unison [options]
or unison root1 root2 [options]
or unison profilename [options]
Basic options:
-auto automatically accept default (nonconflicting) actions
-batch batch mode: ask no questions at all
-doc xxx show documentation ('-doc topics' lists topics)
-fat use appropriate options for FAT filesystems
-group synchronize group attributes
-ignore xxx add a pattern to the ignore list
-ignorenot xxx add a pattern to the ignorenot list
-nocreation xxx prevent file creations on one replica
-nodeletion xxx prevent file deletions on one replica
-noupdate xxx prevent file updates and deletions on one replica
-owner synchronize owner
-path xxx path to synchronize
-perms n part of the permissions which is synchronized
-root xxx root of a replica (should be used exactly twice)
-silent print nothing except error messages
-terse suppress status messages
-testserver exit immediately after the connection to the server
-times synchronize modification times
-version print version and exit
Advanced options:
-addprefsto xxx file to add new prefs to
-addversionno add version number to name of unison on server
-backup xxx add a pattern to the backup list
-backupcurr xxx add a pattern to the backupcurr list
-backupcurrnot xxx add a pattern to the backupcurrnot list
-backupdir xxx directory for storing centralized backups
-backuploc xxx where backups are stored ('local' or 'central')
-backupnot xxx add a pattern to the backupnot list
-backupprefix xxx prefix for the names of backup files
-backups keep backup copies of all files (see also 'backup')
-backupsuffix xxx a suffix to be added to names of backup files
-confirmbigdel ask about whole-replica (or path) deletes (default true)
-confirmmerge ask for confirmation before commiting results of a merge
-contactquietly suppress the 'contacting server' message during startup
-copymax n maximum number of simultaneous copyprog transfers
-copyprog xxx external program for copying large files
-copyprogrest xxx variant of copyprog for resuming partial transfers
-copyquoterem xxx add quotes to remote file name for copyprog (true/false/default)
-copythreshold n use copyprog on files bigger than this (if >=0, in Kb)
-debug xxx debug module xxx ('all' -> everything, 'verbose' -> more)
-diff xxx set command for showing differences between files
-dontchmod when set, never use the chmod system call
-dumbtty do not change terminal settings in text UI
-fastcheck xxx do fast update detection (true/false/default)
-follow xxx add a pattern to the follow list
-force xxx force changes from this replica to the other
-forcepartial xxx add a pattern to the forcepartial list
-halfduplex force half-duplex communication with the server
-height n height (in lines) of main window in graphical interface
-host xxx bind the socket to this host name in server socket mode
-ignorearchives ignore existing archive files
-ignorecase xxx identify upper/lowercase filenames (true/false/default)
-ignoreinodenumbers ignore inode number changes when detecting updates
-ignorelocks ignore locks left over from previous run (dangerous!)
-immutable xxx add a pattern to the immutable list
-immutablenot xxx add a pattern to the immutablenot list
-key xxx define a keyboard shortcut for this profile (in some UIs)
-killserver kill server when done (even when using sockets)
-label xxx provide a descriptive string label for this profile
-links xxx allow the synchronization of symbolic links (true/false/default)
-log record actions in logfile (default true)
-logfile xxx logfile name
-maxbackups n number of backed up versions of a file
-maxerrors n maximum number of errors before a directory transfer is aborted
-maxthreads n maximum number of simultaneous file transfers
-merge xxx add a pattern to the merge list
-mountpoint xxx abort if this path does not exist
-nocreationpartial xxx add a pattern to the nocreationpartial list
-nodeletionpartial xxx add a pattern to the nodeletionpartial list
-noupdatepartial xxx add a pattern to the noupdatepartial list
-numericids don't map uid/gid values by user/group names
-prefer xxx choose this replica's version for conflicting changes
-preferpartial xxx add a pattern to the preferpartial list
-repeat xxx synchronize repeatedly (text interface only)
-retry n re-try failed synchronizations N times (text ui only)
-rootalias xxx register alias for canonical root names
-rsrc xxx synchronize resource forks (true/false/default)
-rsync activate the rsync transfer mode (default true)
-selftest run internal tests and exit
-servercmd xxx name of unison executable on remote server
-showarchive show 'true names' (for rootalias) of roots and archive
-socket xxx act as a server on a socket
-sortbysize list changed files by size, not name
-sortfirst xxx add a pattern to the sortfirst list
-sortlast xxx add a pattern to the sortlast list
-sortnewfirst list new before changed files
-sshargs xxx other arguments (if any) for remote shell command
-sshcmd xxx path to the ssh executable
-stream use a streaming protocol for transferring file contents (default true)
-ui xxx select UI ('text' or 'graphic'); command-line only
-unicode xxx assume Unicode encoding in case insensitive mode
-xferbycopying optimize transfers using local copies (default true)
Configuration et tests
La configuration des répertoires et données à synchroniser doit se faire dans un répertoire situé sur le home de l'utilisateur : ~/.unison/, à créer manuellement. Ce répertoire peut contenir un ou plusieurs fichiers de configuration ("profil") qui se présentent sous cette forme :$ cat ~/.unison/paper.prf times = true owner = true group = true root = /home/kbux/ root = ssh://kbux@ktux1//home/kbux/ path = ./paper/ logfile = /home/kbux/.unison/paper.logRespectivement, nous avons :
- times : synchroniser les dates de modification
- owner : synchroniser le propriétaire des données
- group : synchroniser les permissions pour le group
- root (première instance) : répertoire source (avec un cran au dessus par rapport au répertoire contenant les données que nous voulons répliquer)
- root (seconde instance) : répertoire de destination, les 2 instances pouvant adresser soit en ssh, soit en local soit à travers des sockets
- path : chemin relatif où sont stockées nos données à répliquer, à partir du répertoire mentionné à la première instance de root
- logfile : fichier journal
Premier lancement
Du coup, si on lance unison sur un profil pareil, nous aurons :$ unison paper Contacting server... Connected [//ktux0//home/kbux/ -> //ktux1//home/kbux/] Looking for changes Warning: No archive files were found for these roots, whose canonical names are: /home/kbux/ //ktux1//home/kbux This can happen either because this is the first time you have synchronized these roots, or because you have upgraded Unison to a new version with a different archive format. Update detection may take a while on this run if the replicas are large. Unison will assume that the 'last synchronized state' of both replicas was completely empty. This means that any files that are different will be reported as conflicts, and any files that exist only on one replica will be judged as new and propagated to the other replica. If the two replicas are identical, then no changes will be reported. If you see this message repeatedly, it may be because one of your machines is getting its address from DHCP, which is causing its host name to change between synchronizations. See the documentation for the UNISONLOCALHOSTNAME environment variable for advice on how to correct this. Donations to the Unison project are gratefully accepted: http://www.cis.upenn.edu/~bcpierce/unison Press return to continue.[<spc>] Waiting for changes from server Reconciling changes local ktux1 file ----> paper/paper0 [f] file ----> paper/paper1 [f] file ----> paper/paper2 [f] file ----> paper/paper3 [f] Proceed with propagating updates? [] y Propagating updates UNISON 2.40.61 started propagating changes at 12:05:29.60 on 15 Mar 2011 [BGN] Copying paper/paper0 from /home/kbux to //ktux1//home/kbux [BGN] Copying paper/paper1 from /home/kbux to //ktux1//home/kbux [BGN] Copying paper/paper2 from /home/kbux to //ktux1//home/kbux [BGN] Copying paper/paper3 from /home/kbux to //ktux1//home/kbux Shortcut: copied /home/kbux/paper/paper3 from local file /home/kbux/paper/.unison.paper2.d2917910e40aeb61dcb98658569b3b8c.unison.tmp Shortcut: copied /home/kbux/paper/paper2 from local file /home/kbux/paper/.unison.paper1.d2917910e40aeb61dcb98658569b3b8c.unison.tmp Shortcut: copied /home/kbux/paper/paper1 from local file /home/kbux/paper/.unison.paper0.d2917910e40aeb61dcb98658569b3b8c.unison.tmp [END] Copying paper/paper3 [END] Copying paper/paper2 [END] Copying paper/paper1 [END] Copying paper/paper0 UNISON 2.40.61 finished propagating changes at 12:05:29.60 on 15 Mar 2011 Saving synchronizer state Synchronization complete at 12:05:29 (4 items transferred, 0 skipped, 0 failed) ktux1 $ ll /home/kbux/paper/ total 8 drwxr-xr-x 2 kbux kbux 4096 2011-03-15 12:05 . drwxrwxrwt 38 root root 4096 2011-03-15 12:01 .. -rw-r--r-- 1 kbux kbux 0 2011-03-15 11:59 paper0 -rw-r--r-- 1 kbux kbux 0 2011-03-15 11:59 paper1 -rw-r--r-- 1 kbux kbux 0 2011-03-15 11:59 paper2 -rw-r--r-- 1 kbux kbux 0 2011-03-15 11:59 paper3Au second lancement, si rien ne bouge :
$ unison paper Contacting server... Connected [//ktux0//home/kbux -> //ktux1//home/kbux] Looking for changes Waiting for changes from server Reconciling changes Nothing to do: replicas have not changed since last sync.
Modifications de données sur l'un des réplicas
Suppression
Si l'on supprime le contenu de notre /home/kbux/paper/ sur la machine de destination ktux1, par défaut et sans stipuler l'inverse, nous allons directement à la réplication des données modifiées sur celles antérieures : donc suppression pure et simple du contenu de /home/kbux/paper/ sur ktux0. Pour information, on peut -et heureusement- spécifier la décision à l'énumération de chaque donnée modifiée, en lieu et place du [f].$ unison paper Contacting server... Connected [//ktux0//home/kbux -> //ktux1//home/kbux] Looking for changes Waiting for changes from server Reconciling changes The following paths have been completely emptied in one replica: 'paper' Unison may delete everything below these paths in the other replica. (Set the 'confirmbigdel' preference to false to disable this check.) Do you really want to proceed? [] y local ktux1 <---- deleted paper/paper0 [f] <---- deleted paper/paper1 [f] <---- deleted paper/paper2 [f] <---- deleted paper/paper3 [f] Proceed with propagating updates? [] y Propagating updates UNISON 2.40.61 started propagating changes at 12:13:20.04 on 15 Mar 2011 [BGN] Deleting paper/paper0 from /home/kbux [BGN] Deleting paper/paper1 from /home/kbux [BGN] Deleting paper/paper2 from /home/kbux [BGN] Deleting paper/paper3 from /home/kbux [END] Deleting paper/paper0 [END] Deleting paper/paper1 [END] Deleting paper/paper2 [END] Deleting paper/paper3 UNISON 2.40.61 finished propagating changes at 12:13:20.04 on 15 Mar 2011 Saving synchronizer state Synchronization complete at 12:13:20 (4 items transferred, 0 skipped, 0 failed)
Rajout
Reprenons avec notre /home/kbux/paper plein sur ktux0 (la source) et après avoir lancé notre mirroring sur ktux1 (la destination). Nous avons donc ktux0:/home/kbux/paper strictement égal à ktux1:/home/kbux/paper. On crée un fchier sur ktux1 : /home/kbux/paper/paper4.ktux1 $ touch /home/kbux/paper/paper4 ktux0 $ unison paper Contacting server... Connected [//ktux0//home/kbux -> //ktux1//home/kbux] Looking for changes Waiting for changes from server Reconciling changes local ktux1 <---- new file paper/paper4 [f] Proceed with propagating updates? [] y Propagating updates UNISON 2.40.61 started propagating changes at 12:17:15.79 on 15 Mar 2011 [BGN] Copying paper/paper4 from //ktux1//home/kbux to /home/kbux Shortcut: copied /home/kbux/paper/paper4 from local file /home/kbux/paper/paper3 [END] Copying paper/paper4 UNISON 2.40.61 finished propagating changes at 12:17:15.80 on 15 Mar 2011 Saving synchronizer state Synchronization complete at 12:17:15 (1 item transferred, 0 skipped, 0 failed)Et donc, sur ktux0, qui n'avait jusque là pas eu de paper4 sur /home/kbux/paper :
$ ll /home/kbux/paper/paper4 -rw-r--r-- 1 kbux kbux 0 2011-03-15 12:17 /home/kbux/paper/paper4Si on tend un peu l'oreille, on s'aperçoit qu'il s'agit en réalité d'utiliser ssh pour ce cas-ci. Il existe 3 manières d'ouvrir une connexion : local, SSH et socket. Du coup, rien n'empêche d'adresser, au hasard, un montage en CIFS (plus connu sous le nom de SMB).
Modification
Lors de mise à jour des données, mieux vaut ne pas se tromper, car cela risque de laisser des souvenirs déplaisants. Si l'on veut bien se rendre compte de ce que l'on est sur le point de transférer, on peut utiliser l'option diff pour déjà donner la commande qui nous servira à montrer les différences entre les réplicas. Meilleure visibilité donc meilleure anticipation sur les choix à faire. Au passage, un petit tour sur l'aide qui permet notamment de spécifier le sens de réplication (qui vers qui).$ unison paper -diff diff Contacting server... Connected [//ktux0//home/kbux -> //ktux1//home/kbux] Looking for changes Waiting for changes from server Reconciling changes local ktux1 <---- changed paper/paper0 [f] ? Commands: <ret> or f or <spc> follow unison's recommendation (if any) I ignore this path permanently E permanently ignore files with this extension N permanently ignore paths ending with this name m merge the versions d show differences x show details L list all suggested changes tersely l list all suggested changes with details p or b go back to previous item g proceed immediately to propagating changes q exit unison without propagating any changes / skip > or . propagate from from local to ktux1 < or , propagate from from ktux1 to local <---- changed paper/paper0 [f] d diff '/home/kbux/paper/paper0' '/home/kbux/paper/paper0.unison.diff-' 0a1 > paper0Comme je vous l'ai dit un peu plus haut, il est possible également de stipuler dans quel sens les modifications sont à faire, par exemple en interactive, en précisant > (src to dest) ou < (dest to src).
