Results 1 to 3 of 3

Thread: sshfs + rdiff-backup

  1. #1
    Join Date
    Jun 2008
    Posts
    232

    Default sshfs + rdiff-backup

    I'm trying to use rdiff-backup on an sshfs mounted remote filesystem.

    All is good until rdiff-backup hits a symbolic link, at which time it goes pear-shaped.

    After a day of trying various solutions, I've just about given up and was wondering if anyone here is using it successfully.

    Basically, I'm mounting the remote dir locally like so:
    Code:
    sshfs root@remote.dom:/remote/dir /mnt/localmount -o workaround=rename,port=1234
    Then run rdiff-backup as such:
    Code:
    rdiff-backup -v5 --no-hard-links --preserve-numerical-ids --print-statistics --exclude-filelist excludelist.txt /local/backupdir /mnt/localmount/destbkpdir/
    rdiff-backup runs fine until the first symlink, at which time it throws this error:
    Code:
    Exception '[Errno 2] No such file or directory: '/mnt/localmount/destbkpdir/testdir/rdiff-backup.tmp.7'' raised of class '<type 'exceptions.OSError'>':
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 302, in error_check_Main
        try: Main(arglist)
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 322, in Main
        take_action(rps)
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 278, in take_action
        elif action == "backup": Backup(rps[0], rps[1])
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 344, in Backup
        backup.Mirror(rpin, rpout)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 38, in Mirror
        DestS.patch(dest_rpath, source_diffiter)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 225, in patch
        ITR(diff.index, diff)
      File "/var/lib/python-support/python2.5/rdiff_backup/rorpiter.py", line 281, in __call__
        last_branch.fast_process(*args)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 522, in fast_process
        if self.patch_to_temp(mirror_rp, diff_rorp, tf):
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 543, in patch_to_temp
        result = self.patch_snapshot_to_temp(diff_rorp, new)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 568, in patch_snapshot_to_temp
        rpath.copy_attribs(diff_rorp, new)
      File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 176, in copy_attribs
        rpout.chown(*rpout.conn.user_group.map_rpath(rpin))
      File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 884, in chown
        try: self.conn.C.lchown(self.path, uid, gid)
    
    Traceback (most recent call last):
      File "/usr/bin/rdiff-backup", line 23, in <module>
        rdiff_backup.Main.error_check_Main(sys.argv[1:])
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 302, in error_check_Main
        try: Main(arglist)
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 322, in Main
        take_action(rps)
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 278, in take_action
        elif action == "backup": Backup(rps[0], rps[1])
      File "/var/lib/python-support/python2.5/rdiff_backup/Main.py", line 344, in Backup
        backup.Mirror(rpin, rpout)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 38, in Mirror
        DestS.patch(dest_rpath, source_diffiter)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 225, in patch
        ITR(diff.index, diff)
      File "/var/lib/python-support/python2.5/rdiff_backup/rorpiter.py", line 281, in __call__
        last_branch.fast_process(*args)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 522, in fast_process
        if self.patch_to_temp(mirror_rp, diff_rorp, tf):
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 543, in patch_to_temp
        result = self.patch_snapshot_to_temp(diff_rorp, new)
      File "/var/lib/python-support/python2.5/rdiff_backup/backup.py", line 568, in patch_snapshot_to_temp
        rpath.copy_attribs(diff_rorp, new)
      File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 176, in copy_attribs
        rpout.chown(*rpout.conn.user_group.map_rpath(rpin))
      File "/var/lib/python-support/python2.5/rdiff_backup/rpath.py", line 884, in chown
        try: self.conn.C.lchown(self.path, uid, gid)
    OSError: [Errno 2] No such file or directory: '/mnt/localmount/destbkpdir/testdir/rdiff-backup.tmp.7'
    I've tried combinations of various sshfs options, including:
    Code:
    readdir_ino
    transform_symlinks
    follow_symlinks (this borks rdiff-backup really early)
    The only thing that works is to use --exclude-symbolic-links option for rdiff-backup, but then of course there's no symbolic links processed.

    What's most annoying is that I can't find anyone with the same reported problem, so I guess it's just me.

    I'm using ubuntu 8.04.2 (both ends) which ships rdiff-backup v 1.1.15 by default, but I downloaded and installed versions 1.2.8 and 1.3.3 with no improved results.

    Running the rdiff-backup without the sshfs remote mount works fine, so the problem isn't likely in rdiff-backup.

    Any sshfs or rdiff-backup gurus hanging out in here?

  2. #2
    Join Date
    Dec 2007
    Posts
    1,141

    Default

    The exceptions.OSError being thrown by Python would suggest that there is an issue creating or changing directories (as noted by a user on this rdiff-backup error thread).

    This rdiff-backup exception thread at BackupCentral.com suggests an issue with symbolic link support across filesystems, which may be the underlying cause.

    Have you tried testing with a hard-linked file?

  3. #3
    Join Date
    Jun 2008
    Posts
    232

    Default

    Quote Originally Posted by DanL@VPSLink View Post
    Have you tried testing with a hard-linked file?
    No, the docs I read so far regarding all this seem to indicate that hardlinks are not supported, hence the --no-hard-links option passed to rdiff-backup. As I understand it, when using that option, the hardlinks are not created in the backup set. Instead, a copy of the file is made and the meta data stores the hardlink details and restores them upon restoring from the backup set. Well, that's the way I understood it.

    On the problem of the symlinks, I did a lot of stuff (trying, reading, digging) and not all of it is well remembered.
    What I did find was that the exception was thrown when processing a symlink that was created before the file it referenced. A 'broken' symlink, if you will. In the exception being thrown, the most recent call was a chown, so I inferred that this was the problem and trying to chown a broken symlink, it did throw an error. However, if the file to which the symlink points exists, then the chown succeeds. (I think so anyway - it was a long day )

    So the secret seems to be, create the symlinks last, after all the files are created, but I couldn't find such an option in rdiff-backup.

    I even tried to fake it, by running the rdiff-backup once with the --exclude-symbolic-links option (which is why I know that works) and then a second time without it, hoping that the symlink data would then be factored in. But that didn't work...

    But thanks for the links, I'll have a look at them and see if I can glean anything further from them.

    I will also try it on a non-vps test bench, just in case that has something to do with it. I haven't ruled out the possibility that it might be a problem with the fuse module either. I'm just surprised that no-one else has documented this type of experience. Being unique with a problem is a rare thing since the advent of google.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •