Libferris support in TRAMP

[Ben Martin]

Hi,
  I'm the author of the libferris virtual filesystem [1]. I've been
meaning to add libferris support to TRAMP for many years, unfortunately
it has been keeping many items on the TODO list company and didn't get
any time until recently.

  The attached patch is certainly not for applying, but is attached in
the interest of raising comments.

  I was originally poking around the lisp sources for the various things
that TRAMP can use as a virtual filesystem when a fairly clean design
hit me. Instead of bringing TRAMP to libferris, why not do the opposite?

  So I started digging into the main tramp.el file and tracing ssh
connections to see what commands were executed in the background. I
already have replacements for many of the normal coreutils programs:
fcat, ferrisls, ferriscp, ferris-redirect etc, so the plan was to try to
get TRAMP to just use the libferris implementation of these commands.

  The plan then became to add any of the POSIX commands that tramp
wanted which libferris didn't already have clones of. It turns out there
weren't too many, mainly ftest and fstat. I need to do something with
fchown and fchmod too, I have one of those reimplemented already.

  The twist of course for these "ferris" versions of the normal commands
is that much of the "stat" data is not available for some libferris
filesystems. For example, libferris can mount XML and berkeley db files,
and an element in an XML file might not have a proper octal access mask,
or indeed any explicit user/group information. So my fstat command has
options to produce something valid in the face of missing data from the
VFS.

  I currently have load/save working for subelements in XML files, and
keys in an ISAM file like a berkeley db4. The patch to tramp itself is
around 10-15 lines. Much of this can be abstracted out so you can
nominate to use "the libferris command" instead of the normal coreutils
one. eg, using fstat instead of stat. The other way to go would be to
allow the shell to setup these aliases so tramp code changes even less. 

  The only two areas where this would not work cleanly is the base64
inline transmission. For example, encoding
(format "%s < %s" 
  rem-enc 
  (tramp-shell-quote-argument localname))
doesn't work for libferris because the shell redirection doesn't know
how to handle my filesystems. The work around is simple in this case,
just cat the file to the base64 encoder:
(format "fcat %s | %s" 
  (tramp-shell-quote-argument localname)
  rem-enc 
  )
A slight kink presents itself for decoding because of the here document.
The old:
(format "%s > %s <<'EOF'\n%sEOF"...
becomes a shell function of its own in order to pipe a here document to
an explicit redirection command:
(format "shovetofile() { cat - | %s | ferris-redirect --trunc %s; }\n
shovetofile <<'EOF'\n%sEOF"

Those two abstractions are the main substantial code changes to
tramp.el. I imagine the clean thing here is to turn them both into lisp
functions that can be redefined when using libferris.

I still haven't converted over chown/chmod and there are some rough
edges, but it works for load/save during my initial testing on remote
XML and db4 files. Oh, and I haven't done anything about porting
filename completion as yet. Or indeed leaving things "normal" and having
the way to turn on libferris is desired. Hopefully I'll have a patch
aimed more toward application than discussion soon. Sorry about my poor
elisp skills.

As a small usage example, on the remote server;

$ fcreate --create-type=db4 --rdn=isam.db .
$ date | ferris-redirect -T isam.db/subfile1
$ db_dump -p isam.db
VERSION=3
format=print
type=btree
db_pagesize=4096
HEADER=END
subfile1
Fri Feb 26 18:24:55 EST 2010\0a
DATA=END

And to edit the subfile1 inside that berkeley db file over the network:
$ emacs "/address@hidden:/tmp/isam.db/subfile1"

For performance, it is handy to play with version control and how backup
files are made. The later is actually quite important for editing part
of an XML file, because you can't have "elementname~" as a valid element
in an XML document. 

The below keeps things semantically closer, but having the backups on
the local machine is even faster.
(setq backup-directory-alist 
'(("^/[a-z]*:address@hidden:.*" 
. "/address@hidden:/tmp/.saves")))

[1] http://www.libferris.com/

tramp-2.1.17-ferris.patch
Description: Text Data

signature.asc
Description: This is a digitally signed message part