Re: file storage in IMAP (eventually for Tramp) working and needs testing

[Michael Albinus]

Ted Zlatanov <address@hidden> writes:

> Say you have these article UIDs and subjects, and UIDs are ordered by
> date.  Note the ENCODED_FILENAME will be the filename encoded in some
> way and /a/b/c is the filename without any risky characters; Gnus has
> the subject encoding functionality already.  The subject format is
> specifically to allow easy searching.
>
> 1 "tramp-imap-marker /a/b/c ENCODED_ABC"
> 2 "tramp-imap-marker /a/b/c ENCODED_ABC"
> 3 "tramp-imap-marker /a/b/c ENCODED_ABC"
> 4 "hello world"
>
> Do we 
>
> A) allow for three identically named files, dynamically renaming 2 and 3
> with the Emacs <1> and <2> convention?
>
> B) ignore 1 and 2, never create them intentionally, and (optionally)
> erase duplicates when a file is saved?  This is probably easiest to
> implement in code.
>
> C) treat 1 and 2 as backups (there will be a tramp-imap-backups
> parameter to control how many are allowed per filename, and older ones
> over that limit are erased when a file is saved)?  We'll need a
> mechanism to revert to a backup, which I don't think Tramp has built-in
> at the moment.

If 1, 2 and 3 are created only by tramp-imap, then C might be the
preferred choice. B doesn't look good to me, I believe all files must
be accessible.

> Second question: is message 4 ignored?  I would prefer to do so, to
> allow coexistence of tramp-imap.el with other messages (or even later
> versions of tramp-imap.el).

Maybe. But then tramp-imap would only access files it has created
itself, this might be a restriction.

Of course it wouldn't be good if tramp-imap would change subject lines
of existing messages. Do we need to encode the filename in the subject
line? Couldn't it be any other header, like X-Tramp-Filename?

> Third question: namespaces.  I feel that it's much better
> for the user to store all the files in a single mailbox:

[...]

Honestly, I'm not so familar with IMAP conventions. I have no
precedence for this, you might decide yourself.

> Here's the list (omitting the commented-out items from tramp-smb.el)
> with some questions and comments.  Sorry if most of this is obvious,
> this is my first time diving into the Tramp code.  What function would
> handle the truename-to-visualname translation?

There is an "atomic" function tramp-smb-send-command, which makes the
communication with the server. Similar functions are
tramp-send-command and tramp-send-fish-command in their respective
packages. I guess you'll need a tramp-imap-send-command or so; this
could be a central place for the mapping. Other places might be needed
as well when *interpreting* the result of a sent command.

[...]

> add-name-to-file: could be a special "link message" or just a copy, like in 
> Windows

Both is possible. Btw, on my todo is implementing a real link for
Windows, but this has no priority.

> copy-file: implemented as an APPEND
> delete-directory: implemented with a search+delete for all matching messages

OK.

> delete-file: search+delete of all matching messages

It depends how we decide with your first question. If there is
uniqueness in mapping file names to true names, than there is only one
message to be deleted.

> directory-file-name: tramp-handle-directory-file-name?
> directory-files: search for matches

OK.
> directory-files-and-attributes: search for matches, attributes always 777

It returns more than just file permissions.

> dired-call-process: ignore

OK.

> dired-compress-file: ignore

Likely yes, but there might be an implementation depending on how a
message is stored.

> file-accessible-directory-p: always t

No. Use default implementation (which checks both file-directory-p and
file-executable-p).

> file-attributes: always 777

No. file-attributes returns a list of 12 different attributes. Must be
implemented.

> file-directory-p: needs a search, but we could have a file name that 
> conflicts with a directory name

We shall avoid such conflicts. A file name shall be unique.

> file-executable-p: always nil

Better is t (see file-accessible-directory-p). It doesn't hurt,
because we don't support processes.

> file-exists-p: needs a search

OK.

> file-local-copy: ?

That is the basic function for retrieving a file from the server. It
stores the file in the temp directory; some other functions use it
then (like insert-file-contents etc).

> file-remote-p: tramp-handle-file-remote-p

OK

> file-modes: tramp-handle-file-modes or hard-code

Both is possible. tramp-handle-file-modes is shorter to write.

> file-name-all-completions: needs a search
> file-name-completion: tramp-handle-file-name-completion
> file-name-directory: tramp-handle-file-name-directory
> file-name-nondirectory: tramp-handle-file-name-nondirectory
> file-newer-than-file-p: needs a search, plus backups may matter
> file-ownership-preserved-p: ignore

OK

> file-readable-p: always t
> file-regular-p: always t

No. You must return nil when the file does not exist.

> file-symlink-p: always nil
> file-truename: returns UID

OK

> file-writable-p: always t

No, again the case of non existing file names.

> find-backup-file-name: we need to decide
> insert-directory: ?

Both can be postponed, they are not basic functions.

> insert-file-contents: search+retrieve

file-local-copy and insert-file-contents of that local copy.

> load: tramp-handle-load
> make-directory, make-directory-internal: we need to decide what this should do
> make-symbolic-link: ignore or implement as a special message
> rename-file: append with new subject, delete original and its backups if
> we decide to do backups
> set-file-modes: ignore
> set-visited-file-modtime: we could use a header for this

OK

> shell-command: ignore?

Yes

> substitute-in-file-name: ?

tramp-handle-substitute-in-file-name

> unhandled-file-name-directory: tramp-handle-unhandled-file-name-directory?
> vc-registered: always nil

OK

> verify-visited-file-modtime: ?

Maybe ignore for the time being, it's not core functionality.

> write-region: needs to do an append+delete of original+backups as needed; 
> IMAP can't rewrite a message

OK



I would start with file-local-copy and write-region, these are the
functions for retrieve and store a file.

> Thanks
> Ted

Best regards, Michael.