|
| From: | Zbynek Winkler |
| Subject: | Re: [Monotone-devel] announcing preliminary "dumb server" support for monotone |
| Date: | Fri, 14 Oct 2005 21:48:18 +0200 |
| User-agent: | Debian Thunderbird 1.0.2 (X11/20050602) |
Nathaniel Smith wrote:
On Fri, Oct 14, 2005 at 04:37:04PM +0200, Zbynek Winkler wrote:Yeah, I've noticed that too. I suppose the trees are rebuilt for each push/pull because different branches can be included in it every time, right? Would it make sense to precompute the merkle trees for each branch and store it in the db? It could get updated in a lazy manner - when adding new things to the branch, delete the hash indexes that would need to be updated; on the next sync rebuild the all the missing hash indexes...Maybe. There isn't a separate merkle trie for each branch; there's a single merkle trie for a _set_ of branches.
I know. I was suggesting it as a possible improvement.
How can A and B be in different branches? I thought something like that could happen only in case of a propagate, when B would actually have two parents, each in different branch... Do you mean that every time there is some propagate between branches it is not possible to upload just one of them any more?(Or something like that -- there's some subtlety where if we have A -> B, where A and B are in a separate branches, any merkle trie that includes B will still include A, to make sure the receiving db stays consistent.)
I agree that the relation might be not very obvious :-). The the thing is that monotone-dumb shows a way how to experiment with different synchronization methods without changing monotone itself. What I am thinking is that monotone-dumb is just one of possible synchronization methods. The cached tries could be another one. Maybe if it proves itself, it could later be integrated.In the normal case, the user always syncs the same set, so this kind of caching might be a win. It's not really related to monotone-dumb, though :-).
I tried to explain myself better above. Actually, the per-branch tries seem to you to be orthogonal to what exactly? Maybe I am missing something :-)Yes, that would. I might give it a shot... How hard would it be to implement something like the above (including the precomputed per-branch merkle trees) outside of monotone first, let's say as a python wrapper? When adding stuff to a branch it would invalidate part of the stored cache and recompute it on next sync. One merkle dir would store only one branch...?However, this doesn't actually solve any of the problems that merkle_dir.py solves, since all its heavy lifting has to do with the other end -- how do you deal with a simple remote filesystem to make it possible to efficiently push and pull. It isn't trivial to integrate this with something like the above 'automate merkle_hash' idea, because then you'd have to make sure you could efficiently calculate hashes _in the same way monotone does_, and that takes a bit of thought. Oh, however however, you're actually quite right -- you could do something very conceptually simple in just monotone-dumb, which is implement a class that acts like a MerkleDir, but that is constructed in memory directly off a monotone database. Basically, you'd iterate over the db like do_export does, but instead of actually fetching stuff, you could just keep track of which ids exist, build HASHES files in memory, and use them to sync. Then you pull stuff out of the db as necessary, when it turns out you want to send it to the remote side. That'd be neat.The precomputed per-branch merkle tries seems totally orthogonal to me; maybe I'm missing something.
Zbynek -- http://zw.matfyz.cz/ http://robotika.cz/ Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic
| [Prev in Thread] | Current Thread | [Next in Thread] |