Re: [Monotone-devel] announcing preliminary "dumb server" support for monotone

[Nathaniel Smith]

On Wed, Oct 12, 2005 at 09:34:26PM +0200, Zbynek Winkler wrote:
> Hmm. I quite didn't get the picture until I tried it ;). I didn't have 
> the patience to wait for the local do_export to finish on the monotone 
> database... But the speed seems to be (unfortunately) comparable to the 
> verification of the incomming changesets when doing regular pull. BTW: 

No, different problem entirely -- do_export is currently quadratic in
the history length, mostly because it uses a separate invocation of
monotone to request each manifest delta, and since monotone still does
unbounded delta chaining, it takes linear time to retrieve an
arbitrary manifest.  (This also applies to files, but files tend to
have much shallower histories than the tree as a whole, so it doesn't
matter there as much.)

One thing that would help a lot would be to move the packet commands
to automate (where they probably should be anyway), and then teaching
monotone.py to use 'automate stdio'.  That way we're using a
persistent monotone process, and the db layer's internal caching
should be able to turn this back into a linear operation.

> What is the limiting factor of the verification step? Does it do some 
> sorting?

No, it's doing a bunch of really torturous checks of different sorts
of data inconsistencies each revision might have.  "Torturous" because
our data structures were not well chosen (because when we were first
inventing this stuff, we didn't know as much as we do now).  The
rosters code replaces this stuff entirely, and shouldn't suffer from
the same problems.  (Instead, it will suffer from new, different
problems!  Hopefully less severe, though :-).)

> >it can't push to FTP yet, not a
> >proper python package.  Oh, and I just noticed that it doesn't include
> >pubkeys in the data to synchronize, oops.
> > 
> >
> Could it have something to do with the fact that there seems to be no 
> 'automate keys' anymore? It prints the help message instead... I tried 
> to use 'list keys' instead. BTW why does the output differ from the 
> other 'list *'? Anyway, this could help:
> 
> in monotone.py:
>    def key_names(self):
>        ids = []
>        for line in self.run_monotone(["list", "keys"]).split("\n"):
>            if line == "" or line == "[public keys]":
>                continue
>            if line == "[private keys]":
>                break
>            ids.append(line.split(" ")[1])
>        return ids

...actually, the problem is there isn't an 'automate keys' _yet_ :-).
'automate keys' exists only in mainline, not in 0.23.  I think I
actually added basically the code you mention to .dumb, and then Matt
switched it over to the new, better 'automate keys' stuff last night
or so?  But I might be misremembering or have misunderstood exactly
what's going on, I'm trying to focus on the rosters branch right now,
since there seem to be lots of other smart people interested in
working on things like this :-).

> >OTOH, it supports monotone's full sync semantics (multiple people can
> >push to the same "repo", you can have backup "repos" and sync with
> >them indiscriminately, etc.), and should be reasonably efficient (it
> >uses merkle tries to do low-overhead set synchronization).  Won't be
> >as fast as netsync, or as flexible (whoever puts up the repo gets to
> >choose what branches are included, you don't get to pick on the fly
> >like for netsync), but might be handy for some people...
> > 
> How do you pick what branches are included? Doesn't it always export the 
> whole database? I was surprised to find out that it does not 
> differentiate between files, changesets etc.

It always exports the whole database.  This could be made smarter, I
guess, but it certainly didn't seem worth the effort for the first
pass.  One of many possible improvements for someone to make, if they
want :-).

-- Nathaniel

-- 
.i dei jitfa fanmo xatra