Re: [Monotone-devel] Re: ANNOUNCE: monotone 0.17

[Nathaniel Smith]

On Mon, Mar 21, 2005 at 01:15:39PM +0000, Catalin Marinas wrote:
> Nathaniel Smith <address@hidden> wrote:
> > 0.17 is now available from http://venge.net/monotone/ , together with
> > info on what's new and upgrade instructions.
> 
> I posted this e-mail about a week ago but, for good reasons, there was
> no reply. I will try again below:

Sorry I missed it :-)

> >   # if you're using 0.16:
> >   $ monotone-0.16 --db=foo.db db dump | monotone-0.17 --db=test.db db load
> >   # if you've been tracking mainline and already switched to sqlite3:
> >   $ cp foo.db test.db
> >   # everyone:
> >   $ monotone-0.17 --db=test.db db migrate
> 
> The above steps took couple of minutes each. I have a quite big
> database (172MB) with 3366 check-ins. I try to import the BKCVS
> revisions since Linux 2.6.9. After the dump/load conversion, the new
> database is about 120MB.
> 
> >   $ monotone-0.17 --db=test.db db rebuild
> 
> This one is still running after about 18 hours, using 100% CPU. It
> still updates the output from time to time. Do you have any estimation
> about how long it will take (i.e. few more hours or weeks)?  My
> machine is a P4 at 2.5GHz.

Hmm, no, I don't have a good feel.  As I said, the monotone database
took something on the magnitude of 3 hours, and that was doing about
1150 revisions.  (The time is dependent not on the size of the
database, but on the number of revisions, and the number of files in
each manifest... you probably have much bigger manifests.)

> $ time monotone-0.17 --db=linux-test-0.17.db db rebuild
> monotone: rebuilding revision graph from existing graph
> monotone: [certs in: 13470] [certs out: 0] [nodes: 3366] [revs out: 0]
> monotone: scanning for nodes with 3+ parents
> monotone: rebuilding 3366 nodes
> monotone: [certs in: 13470] [certs out: 0] [nodes: 3366] [revs out: 1]
> monotone: verifying new revisions (this may take a while)
> monotone: [certs in: 13470] [certs out: 0] [nodes: 3366] [revs out: 23]
> 
> Should the "revs out" above equal the number of nodes (I had to stop
> it at this stage though)? With this speed, it might not complete the
> db rebuild this year :-)

Yes, it's done when "revs out" = "nodes".  So... yeah, sounds like you
do have a problem :-/.  There are a few things you can do:
  - re-importing from CVS may be faster than rebuilding (I'm not
    completely sure, but I _think_ it should be)
  - wait for us to optimize monotone more (or help yourself)
    Matt already made a change on mainline that he says gives him a
    25% speedup on the checking 'pull' does (which is very similar);
    it may have also removed an algorithm that was quadratic in the
    size of the manifest.  Profiling continues...
You could also hack a version of monotone to not be so careful about
checking things, but I don't recommend it overly -- the danger is
that there's some bug we don't know about, that would then be allowed
to silently corrupt your history and leave you needing to do _another_
rebuild later when it was finally discovered...

-- Nathaniel

-- 
"But in Middle-earth, the distinct accusative case disappeared from
the speech of the Noldor (such things happen when you are busy
fighting Orcs, Balrogs, and Dragons)."