Re: [Monotone-devel] monotone server hangs

[Zack Weinberg]

[please keep address@hidden in the cc:, there are several
people reading that list who know more about netsync than I do.]

On Wed, Sep 23, 2009 at 2:15 PM, Hugo Cornelis <address@hidden> wrote:
> Yes it hangs.  Sometimes the transfer completes in a couple of
> minutes, but most of the time it takes an indefinite amount of time
> for exactly the same transfer.  Sometimes it hangs after importing 50
> revisions, sometimes after 120, sometimes it hangs from the early
> start.  It seems fairly undeterministic where it hangs, so my first
> guess is that it is a race condition of some sort.
...
> What can I do to help you guys debug this problem?  strace? ltrace?
> netstat?  Reconfigure the database?

If it's blocking other clients, that means it's not making it back to
the select() loop in the server, which suggests a computational bug.
You're the first person to report an actual hang rather than just "OMG
so slow", so I suspect it depends on the exact contents of your
database.

I can think of four things that would be useful:

- Try enabling forward deltas; see if that makes the problem go away.
- When the hang happens, is the server consuming 100% CPU, blocked on
a system call, or what?
- Build a mtn binary with debugging information; run it on the server
under gdb; when it hangs, hit ^C to break into the debugger, and get a
backtrace.  (If it's consuming 100% CPU, please capture *several*
backtraces, continuing execution in between and waiting for a few
seconds before hitting ^C again, to probe the bounds of the infinite
loop.)
- Put a copy of your database somewhere we can retrieve it via FTP or
HTTP, so we can attempt to reproduce the problem on our own machines.
(I don't want to try to pull from you in case I trigger the bug!)

zw