Dealing with thread storms

[Sergio Lopez]

Hi all,

Since it has been proved that throttling thread creation is not the
solution (not even a proper workaround) to prevent thread storms, I'd
like to discuss some strategies to deal with them.

To my knowledge, thread storms are usually caused by GNU Mach
continuously sending m_o_data_return request to a server, due to a
m_o_lock_request (called by libpager for syncing operations) or by
the kernel memory laundry process.

For the kernel, searching for dirty pages and sending m_o_data_return
requests is a cheap operation, but for a server like ext2fs, dealing
with that kind of request involves a considerable amount of work,
including I/O operations.

So I'm thinking about the following changes:

  - Implement a queue in libdiskfs to deal with m_o_data_return
    requests and a static amount of consumers which will do the real
    work. Each time the kernel sends us a m_o_data_return, we just add
    the request to the queue and return the thread to the pool.

  - Extend libpager interface to be able to write more than one page at
    once, and progressively change the servers to make use this.

  - Whenever is possible, send more than one page a time. Right now,
    m_o_lock_request tries to send up to 32 continuous pages, but vm
    pageout process sends them one by one. OSF Mach implements
    multipage discipline both for pageout and pagefault operations, so
    it should be possible to do the same in GNU Mach (I've made some
    work in this direction some time ago).