[Maposmatic-dev] Recent problems on MapOSMatic production instance

[Maxime Petazzoni]

Hi,

Since the deployment of the new MapOSMatic rendering daemon on the
production instance of MapOSmatic at www.maposmatic.org a few
bugs/problems slipped out, most importantly:

  * empty CSV indexes output
  * timing out jobs not really timing out and stalling the queue

This email is to inform the rest of the team and MapOSMatic users of the
state of these problems.

Firstly, the empty CSV index output problem is fixed. The problem came
from a missing datetime import in ocitysmap/street_index.py after the
change to a dynamic copyright notice year.

The other issue is a bit more problematic and is inherent to the fact
that we run GRSecurity on the MapOSMatic server, and somehow it's
preventing the thread kill exception to be raised by the rendering
daemon to kill its rendering worker thread.

I put a small warkaround in place in the prod-instance branch (so local
to the production instance setup) to respawn the daemon over itself
instead of having the rendering thread continuing after the timeout
occurred. Unfortunately I don't know enough about grsec to determine why
it's doing that, and how to work around it more cleanly. My main focus
here was to have the production rendering pipeline to work as excepted
and not get stuck as it did a few times in the past few days.

Finally, note that I've rescheduled the last ~15 jobs for rendering so
they have a correct CSV index, so the queue will be busy for a little
while while it chews through these few jobs.

- Maxime

-- 
Maxime Petazzoni <http://www.bulix.org>
 ``One by one, the penguins took away my sanity.''
Linux kernel and software developer at MontaVista Software

signature.asc
Description: Digital signature