Re: [Maposmatic-dev] Outstanding points before releasing MapOSMatic

[Thomas Petazzoni]

Hello,

Le Tue, 10 Apr 2012 23:45:23 +0200,
Thomas Petazzoni <address@hidden> a écrit :

>  * Many translations are still out of date, though I will not consider
>    this as a blocker for the release. See attached file for the
> current status of the translations.

We haven't progressed a lot here. Basically, the status is that the
french translation, the brasilian portuguese translation are ok. The
german translation is almost ok, the italian translation is almost ok,
the dutch translation is almost ok, the russian translation is almost
ok. The other translations are out of date.

>  * Gaël patches on the scale problem has also changed significantly
> the scale of booklets. The city of Chavagne required sixteen A4
> portrait pages before Gaël patches (just for the divided map) and now
>    requires 28 pages (just for the divided map) after Gaël patches. I
>    think the previous version with 16 pages was readable, so I guess
> we should change the scale used for booklets to something more
>    reasonable.
> 
>    See http://dev.maposmatic.org/smedia/chavagne.pdf (before) and
>    
> http://dev.maposmatic.org/rendered/000472_2012-04-10_23-18_ChavagneRennesIlleetVilaineBrittany35310France.pdf
>    (after).
> 
>    Patches welcome.

I have integrated David Mentré proposal, so this problem is fixed.

>  * Renderings of Paris in booklet mode is timeout-ing on the main
>    server, taking more than 20 minutes to complete. We could solve
> this by further increasing the accepted maximum rendering duration up
> to 30 or 40 minutes.

I tried again today, and after 9 minutes, the rendering failed because
the rendering daemon could not allocate enough memory. Interestingly, I
have been able to render Paris on my laptop, which also has 4 GB of
RAM, exactly like the server, and both run a 64 bits system.

I think we will have to leave with a first production version that is
not capable of rendering such huge cities, and we'll see later how to
improve that.

>  * The automated updates of the planet database have stopped, and we
>    don't know how to start them again. We have hit the period during
>    which the OSM database was read-only and things seem to have
> changed with regard to the upgrade process. So our replication lag is
>    growing again, and we don't know how to fix it, see
>    http://stats.maposmatic.org/r2d2/browse/gcc20.fsffrance.org/replication/.
>    Maxime and I will try to send an e-mail to the OSM community
>    tomorrow to have some feedback on this.

This problem is fixed, even if we know that we will have to do a new
planet import rather soon.

>  * The MapQuest stylesheets on the server are broken. The rendering
>    fails with:
> 
>   File
> "/home/maposmatic/maposmatic-dev/ocitysmap/ocitysmap2/maplib/map_canvas.py",
> line 100, in __init__ mapnik.load_map(self._map, stylesheet.path)
> RuntimeError: The following nodes or attributes were not processed
> while parsing the xml file:
> * node 'MaxScaleDenominator' in line 0
> * node 'MaxScaleDenominator' in line 0
> * node 'MaxScaleDenominator' in line 0
> * node 'MaxScaleDenominator' in line 0
> * node 'MaxScaleDenominator' in line 0
> * node 'MaxScaleDenominator' in line 0

This problem is fixed thanks to David Mentré.

>  * We have a regular Out-of-memory issue on the rendering server, but
>    unfortunately, the kernel seems to be unable to kill the
>    memory-hungry process and simply crashes the VM. This is the reason
>    for the last 2/3 outages of the maposmatic.org site. A potential
>    solution would be to start the rendering daemon with a fixed ulimit
>    so that the process gets killed when it consumes too much memory.

No progress has been made here. We have asked the sysadmin of our
rendering virtual machine for details on the date and time of the
crashes to correlate that with the rendering that triggered the OOM
killer. However, our sysadmin will only have the time this week-end to
look at this, so most likely we won't wait for that.

The current plan is :

 * Deploy on www.maposmatic.org on Saturday morning, but no
   public announcement should be made at this point. Maxime and I would
   take care of this deployment.

 * Start the announcements on Monday morning.

We are quite sure there will be issues with the sudden peak of visitors
and renderings that the announcement will trigger, but I do think we
should go on with the deployment because those features have waited for
too long, and we need to get back to a more release soon, release often
kind of development in order to do gradual improvements.

Best regards,

Thomas
-- 
Thomas Petazzoni                http://thomas.enix.org
MapOSMatic                      http://www.maposmatic.org
Logiciels Libres à Toulouse     http://www.toulibre.org
Embedded Linux                  http://www.free-electrons.com