[Maposmatic-dev] Database update becoming critical

[Maxime Petazzoni]

Hi all,

There has been several database updates taking more than 1000min in the
past 10 days, with yesterday's update taking 1398min, which is awfully
close to 24 hours.

Yesterday was a bit special since the queue has been running all day to
catch up for the daemon being offline over the week-end. But this
clearly shows that we will very soon no longer we able to sustain both
the database update with osm2pgsql and a continuously demanding queue.

Given that we will soon publicize MapOSMatic even more during the LSM
and SOTM conferences, I feel it is becoming more and more urgent that we
consider other means of updating the database.

I plan on implementing the query optimization we discussed a little
while ago on this list, which will hopefully speed up the processing
time for each job by considerably reducing the stress on the database,
which /should/ bring some relief to the update process so it can still
complete within 24 hours.

But this is obviously not a perennial solution, and we should look to
rapidly move to hourly our minute diffs using Osmosis. I'm currently
investigation this solution in more detail. If one of you already had
some information to share on Osmosis other than what's in task #10148
<https://savannah.nongnu.org/task/?10148>, please do!

Thanks,
- Maxime

-- 
Maxime Petazzoni <http://www.bulix.org>
 ``One by one, the penguins took away my sanity.''
Linux kernel and software developer at MontaVista Software

signature.asc
Description: Digital signature