[mkgmap-dev] MDR building out-of-memory

Tue May 11 18:19:45 BST 2021

Hi Gerd

I've looked at this, and if Mdr5 space becomes a problem again, I'll
consider converting to it.

A couple of comments:

My map had 2909735 poi so the sort chunk size was ~727433. The cache
sizes after each chunk were 563095, 603595, 597239 & 605718. Is the
cache worth-while for this low hit-rate? Just running the Gmapsupp
combiner on existing tiles (without --route, so no streets), I got a
run time of 1 min 44 secs with the cache and 1 min 30 without! However
most of this time is copying the tiles into gmapsupp.img so not an
accurate statistic.

You could pre-allocate List<> "merged" with the correct size.

Ticker

On Tue, 2021-05-11 at 14:40 +0000, Gerd Petermann wrote:
> Hi Ticker,
> 
> I've committed the patch as is. I've not seen big changes in
> performance, but I've used a different (already existing) set of
> files which was created with my own style. For me,
> Mdr11.preWriteImpl() is the most problematic part reg. OOM errors.
> 
> Maybe look at the code which uses LargeListSorter.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> Gesendet: Dienstag, 11. Mai 2021 13:27
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] MDR building out-of-memory
> 
> Hi Gerd
> 
> Here is updated version of patch.
> 
> Changes from last:
> 
> Uses your cache code for region and country (in 2 places). For
> British
> Isles, there are 190 regions and 7 countries, so I don't think the
> extra memory will be a problem and there should be some performance
> benefit.
> 
> Delays allocating cities until it can use sortKeys.size() for initial
> allocation. For above map this is 0.07% too big, so I don't think
> trimToSize() is worthwhile.
> 
> Shares the Sort object between the 4 methods.
> 
> Ticker
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev