[mkgmap-dev] [PATCH v5]splitter memory usage

Thu Nov 17 18:42:44 GMT 2011

Well I think you didn't read my last message yet....

I think currently about 6GB of free RAM are needed to have max-area=1024 
to be working (assuming max-nodes 900 000). What is the actual maximum? 
Could one set max-area=1300? (that is about what I think is max on my 
machine with max-nodes 900 000) As more RAM is available in 1-2 years 
some people maybe would like to go for max-area 4000 when splitting 
Europe (creating a map of the whole world doesn't make sense due to 
Garmin limits anyhow).

As it needed 1pass instead of 4, the time improvement was well 
worthwhile (about 40% faster).

I think the max-area default could be set to 512, that should need about 
as much memory as the old 255, shouldn't it? So it wouldn't break 
anything due to out of memory errors for people simply updating the 
splitter.

On 17.11.2011 19:29, Gerd Petermann wrote:
> Hello Felix,
>
> thanks for trying. I think the most interesting part of the fix is 
> that it allows to split very large files with a higher max-area value 
> when memory is an issue. As long as only one pass is needed, the 
> benefits are rather small.
> BUT if you can split europe in two or three passes instaead of 5 or 
> more the speed is probably much better.
> Anyhow, I am no longer sure about my change reg. the default value for 
> max-areas. I think the old default 255 was better.
> On my linux box with -Xmx3200m I still needed two passes with 
> max-areas=512 for europe.
>
> ciao,
> Gerd
>
>
> >
> > some more speed observations:
> > country germany.pbf from Geofabrik
> >
> > splitter_patched with optimize-mem: 3 min 6 seconds
> > splitter_patched without optimize-mem: 3min 2 seconds
> > old splitter: 3 min 42 seconds
> >
> > I am currently running against europe.pbf, will post the results later.
> > But it seems that the patched mkgmap is consistently a bit faster. I
> > think optimize-mem is better of by default though (most people using
> > mkgmap/splitter have powerful computers anyhow).
> > I'm interested if the 1024 max areas vs the old 255 and subsequent less
> > runs make for a bigger speedup than on small(er) extracts.
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20111117/ff29105d/attachment.html 