[mkgmap-dev] [PATCH v6]splitter memory usage

Fri Nov 18 16:00:22 GMT 2011

Hello list,

thanks for the comments reqarding patchv5.
Attached is one more patch to be applied against the branch.
Changes:
- Enhanced dictionary size to 65535
- Avoid allocating chunks (short[] objects) with a size that is not a
multiple of 8 (r189 wasted 4 bytes for each chunk)
- introduced special code for one-value-chunks 
- max-areas=255 is the default again, but maximum allowed is now 2048

I did not find a simple way to allow also "auto" for the parameter
optimize-mem. Maybe Steve or WanMil can help? auto should set
optimize-mem=true if Runtime.getRuntime().maxMemory() < 2GB

Remarks regarding performance:
On WinXP systems, use java -Xms1600m -Xmx1600m -jar splitter.jar ...
On 64bit boxes use also -XX+UseCompressedOops
This avoids a lot of GC work. I did not yet try that with very large files
like europe.osm.pbf.
http://gis.638310.n2.nabble.com/file/n7008720/memory_patch_v6.patch
memory_patch_v6.patch 
Some numbers for splitting germany.osm.pbf:
patchv6 with -Xms and optimize-mem=true: 588s
patchv6 with -Xms and optimize-mem=false: 594s
patchv6 without -Xms and optimize-mem=true: 651s
patchv6 without -Xms and optimize-mem=false: 622s

r189 without -Xms and optimize-mem=false: 639s

--
View this message in context: http://gis.638310.n2.nabble.com/PATCH-v6-splitter-memory-usage-tp7008720p7008720.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.