[mkgmap-dev] Splitter output files are nearly empty

Wed May 4 13:08:07 BST 2011

On 2011-05-04 11:51, Steve Ratcliffe wrote:
>> ***** Full GC *****
>> Elapsed time: 18m 0s   Memory: Current 135MB (106MB used, 29MB free) Max 6666MB
>
> The 'Full GC' line is not a problem, it is a message printed by
> splitter every so often before it attempts to force a garbage collect.
> This is not usually a useful thing to do, but it can help make the
> printed memory usages more accurate, I suppose.
>
Thanks for clearing that up Steve. The newer splitter apparently needs 
only a fraction of the memory it needed before. Great work devs!

> Splitter can still read .osm.gz from stdin as long as there is a
> --split-file
> option, and as long as it doesn't have to use more than one pass
> through the data.
>
Alright. I don't think splitter is able to only use one pass on a planet 
file so it's another reason to move to the pbf format.

> You should just provide no file name on the command line though.  (ie
> do not give /dev/stdin) However /dev/stdin should work as long as the
> input is uncompressed.
>
Thanks, the /dev/stdin ended up there because of other problems I had 
and it worked. I can't confirm or deny that it works without it ;-)

> Previously there was a --cache option which would copy and convert the
> input to a cache file, which would then be re-read if --split-file was
> not given or if more than one pass was needed.
>
> Since the newer pbf format is as or more efficient than the cache file
> format and osmosis is capable of producing it directly it seems best
> to just use osmosis to write it. Having splitter convert to pbf will
> not save any disk space and will probably be slower than overall
> than osmosis producing it directly.
>
Again: thanks. It's clear I'll better migrate to pbf.