[mkgmap-dev] Splitter --cache parameter

Sun Aug 23 21:52:19 BST 2009

CD> Does it mean with --cache splitter will take the same time using bz2
CD> compressed or uncompressed input osm files?
CD> Thanks for this new feature.
CD> Cheers Carlos

The first time you use the --cache option, the splitter still needs to uncompress 
the osm file once so it can parse the osm file and generate the cache. As 
a result of this, the initial cache creation will still take longer with 
a compressed osm file than an uncompressed one. Once the cache has been built 
however, the splitter doesn't need the osm file anymore. This means that 
the second stage of the split will take the same amount of time regardless 
of the compression. Additionally, if you run the splitter and the cache already 
exists from a previous run, the splitter will run very quickly regardless 
of the compression because it doesn't need the osm file at all in this scenario.

If you don't use --cache, the splitter has to uncompress the bz2 file at 
least twice during the split (once during the first stage, once or more during 
the second) which slows things down significantly.

Based on the tests I've done, I'd recommend always using --cache in the following 
situations:

- if you are splitting compressed osm files
- if you intend to run the splitter more than once on the same osm file
- if you are splitting a file into lots of areas and the second stage of 
the split requires more than one pass

The one situation where --cache might not make sense is if you are doing 
a one-off split on an uncompressed osm file, and the second stage only requires 
a single pass to write out the split files. In this situation --cache isn't 
much faster and may even slow things down a fraction, since the overhead 
in creating the cache can outweigh the benefits the cache provides.

Chris