[mkgmap-dev] Idea: map layers (multiple output tiles per input tile)

Sun Apr 10 19:41:07 BST 2011

On Sun, Apr 10, 2011 at 07:42:53PM +0200, WanMil wrote:
>* The advantage to create multiple img files in one mkgmap run is that 
>parsing and preparing of the OSM data must happen once only. Do you 
>have numbers how many percent of the time is used for these steps?

No, I haven't collected any profiling data yet. Which tool would you 
recommend? For C and C++, which is what I have mainly been developing 
in, I have been using OProfile and before that, gprof. Which Java tools 
would come closest?

>Please be aware that mkgmap is optimized in such a way that it loads 
>only the tags which are needed in the style file.

Based on the log files, multipolygon relations seem to be processed even 
though the style does not contain any rules for polygons. Could you fix 
this?

>* I think the layer concept will be too complicated. I would prefer to 
>have multiple styles (as Torsten Leistikow proposed in a seperate 
>mail).

Yes, doing it with styles would be better from the compatibility point 
of view. My example of "ski routes", "bus routes", "hiking routes", "mtb 
routes" could be implemented with a parameterized style, something like 
-style=routes --style-param route=bus.

>* From my point of view it won't be possible to run mkgmap with 
>different options for the different styles (or layers in your concept).  
>Will this remove the timing advantages?

Why wouldn't it be possible to use style-specific options? I think that 
there could be a command line option for associating subsequent options 
with an output file. Something like this:

java -jar mkgmap.jar \
--output-file 10000001.img -c family1.args \
--output-file 20000001.img -c family2.args \
--output-file 30000001.img -c family3.args \
--input-file 10000001.osm.gz \
--output-file 10000002.img -c family1.args \
--output-file 20000002.img -c family2.args \
--output-file 30000002.img -c family3.args \
--input-file 10000002.osm.gz \
and so on, for every input file.

Basically, we would have a producer (input file parser) and several 
consumers (img file generators). Each img file generator would have its 
own set of options. 

Currently, another problem with multiple styles (apart from the alleged 
parsing overhead) is that the processing cannot be parallelized in an 
optimal way. For example, if I have an N-core machine, --max-jobs does 
the right thing for splitter and each mkgmap run, but my script would 
invoke mkgmap several times in succession, once for each output style.  
If I have M output tiles so that M is not an integer multiple of N, some 
cores would be sitting idle when a mkgmap run is close to completion.

Best regards,

	Marko