logo separator

[mkgmap-dev] Idea: map layers (multiple output tiles per input tile)

From WanMil wmgcnfg at web.de on Sat Apr 16 20:05:47 BST 2011

Am 15.04.2011 23:23, schrieb Marko Mäkelä:
> On Sun, Apr 10, 2011 at 09:18:00PM +0200, WanMil wrote:
>>> On Sun, Apr 10, 2011 at 07:42:53PM +0200, WanMil wrote:
>>>> * The advantage to create multiple img files in one mkgmap run is that
>>>> parsing and preparing of the OSM data must happen once only. Do you
>>>> have numbers how many percent of the time is used for these steps?
>>>
>>> No, I haven't collected any profiling data yet. Which tool would you
>>> recommend? For C and C++, which is what I have mainly been developing
>>> in, I have been using OProfile and before that, gprof. Which Java tools
>>> would come closest?
>>
>> I would add some time logging to the mkgmap source code. There are
>> profiles for Java but I think they are not useful for such a job.
>
> I made an experiment. Generating the full map with the default style
> from 9 osm.gz tiles on a dual-core machine took some 7 minutes, with
> assertions enabled.  With assertions disabled, generating a routes layer
> of the whole Finland takes about 2.5 minutes when parsing the whole
> country extract in osm.pbf (one input tile, thus one core used). I
> repeated this test for 5 layers, which I split from the currently rather
> useless --style=routes. These styles were routes-foot
> (route=foot|route=hiking), routes-bicycle, routes-bus, routes-rail
> (route=subway|route=train|route=tram), and routes-ferry. I did not
> generate routes-ski or routes-road.
>
> Each of these 5 minimal styles would spit out a number of multipolygon
> error messages, even though the styles are this simple:
>
> cat>  relations<<  EOF
> type=route&  route=ferry
> {
>     add ref='${name}'; # if ref is missing, use name
>     set ref='${network} ${ref}' | '${ref}' | '${network}';
>     apply { set mkgmap:route='$(mkgmap:route),${ref}' | '${ref}' }
> }
> EOF
> cat>  lines<<  EOF
> mkgmap:route=* { name '${mkgmap:route}' } [0x1b resolution 20]
> EOF
>
> I think that 2.5 minutes for these simple layers is too much. Because
> the processing time per layer did not vary much, no matter how much data
> was generated (13312 bytes .img of ferry routes, 986624 bytes of bus
> routes), I would guess that parsing is dominating the execution time,
> and the processing time would reduce from N*2.5 minutes to about 2.5
> minutes if we implemented a 'multiple output tiles from one input tile'
> feature.
>
> Do you have any hints where to add the "poor man's profiler" timestamp
> printouts to the mkgmap source code?
>
> 	Marko

I would like to add some profiling code to mkgmap.
My idea is, that it is possible to add a static class that collects 
profiling information from mkgmap classes. The classes can call:
Profiler.start(String stepname) and Profiler.stop(String stepname).

At the end of mkgmap it creates a statistic file how many time is spent 
in each step. Of course the profiler is active only if a specific option 
is set.

This could be used to track the processing time of mkgmap with 
subsequent commits.

Good idea? Bad idea? Any more ideas?

WanMil



More information about the mkgmap-dev mailing list