[mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

Sat Dec 19 11:09:59 GMT 2020

Hi Franco,

OK, I guess 2048 areas should be enough and memory is for sure enough. The major reason that I added o5m support to splitter was that this format was much faster to read, esp. when the file is read multiple times some passes only need relations or only ways. The pbf format didn't support this as well, but I thought that I also improved handling of that format.
If you like you can zip the two logs and upload them to http://files.mkgmap.org.uk .

I'll do some tests on my own machine again.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Franco Bez <franco.bez at web.de>
Gesendet: Samstag, 19. Dezember 2020 11:45
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to       "o5m"

Hi Gerd,

thanks for the fast reply.

I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each

see the log extracts below.

Ciao,

  Franco

----O5M--------------------------------------------------------------------------------------------------------------------------------
Splitter version 597 compiled 2020-01-30T14:10:47+0000
boundary-tags=use-exclude-list
cache=
description=
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=65000001
max-areas=2048
max-nodes=1200000
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=o5m
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=200000
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:10:21 CET 2020
Time finished: Sat Dec 19 10:15:44 CET 2020
Total time taken: 5 minutes 22 seconds

---PBF------------------------------------------------------------------------------------------------------------------------------
Splitter version 597 compiled 2020-01-30T14:10:47+0000
boundary-tags=use-exclude-list
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=65000001
max-areas=2048
max-nodes=1200000
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=pbf
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=200000
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:33:28 CET 2020
Time finished: Sat Dec 19 10:51:12 CET 2020
Total time taken: 17 minutes 43 seconds