[mkgmap-dev] Error when running splitter with several input files, could there be keep-complete=single mode?

Mon Aug 30 01:25:42 BST 2021

I split france.osm.pbf with D:\openmtbmap\maps>if france == legend start
/belownormal /b /wait java -jar -XX:+AggressiveHeap -Xms1000m -Xmx3800m
C:\openmtbmap\splitter.jar --max-nodes=9600000  --max-threads=12
--search-limit=1000000 --output=pbf "--keep-complete" --max-areas=1024
--geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip
--description=france --mapid=83910000 france.osm  1>NUL

Then I compiled the map using mgkamp.jar - and want to resplit all
remaining files that did not compile with smaller max-nodes value. However
splitter.jar chokes if I pass several files... Is there any way to have
splitter just working with several files without trying to merge them? I
just need the files split to the new max-nodes value and incrementing the
map id. If I pass only one file at a time, it is a PITA to script this (is
already hard enough figuring out the new map-id and creating a list of the
tiles).

D:\openmtbmap\maps>if yes EQU yes start /belownormal /b /wait java
-XX:+AggressiveHeap -Xms5000M -Xmx54000M -jar C:\openmtbmap\splitter.jar
--max-nodes=4800000 --max-threads=12 --search-limit=1000000 --output=pbf
--geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip
--description=france --mapid=83910046  "83910015.osm.pbf"
"83910025.osm.pbf" "83910026.osm.pbf" "83910042.osm.pbf"    1>NUL
Warning: --keep-complete is only used for the first input file. Further
files must use higher ids.
Error: Node ids are not sorted. Use e.g. osmosis to sort the input data.
This is not supported with keep-complete=true or --problem-list
uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted
        at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:497)
        at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:126)
        at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:82)
        at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)
        at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
        at uk.me.parabola.splitter.Main.useProblemLists(Main.java:503)
        at uk.me.parabola.splitter.Main.start(Main.java:127)
        at uk.me.parabola.splitter.Main.main(Main.java:81)

The option to first join the files with osmconvert - is a PITA - as you
cannot parse several osm.pbf files to osmconvert (it only supports joining
osm or o5m files).
The option to supply splitter.jar with each file on it's own - is also a
PITA that would require horrendous for loops because after each split you
need to check which new map-id to use for the next file!

So it would be great to have a keep-complete option that for each input
file, but not for all input files, uses keep-complete. Actually I think
this should be the default mode when supplying several input files. Even if
mkgmap actually cannot run this in parallel / multithreading it would be
way better than parsing each file on it's own...
Or am I dumbstruck in finding a way to resplit too big input files?

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210830/60171aa0/attachment.html>