<div dir="ltr">Sorry - the last error was a bug in my script. I actually chose the mapid for splitter too low and while mkgmap did overwrite the .img file without warning, it did not overwrite the gmapi files without warning (due to a bug in finding out the highest number of tiles created). So with o5m input this all works fine. Seems I have to beef up my server by another NVME disk to accommodate for the larger o5m vs osm.pbf files. <div>I need to store all splitter created files - because I run mkgmap several times... Just convenience. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 30 Aug 2021 at 12:50, Felix Hartmann <<a href="mailto:extremecarver@gmail.com">extremecarver@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Well I'm not fully sure this works as intended - though at first glance I cannot find a problem - but running mkgmap on the split files gives an:</div><div dir="ltr"><br><div><div>C:\openmtbmap\maps>start /belownormal /b /wait java -jar -XX:+AggressiveHeap -XX:StringTableSize=1000003 -Xms5000M -Xmx43000M C:\openmtbmap\mkgmap.jar --max-jobs=12 --order-by-decreasing-area --code-page=1252 "--style-file=C:\openmtbmap\buildings_style" --levels="0:24, 1:23, 2:22, 3:21" --ignore-osm-bounds --ignore-turn-restrictions --merge-lines --allow-reverse-merge --transparent --draw-priority=28 --add-pois-to-areas --simplify-polygons=23:4,22:7,21:8 --copyright-file="C:\openmtbmap\openmtbmap_svn\copyrightopm.txt" --license-file="C:\openmtbmap\openmtbmap_svn\licenseopm.txt" --add-boundary-nodes-at-admin-boundaries=2 --poi-excl-index=0x6405,0x4316,0x2f00 --ignore-fixme-values --polygon-size-limits="24:16, 23:14, 22:12, 21:11" --ignore-turn-restrictions --description=buildings_fr --country-abbr=fr --country-name=france --mapname=83910046 --family-id=8391 --product-id=1 --series-name="buildings_france_30.08.2021" --keep-going --family-name="buildings_fr_30.08.2021" --tdbfile --gmapi --gmapsupp --overview-mapname=mapsetb --area-name="france_30.08.2021_buildings" -c D:\openmtbmap\maps\template.franceb1 8391*.img buildfr.typ 1>NUL</div><div>Mkgmap version 4806M</div><div>Time started: Mon Aug 30 12:34:48 CEST 2021</div><div>Number of MapFailedExceptions: 0</div><div>WARNING (global): Could not copy 83910046.RGN uk.me.parabola.imgfmt.FileExistsException: File 83910046.RGN already exists</div><div>WARNING (global): Could not copy 83910046.TRE uk.me.parabola.imgfmt.FileExistsException: File 83910046.TRE already exists</div><div>WARNING (global): Could not copy 83910046.LBL uk.me.parabola.imgfmt.FileExistsException: File 83910046.LBL already exists</div><div>Number of ExitExceptions: 0</div><div>Time finished: Mon Aug 30 12:35:28 CEST 2021</div><div>Total time taken: 39 seconds</div></div><div><br></div><div><br></div><div>Now of course there has been no file at all called 83910046* in the directory except 83910046.o5m (referenced via D:\openmtbmap\maps\template.franceb1 input file option). Google spits out no mention at all concerning this error. I would guess there is an error in the --gmapi file?</div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 30 Aug 2021 at 12:38, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">okay - well the only working solution to this problem is to use o5m output format with mkgmap, and then use osmconvert to join the files before splitting them again.<div>Would it be possible for mkgmap to have a mode to split the files one by one without the detour via osmconvert?</div><div><br></div><div>o5m uses a lot more hdd space, and mkgmap.jar is only marginally faster with 05m input vs osm.pbf (splitter works quite a lot faster on big o5m files vs osm.pbf - but small input files make not much time difference)...</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, 30 Aug 2021 at 02:25, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>I split france.osm.pbf with D:\openmtbmap\maps>if france == legend start /belownormal /b /wait java -jar -XX:+AggressiveHeap -Xms1000m -Xmx3800m C:\openmtbmap\splitter.jar --max-nodes=9600000 --max-threads=12 --search-limit=1000000 --output=pbf "--keep-complete" --max-areas=1024 --geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip --description=france --mapid=83910000 france.osm 1>NUL</div><div><br></div><div>Then I compiled the map using mgkamp.jar - and want to resplit all remaining files that did not compile with smaller max-nodes value. However splitter.jar chokes if I pass several files... Is there any way to have splitter just working with several files without trying to merge them? I just need the files split to the new max-nodes value and incrementing the map id. If I pass only one file at a time, it is a PITA to script this (is already hard enough figuring out the new map-id and creating a list of the tiles).</div><div><br></div><div>D:\openmtbmap\maps>if yes EQU yes start /belownormal /b /wait java -XX:+AggressiveHeap -Xms5000M -Xmx54000M -jar C:\openmtbmap\splitter.jar --max-nodes=4800000 --max-threads=12 --search-limit=1000000 --output=pbf --geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip --description=france --mapid=83910046 "83910015.osm.pbf" "83910025.osm.pbf" "83910026.osm.pbf" "83910042.osm.pbf" 1>NUL</div><div>Warning: --keep-complete is only used for the first input file. Further files must use higher ids.</div><div>Error: Node ids are not sorted. Use e.g. osmosis to sort the input data.</div><div>This is not supported with keep-complete=true or --problem-list</div><div>uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted</div><div> at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:497)</div><div> at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:126)</div><div> at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:82)</div><div> at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)</div><div> at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)</div><div> at uk.me.parabola.splitter.Main.useProblemLists(Main.java:503)</div><div> at uk.me.parabola.splitter.Main.start(Main.java:127)</div><div> at uk.me.parabola.splitter.Main.main(Main.java:81)</div><div><br></div><div><br></div><div>The option to first join the files with osmconvert - is a PITA - as you cannot parse several osm.pbf files to osmconvert (it only supports joining osm or o5m files).</div><div>The option to supply splitter.jar with each file on it's own - is also a PITA that would require horrendous for loops because after each split you need to check which new map-id to use for the next file!</div><div><br></div><div>So it would be great to have a keep-complete option that for each input file, but not for all input files, uses keep-complete. Actually I think this should be the default mode when supplying several input files. Even if mkgmap actually cannot run this in parallel / multithreading it would be way better than parsing each file on it's own...</div><div>Or am I dumbstruck in finding a way to resplit too big input files?</div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>