[mkgmap-dev] split using --polgon-file

Thu Apr 4 09:19:30 BST 2024

Hi Felix,

reg. 2: Hmm, maybe I never tried poly files with holes? The current code more or less ignores them now. I didn't expect that, but unless there is a good reason to use holes I am not motivated to invest time here.
I'd rather document the behaviour as unpredictable ;)
reg. 3: I can't tell you what you should want to achive. Splitter has some goals like
- don't create (almost) empty tiles
- don't create long narrow tiles
- don't create too huge tiles
- try to calculate tiles so that each is close to the -max-nodes value
If all those goals are met the split is considered nice and splitter can stop searching for a better one.
The splitter output is meant to help (me) when splitter doesn't find a solution.
reg. 4: I have no idea what you are trying to optimize. If you think your device works better with smaller tiles you can use a smaller -max-nodes value. I guess most users here in the forum try to minimize the disk space and CPU time costs.
reg. 5: I tried with Niedersachen split into 25, 2000 or 6000 tiles. The gmapsupp sizes are ~128MB, ~209MB and 2370MB. When I use a Hex browser to look at the gmapsupp files I see a lot more zeros in the larger gmapsupp with more tiles. I think you may have hit an error in the block size calculation because of an integer overflow but even when I correct this the size is much larger. That's probably a limit in the FAT file system that is used in *.IMG format. I'll open a new thread regarding the fix for the calculation in mkgmap.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Felix Herwegh <mlmmduk at herwegh.de>
Gesendet: Mittwoch, 3. April 2024 22:08
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] split using --polgon-file

Hi Gerd,

> Hope this helps... It shure does, thanks!

Some follow-up nevertheless, if I may...

reg. 2: Exactly (?) what I tried first. No hole generated though (splitter 653; p.e. @ resolution=17, max-nodes=10000 -> 320 tiles and @ resolution=13, max-nodes=1200000 -> 3 tiles )

MallorcaTest Mallorca 3.218991 40.023410 2.980039 40.004478 2.705381 39.892882 2.504880 39.751547 2.274167 39.631079 2.238462 39.546414 2.471921 39.378897 2.639463 39.448922 2.699887 39.306677 2.903134 39.291799 2.867429 39.102360 2.971799 39.057586 3.188779 39.270539 3.273923 39.306677 3.381040 39.468007 3.570554 39.664916 3.567807 39.783215 3.392026 39.848614 3.249204 39.821196 3.323362 39.951861 3.218991 40.023410 END !MallorcaHole 2.848204 39.748379 2.576291 39.643770 2.848204 39.560707 2.980040 39.560707 3.337094 39.624733 2.980040 39.748379 2.848204 39.748379 END END

Assuming the small overall size of the island might be critical, I unsuccessfully tried a big chunk of Germany with some big hole too. Also unsuccessfully tried "none, 1 and !2" for names. JOSMs .poly export is without the exponential format, but that works well with splitter too, without holes. Since Geofabriks poly uses way more points I tried a new Mallorca poly with >40 points each for outer and inner, but to no avail. I then made a SouthAfrica map with Lesotho successfully spared out (!), using Geofabrics original poly as well as that one exported again by JOSM w/o the exponential notation. Using my identical script and configs, style and type. And Geofabriks South-Africa pbf ;-). Using all of Africa as source, Lesotho is there though :-( Lastly I tried setting --keep-complete=true to false, but that wasn't responsible either.

Fishin' in the dark...

What am I missing/doing wrong? Using: java -Xmx8G -jar /home/felix/AA-Poolm/osm/Soft/splitter-latest/splitter.jar /home/felix/AA-Poolm/osm/AutoGen/SA/SA-R/../SA.o5m --mapid=54320001 --max-nodes=1200000 --max-areas=4096 --geonames-file=/home/felix/AA-Poolm/osm/sources/cities15000.zip --keep-complete=true --output=o5m --polygon-file=/home/felix/AA-Poolm/osm/poly/hgh/Geofabrik-SouthAfrica.poly --output-dir=/home/felix/AA-Poolm/osm/AutoGen/SA/SA-R/5432-SA-R-split
Has anybody successfully used this feature with splitter lately? I also still wonder, that my wrong .poly syntax tries for the second polygon all went through uncommented and assume, Geofabrik using their .poly on other software only.

Is there any characteristic output in splitters log to look for, while processing subtracting polygones? Found none (producing no holes...haha).

I'm done for today ;-), especially since this punching holes is just curiosity and not needed at all.

reg. 3: Not confused, but wondering about reporting "nice" results after having given up and accepted "anything" in what I interpreted as subsequent steps. Nevertheless: just a final "solution is nice" is what to aim for? Are the criteria I found for that about correct?

reg. 4: May I assume then, aiming for < 2048, optionally smaller, but finally "nice" tiles < 100%, using a reasonable resolution, could be a good strategy? If not prioritizing but watching performance and keeping an eye on filesizes? At least for targeting non-rectangular shaped maps?
And that some few tiles with very few nodes may be unavoidable and have to be accepted at times (p.e. due to having nearly all sea, if the polygon is not exactly following shore)? Is a max. accepted tilesize (@ min. tile count) known, and perhaps already handled by splitter?

reg. 5: Global index? The figures I gave where using mkgmap without index. Wouldn't labels space requirement grow linear with tile count?

Cheers Felix

On 03.04.24 15:49, Gerd Petermann wrote:

Hi Felix,

reg 1: The polygon is used to filter the input, splitter cannot write non-rectangulare tiles.
The logic with the 40 or less edges is very special. The idea was to take a split file for a continent or planet and group tiles.
reg. 2: (poly with hole) See e.g. http://download.geofabrik.de/africa/south-africa.poly
reg. 3: Splitter tries several times to split, maybe you were confused by partial results?
reg. 4: If I remember correctly it was reported that some devices cannot handle a gmapsupp with > 2048 tiles, that's why some map providers try to create large tiles.
reg. 5: See above. Reg. size: I assume the global index requires much more space, also the labels which are repeated in each tile.
reg. 6:
--search-limit : It may save some time to use a higher limit when you know that the default will not find a solution
--num-tiles: This is nomally used to divide a huge file (e.g. a continent) into a few (2 .. 6) tiles so that these files can be used again as input for splitter.

Hope this helps...
Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Felix Herwegh <mlmmduk at herwegh.de><mailto:mlmmduk at herwegh.de>
Gesendet: Mittwoch, 3. April 2024 15:18
An: mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
Betreff: [mkgmap-dev] split using --polgon-file

Hi,

recently I started playing with splitting using polygon files, primarily based on the documentation in https://www.mkgmap.org.uk/doc/splitter.html. Got it to work, even with multiple areas in one .poly file (testing only, no application). The idea is to better cut out neighboring countries' and sea area for countries poorly aligned to lat/lon (NL, BE, IT...).

Digging deeper though, severeal questions arose, that I couldn't answer, neither by the doc mentioned above nor by the (seemingly somewhat more topical but brief) --help output of splitter, not even by searching this lists archives.

  1.  Although the doc for --polygon-files says:
"If the polygon area(s) describe(s) a rectilinear area with no more than 40 vertices, splitter will try to create output files that fit exactly into the area, otherwise it will approximate the polygon area with rectangles."
So far I have not been able to generate a single split, that exactly follows the polygon, even if quite simple (<<40 points). I always get tiles on or extending the polygone.
I'm not shure, I understand that quoted sentence. Does it (for a single area) mean a polygone of <= 41 points, hence <=40 lines (if first and last point are identical)?
Is this functionatlity still in place, or has it been deprecated?
Neither am I shure, I understand the target.
Should splitter generate non-rectangular tiles with an alignment according to a polygone at all, or only rectangular aligned to lat/lon? If the latter, "exactly follows" could only work for polygones having each line parallel lat or lon?

  2.  The .poly files should follow the Osmosis syntax, which also specifies:
"The polygon section name may optionally be prefixed with "!" to subtract the polygon. The section(s) containing the larger area from which to subtract should be listed first. All the polygon sections are combined together to create the final filter area."
I couldn't make that work. Tried "!" directly in front of the section-name, separated by blank and on an individual line. Splitter does not complain, but seems generate identical splits for all 3 tries and without that area specified at all.
Does splitter respect this syntax at all? (testing only, no application)

  3.  From what I've read so far, one might want to aim for "solution is nice", sufficiently even distributed node counts over all tiles, right?
Is that 80% tiles @ > 80% targeted nodes and <3% tiles below 33% targeted nodes?
Why do I get nice solutions although (after having the search limit being increased in several steps) splitter comments "No good solution found, trying to find one accepting anything"?
What would define a good split?

  4.  My initial approach was to increase tile count to better follow the polygone. I basically did that by decreasing --max-nodes and, when splitter ran into tiles having >100% targeted nodes, raising resolution to allow for those tiles (cities...) to be smaller. I eaven had the "feeling", that my Edge 1040 appreciated more smaller tiles by zooming and scrolling smoother.
On the other hand from the doc and for example some discussion here between Gerd and Felix Hartmann regarding and around r609 release I've got the impression, that the typical target might be minimum tile count, as long as some (Garmin?) max. tilesize (?) is not reached. Why is that?

  5.  Increasing a significant portion of Germany maps tile count ~by factor 7 from ~280 tiles (@ max-nodes=1200000, resolution=13) to ~2000 tiles (@ max-nodes=150000, resolution=15) only took around an additional 8% in gmappsupp filesize, but another factor of 3 to ~6000 tiles (@ max-nodes=50000, resolution=??) made it "explode" to 300% of the original size. This map did still load (altough with some spinner delay) in QMapShack, but no longer on my Garmin device, not even got listed.
Just out of curiosity: I can understand some increased overhead due to more tiles (as in the first step), but no progressive increase like this, since the OSM data basically stays identical. Is there some distinct border effect involved, like having to switch to some bigger data-type at some tile count or something similar?

  6.  Not being mentioned in the doc but (briefly) in the --help output at least 2 options seem to be accessible and might be involved:
a) --search-limit, seems to be set and increased automatically if needed
b) --num-tiles, seems to be unset/unused by default
Any use to temper with a), p.e. start below default?
What would be the difference (benefit?) of using b) over decreasing max-nodes to control tile count?

Sorry for so many questions; thanks for any input ;-)

//Felix Herwegh
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev