[mkgmap-dev] splitter: option for maximum tile area?From Gerd Petermann gpetermann_muenchen at hotmail.com on Wed Nov 7 06:29:50 GMT 2018
Hi Bernhard, thanks for updating the Srtm2OSM wiki. I think it could still be improved, for example I did not find the -maxwaynodes option when I just looked for it, only found it using the search function of the browser. Reg. your other findings: I know these pros and cons for a max-nodes value different to the default: A larger --max-nodes value can help when you want to split a very large file (a large continent extract or planet) because - splitter might need less memory because it reduces the number of tiles and thus the number of ways crossing tile boundaries which are handled by the --keep-complete processing - the smaller number of tiles allows to avoid a limit in the gmapsupp, if I remember correctly max 4000 tiles are allowed in one mapset - the overall amount of data in the split files is smaller because less data is repeated in the areas outside the bounding boxes A lower --max-nodes value can help because - it avoids to reach several limits of the img format, esp. this one: "There is not enough room in a single garmin map for all the input data." - it reduces the amount of memory needed in mkgmap to process one tile. This can be important when multiple threads start to steal memory from each other and thus decrease throughput - it allows to open and check the tile in JOSM (presuming you have the pbf or o5m plugin installed) So, finding a perfet value depends on style and hardware and what mapset you plan to create. The expected effect of the --precomp-sea option in splitter is that it produces smaller tiles in areas with many islands because it will add extra weight to the coastline nodes. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil at gmx.de> Gesendet: Dienstag, 6. November 2018 14:54 An: mkgmap-dev at lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area? Thanks all, meanwhile I could generate a map which looks fine in QLandkarte and on my Oregon. I'd like to summarize the findings for those who try to create contour lines with srtm2osm and feed them into a Garmin map. I updated the wiki page https://wiki.openstreetmap.org/wiki/Srtm2Osm First of all, it is possible to use srtm2osm with elevation data downloaded from Viewfinderpanoramas and unzipped into its cache folder. In contrast to what one might expect, the folder where the *.hgt files must be placed is not the folder indicated with the -d option, but a folder "SrtmCache" below it (that's hard coded in the source code of srtm2osm). Next, the default start ids for nodes and ways generated by the program is long.MaxValue, which is a reserved value in splitter, thus causing an exception (see a previous thread: http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2018q4/029073.html ). The options -firstnodeid 9023372036854775807 -firstwayid 9023372036854775807 (or other high values less than long.MaxValue) remove this issue. It is important to note that a misspelled option will not cause an error message by srtm2osm - that option is merely ignored. So do not feel surprised when a problem later on a step later on just happened again, though you thought that you changed something before... Another issue is the unlimited size of ways. In OSM data, ways longer than 2000 nodes do likely not exist because the API prevents the upload of such long ways. But srtm2osm produced ways with more than a hundred thousand nodes. splitter tries to keep ways complete. The long ways lead to a slow processing by splitter, and to an extremely inflated size of the tiles created by splitter (if the original file and the tiles have the same format, the sum of file sizes of the tiles should not be much more than the file size of the original file): upto 9 times the original size. There after, mkgmap crashes with an OutOfMemory error. The solution is to limit the file size with option -maxwaynodes 5000 (or an even lower value: the files were still inflated by about a half). Adding the option --precomp-sea to the splitter options is irrelevant for the OutOfMemory exception in mkgmap. Also, the value of the --max-nodes splitter option proofed irrelevant for the exception. Running mkgmap with --max-jobs=3 instead of 4 was also irrelevant. But running splitter with a high value of --max-nodes in combination with --precomp-sea lead to a different exception in mkgmap: (MapFailedException): 47120100.osm.pbf: (thrown in RoadDef.writeRgnOffsets()) Overflow of the NET1. The tile (47120100.osm.pbf) must be split so that there are fewer roads in it Then, that tile was left empty. I could see this issue with big cities (Bangkok, Kuala Lumpur) now, but never experienced it before (I created a map of Central Europe with a value of 3 million without precomp-sea - absolutely no problem). I hope this information won't be outdated when some else tries to follow these steps. Kind regards, Bernhard Am 31.10.2018 um 22:03 schrieb Bernhard Hiller: > Hi all, > currently a Java OutOfMemory exception prevents me from creating a > map. I already use option --max-jobs=3 (the machine has 4 physical > cores) and -Xmx5G (of 8 GB installed). Beyond OSM data, the map > contains DEM and elevation contour lines. > From the tiles finished and those with a new timestamp but about 0 > bytes length, I can see that mkgmap was rendering tiles 47120005, > 47120006, 47120007 at the time of crash. > Tile 47120005 is extremely large by physical area - some 6° x 5.5° > (see attached file), covering a lot of the south chinese sea, i.e. > there are not many actual data in that area. > I guess that the problem arises with that tile. I remember some case > in the past where a single tile covering such a large area of mainly > sea caused mkgmap to take an enormous amount of time for rendering - > also here, mkgmap already spent about 1 hour before crashing. > So I'd like to ask: is there some possibility for limiting the area of > a tile among the splitter options? > Kind regards, > Bernhard _______________________________________________ mkgmap-dev mailing list mkgmap-dev at lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
- Previous message: [mkgmap-dev] splitter: option for maximum tile area?
- Next message: [mkgmap-dev] DEM and SRTM hgt files: size too small
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the mkgmap-dev mailing list