[mkgmap-dev] Tiles pruned in DEM map

Sun Dec 27 18:35:19 GMT 2020

El 27/12/20 a las 19:13, Gerd Petermann escribió:
> Hi Carlos,
>
> most of the options can be prefixed with no- since many years.
I didn't remember that
> Just to make sure: My suggestion was to keep --order-by-decreasing-area for the tiles and add --no-order-by-decreasing-area at the end for the combiners, esp. the overview map.
I had understood. I built Australia map that way and worked fine, thanks.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos at alternativaslibres.org>
> Gesendet: Sonntag, 27. Dezember 2020 19:07
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> I had removed --order-by-decreasing-area from my command for DEM maps,
> but --no-order-by-decreasing-area seems to fix the problem by now. By
> the way, this option is not documented in options file and I can't find
> it in the code. Is --no a general switch for all options or is it a
> particular case for order-by-decreasing-area? I don't remember having
> seen it before.
>
> El 27/12/20 a las 10:36, Gerd Petermann escribió:
>> Hi Carlos,
>>
>> I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.
>>
>> I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.
>>
>> As a work-around for you:
>> Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com>
>> Gesendet: Sonntag, 27. Dezember 2020 09:36
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> Hi Carlos,
>>
>> reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
>> java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
>> It would be easier if you would post a link to your style.
>>
>> I am now able to reproduce the display error using this command
>> java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m
>>
>> and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos at alternativaslibres.org>
>> Gesendet: Samstag, 26. Dezember 2020 19:12
>> An: mkgmap-dev at lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>
>> I'm sorry for the late response and for breaking the thread, but I
>> didn't receive Gerd's last message, so I've had to copy it from mailing
>> list archive. Reply inline.
>>
>> OK, I think I understand what problem you see. I used JaVaWa
>> MapConverter to install the map in 11-error.zip and 12-OK.zip played
>> with it. With 11-error.zip only the data from the overview map is
>> displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
>> shows more details (this is with MapSource, in Basecamp I see details in
>> both maps).
>>
>> Yes, everything to the North of approx. S32.39125 is missing.
>>
>> I tried to analyse your files and I see three suspicious things so far:
>>
>> 1) The routing data seems to be wrong, NodCheck reports "Could not find
>> node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
>> error with the default style.
>>
>> How can I check that? If both tiles are affected, probably this is not
>> related with current issue, but other maps produced with the same style
>> will also be wrong regarding routing.
>>
>> 2) The bad overview map contains a lot more 0x4a polygons. This is
>> probably caused by the --order-by-decreasing-area, and I am not sure why
>> this happens.
>>
>> Do you think the problem is in the overview map or may be in tile map?
>> If I open tile in MapEdit++ the number of non polygon elements is almost
>> the same in wrong and correct maps. For example, number of roads is
>> exactly the same. It seems as if something is masking part of the tile
>> in MapSource (also in BaseCamp in my case); elements are there, but you
>> can't see them.
>>
>> 3) You use a special version of mkgmap, so please try also with the
>> latest version.
>>
>> With latest mkgmap and default style problem persists.
>>
>> My first guess was that the bad NOD file may cause this but the problem
>> doesn't disappear when I remove the file 51145001.NOD, so this is
>> probably not to blame. Same for the 51145001.DEM file.
>>
>> I tried to reproduce the possible problem with the 0x4a tiles using the
>> default style with this command java -Xmx4G -jar map.jar --route --gmapi
>> --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>> --order-by-decreasing-area --location-autofill=is_in,nearest
>> f:\dwnload\temp\51145001.o5m but my overview map contains the same
>> number of 0x4a tiles as your good map.
>>
>> I cannot reproduce your DEM data because I don't know the polygon file
>> (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
>> create the test files again without --remove-ovm-work-files=true.
>>
>> You can download polygon file from here
>> <https://alternativaslibres.org/tmp/australia.poly> and ovm from here
>> <https://alternativaslibres.org/tmp/ovm_51145001.img>.
>>
>> El 22/12/20 a las 18:05, Carlos Dávila escribió:
>>> I'm sorry, probably I didn't explain well enough.
>>>
>>> Overview is always correct, the problem affects only tiles. As you saw
>>> in the screenshots of my first mail, they are different in size, but
>>> they are generated from the same input files, so they should have the
>>> same size. If you zoom in to an area that should be included in a
>>> tile, only overview map is shown, no detailed map. Even more, when you
>>> zoom in, at the given point where detailed map appears, tile boundary
>>> jumps to the correct size, but nothing but overview is displayed
>>> outside the "pruned" tile.
>>>
>>> You can download correct and wrong files from the links below and play
>>> with them to get a better idea of the problem. They correspond to
>>> first tile of Australia as shown in my screenshots.
>>>
>>> https://alternativaslibres.org/tmp/11-error.zip
>>>
>>> https://alternativaslibres.org/tmp/12-OK.zip
>>>
>>> Error was generated with java -Xmx27G -ea
>>> -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>>> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
>>> -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>>> --polygon-size-limits="24:12, 18:10, 16:0"
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> Although removing --overview-dem-dist solved the issue in my first
>>> test (see command below, correct output), after a lot of tests
>>> removing options one by one from the original command, finally it
>>> seems the problem is caused by option --order-by-decreasing-area (or
>>> the combination of both).
>>>
>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>> --overview-mapname=${ABR}5${FID}
>>> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
>>> --process-destination --process-exits --housenumbers
>>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>>> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>> --check-roundabouts --check-roundabout-flares
>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>>
>>> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> It seems I still don't understand what the problem is or how to
>>>> reproduce it.
>>>> I tried this command and the overview map looks OK:
>>>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>>>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>>>> f:\dwnload\temp\51145001.o5m
>>>>
>>>> If it is related to the DEM data I probably cannot help much. You may
>>>> try a slightly different value for the --overview-dem-dist option or
>>>> a modified polygon
>>>> given with the --dem-poly option.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
>>>> von Carlos Dávila <carlos at alternativaslibres.org>
>>>> Gesendet: Montag, 21. Dezember 2020 22:09
>>>> An: mkgmap-dev at lists.mkgmap.org.uk
>>>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>>>
>>>> The problem seems to be caused by overview DEM. If I remove
>>>> --overview-dem-dist option tile is complete in MapSource. The issue can
>>>> be reproduced with this tile
>>>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>>>> vierfinderpanoramas3
>>>>
>>>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>>>> Hi Carlos,
>>>>>
>>>>> not sure if I understand what the problem is. The two screenshots
>>>>> show completely different tile boundaries, so they were not created
>>>>> from the same splitter output.
>>>>>
>>>>> Gerd
>>>>>
>>>>> ________________________________________
>>>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
>>>>> von Carlos Dávila <cardavilam at gmail.com>
>>>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>>>> An: Development list for mkgmap
>>>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>>>
>>>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>>>> and OSM+DEM) for the same area, using same input files. I split
>>>>> country.o5m with splitter and then use the same template.args output by
>>>>> splitter for all maps, just changing mapname for the different types of
>>>>> map. Given that, all resulting mapsets should have the same tiles, but
>>>>> tiles in DEM map are smaller than in the other types. They share
>>>>> part of
>>>>> the boundaries (usually two of them) with other types, but other
>>>>> boundaries are moved in, reducing displayed tile size. See attached
>>>>> screenshots. However, file size is the same (discounting *.DEM) for
>>>>> each
>>>>> tile in *.gmap subfolders.
>>>>>
>>>>> Command is quite similar for OSM and OSM+DEM maps:
>>>>>
>>>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>>>> --country-abbr=${ABR}
>>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>>> --family-id=1${FID} --product-version=$VERSION
>>>>> --series-name="OSM-$MAPA"
>>>>> --overview-mapname=${ABR}1${FID}
>>>>> --overview-mapnumber=${GRUPO}1${FID}000
>>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>>> --check-roundabouts --check-roundabout-flares
>>>>> --license-file=license_OSM
>>>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>>>
>>>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>>>> --overview-mapname=${ABR}5${FID}
>>>>> --overview-mapnumber=${GRUPO}5${FID}000
>>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>>>> --check-roundabout-flares
>>>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>>>> --road-name-config=${CONFIG} --style=mio
>>>>> --remove-ovm-work-files=true -c
>>>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>>>
>>>>> Both OSM and OSM+DEM maps can be downloaded from
>>>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>>>
>>>>> Any idea why this happens?
>>>>>
>>>>> _______________________________________________
>>>>> mkgmap-dev mailing list
>>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev