[mkgmap-dev] Fix Sea Patch

Sat Jan 30 17:05:05 GMT 2021

Hi Mike & Gerd

I've just tried running with this on britain-and-ireland.osm.pbf using:
 --generate-sea=multipolygon,extend-sea-sectors,close-gaps=750

I got a few messages from it that I don't get normally - attached.

Looking at some of these, I found:
1/ islands, tagged with coastline, within lakes.
2/ strange inland coastline triangles (didn't check which way round the
coastline went) Source "OS_Opendata_Vectormap_District"
3/ water areas,
close to the sea - maybe salt-water lagoons, tagged with coastline -
I'm not sure why these are flagged if inland seas are allowed.

Some of these could be fixed in OSM.

The run-time has increases from about 35 mins to 44 mins (would need to
run again in a clean environment to be sure of this). I suggest having
an option to enable the extra checking.

I don't have any problem with more detailed logging.

Good to fix the spelling mistake (but should fix loadLandAndSee as
well)

Ticker

On Fri, 2021-01-29 at 11:18 +0000, Mike Baggaley wrote:
> Hi Gerd, attached is a slightly updated patch, with test data and the
> results from r4600 and the patch. The existing code does not identify
> all
> the problems in the test data and does not provide sufficiently
> detailed
> information on those it does find to enable investigation/correction.
> 
> The test data contains 4 test scenarios:
> land within sea within land, no errors - the existing code is ok here
> sea within land within sea, the outer sea should produce an error -
> the
> existing code is ok here
> land within land within land, the two inner lands should produce an
> error -
> these are not found by the existing code
> sea within sea within sea, all three seas should produce an error -
> the
> existing code finds the errors but does not identify the surrounding
> way for
> the inner two
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com] 
> Sent: 18 January 2021 11:54
> To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike,
> 
> it would be great  to have some unit tests or at least a sample file
> containing the problem cases which are treated by your patch.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Montag, 18. Januar 2021 10:06
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Gerd,
> 
> I had too little logging information in the version I sent
> previously. The
> attached remedies that.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Mike Baggaley [mailto:mike at tvage.co.uk]
> Sent: 18 January 2021 00:28
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: RE: [mkgmap-dev] Fix Sea Patch
> 
> Hi Gerd,
> 
> I'm not sure that mkgmap should be prescriptive about where the input
> data
> comes from. However, I have produced an updated patch that reports on
> errors
> in the input data without providing any functionality to fix the
> problems.
> Please see the attached patch that is able to identify most problems
> in sea
> data.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 05 January 2021 08:40
> To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike & Ticker,
> 
> I would not want to add code to mkgmap to fix problems which only
> occur in
> data that is produced by a 3rd party software. My understanding is
> that the
> "fixing" should happen in that 3rd party software, not in mkgmap.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Montag, 4. Januar 2021 20:40
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Ticker,
> 
> I use coastline data that is generated from just line data and the
> algorithm
> used is not able to properly determine whether polygons are islands
> or
> water. It does however make sure polygons are closed, so I don't have
> problems with having to join ways together and hence I don't use
> close-gaps
> or extend sea-sectors.
> 
> I'm unclear exactly what an anti-island is supposed to be - I have
> considered it to be a separate bit of sea, and in my data I have lots
> of
> those that are correct. They are only incorrect if they are within
> the sea,
> rather than the land. For example, if there is a coast road with sea
> on both
> sides of it, I am likely to have a separate sea polygon within the
> land if
> its connection to the sea is under a bridge. The patch checks for sea
> within
> sea and land within land. It doesn't then go back and recursively
> check for
> nested problems - I suspect this would be quite an overhead.
> 
> The code is certainly not perfect, but does produce enormously
> improved
> results for me at least. I can still see at least one error in my
> map, but I
> think the remaining errors are tile boundary issues that are resolved
> differently on adjacent tiles. I am thinking of a small app to read
> the
> mkgmap log file and the coastline data, reversing whatever mkgmap
> says is
> wrong. By running mkgmap without splitter on just the coastline data
> I
> should be able to avoid any tile boundary problems.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk]
> Sent: 04 January 2021 13:58
> To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike & Gerd
> 
> I always try to use the coastline data in the downloaded OSM map to
> generate the sea with option:
>   --generate-sea=multipolygon,extend-sea
> -sectors,close-gaps=750
> and have only ever had 1 problem with faulty
> coastline. In this case it wasn't a reversed section but a lake had
> been tagged with coastline, worse, in the wrong direction so it
> wasn't
> treated as an internal sea, rather an island, flipping the rest of
> the
> tile to sea.
> 
> I sometimes have problems with the continuation of the downloaded
> country's
> coastlines being cut off and ending up near the middle of tiles, but
> closer
> to the wrong edge or, when extended to the edge, they are in the same
> direction as an adjacent coastline, because the un-crossing is
> outside the
> downloaded area.
> 
> @Mike - Do you use --coastlinefile to circumvent this type of problem
> and is
> there an advantage to this over using --precomp-sea? The data seems
> to need
> a lot of correcting!
> 
> If arbitrary bits of coastline are in the wrong direction, should you
> have
> another phase, after joining in the same direction and removing
> anti/islands. Try to join against their direction and moving closed
> ways to
> an ambiguous closed list and other ways that joined to an ambiguous
> linear
> list. Then see if the ambiguous ones can be resolved, maybe by
> weighting by
> the direction of the majority.
> 
> I feel that some of your fixes of land/sea are too late in the flow
> of code,
> ie after generateSeaBackgound has been set.
> 
> It seems a reasonably assertion that all anti-islands are a mistake
> in the
> data. I think the only one is the Caspian Sea, which is unlikely to
> be fully
> contained in a tile.
> 
> checkIslands(), to do the full job, would need to look at the nesting
> of
> sea/land/sea... so, given above, I don't think it worthwhile.
> 
> In verifyHits(), I don't see where any changes that resolves
> "adjacent hits
> in same direction" are applied. The 'fix' could make it totally wrong
> rather
> than partially wrong. Looking for 3 in the same direction and
> reversing the
> middle one might be better.
> 
> As I said earlier, I've never had a problem where this type of fixing
> would
> be of any use.
> 
> The swapping between land and sea needs to do more than just re
> -tagging; the
> multi-polygon handling needs to be considered and also the fullArea
> of sea.
> 
> Ticker
> 
> On Sat, 2021-01-02 at 10:32 +0000, Gerd Petermann wrote:
> > Hi Mike,
> > 
> > > This was removed some time ago because it didn't work properly
> > Do you have a version number where this happened? I only know the
> > floodblocker option which is still evaluated in the source.
> > 
> > Gerd
> > 
> > 
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Mike Baggaley <mike at tvage.co.uk>
> > Gesendet: Donnerstag, 31. Dezember 2020 16:17
> > An: 'Development list for mkgmap'
> > Betreff: [mkgmap-dev] Fix Sea Patch
> > 
> > Hi Gerd,
> > 
> > Mkgmap used to contain some code that attempted to correct
> > incorrect
> > sea and
> > land data where polygons were ordered in the wrong direction. This
> > was
> > removed some time ago because it didn't work properly. I have put
> > together
> > the attached patch that works much better and have been using it
> > for
> > some
> > time. With the sea data that I use, which contains quite a large
> > number of
> > incorrectly ordered polygons, it manages to correct every error it
> > finds. It
> > won't be able to fix really gross errors, but providing most of the
> > data is
> > correct it will produce good results.
> > 
> > Please review and commit if happy.
> > 
> > Cheers,
> > Mike
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sea3.log
Type: text/x-log
Size: 4953 bytes
Desc: not available
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210130/85fb4fbd/attachment.bin>