[mkgmap-dev] RE Commit r3801: merge split-shape branch

Wed Feb 15 08:53:06 GMT 2017

Hi Mike

OK - I think I understand the problem.

Can you try with --order-by-decreasing-area and let me know if that
stops the unbound recursion, and, in the meantime, I'll work on it.

Thanks
Ticker

On Wed, 2017-02-15 at 00:48 +0000, Mike Baggaley wrote:
> HI Ticker, having run this under the debugger now, it appears that it
> is not the postcode data after all - I was getting confused between
> the too small to split messages and too many POIs messages. I was
> previously getting too small to split messages in splitter, but not
> in mkgmap.
> 
> I temporarily changed the code to:
> 
> 				} else if (mustSplit) { // can't reduce
> size, so force more subdivisions
> 					if (depth < 1000) {
> 						log.debug("splitting
> area by contents", area);
> 						MapArea[] sublist =
> area.split(1, 1, bounds, true);
> 						addAreasToList(sublist,
> alist, depth + 1);
> 						continue;
> 					} else {
> 						log.error("Area too
> small to split at " + area.getBounds().getCenter().toOSMURL() + "
> (reduce the density of points, length of lines, etc.)");		
> 				
> 					}
> 				}
> 
> This ran successfully and gave a single error:
>  Area too small to split at 
> http://www.openstreetmap.org/?mlat=51.797190&mlon=0.918646&zoom=17 (r
> educe the density of points, length of lines, etc.
> 
> Stopping in the debugger, I can see that sizes[MapArea.SHAPE_KIND] is
> 68076 and from the coordinate it is processing the coastline. The
> call stack has the same value for the sizes[MapArea.SHAPE_KIND] in
> every instance of addAreasToList, so it looks like the call to
> area.split is not actually splitting. Looking at the resulting file
> in BaseCamp, I can see that the coastline is corrupt.
> 
> Hope that helps,
> Mike
> 
> -----Original Message-----
> From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk] 
> Sent: 14 February 2017 23:21
> To: Mike Baggaley <mike at tvage.co.uk>
> Subject: Re: [mkgmap-dev] RE Commit r3801: merge split-shape branch
> 
> Hi Mike
> 
> I have the raw postcode data (codePointOpen.zip) which I processes
> just
> to take the first few at the same point but I can change it do all of
> them individually. Do you feed it with a map through the splitter or
> just make a postcode only overlay? What is the bit of style you use
> to
> represent each postcode? I can probably knock up something similar.
> 
> The code should cope - I can't see anything obviously wrong so need
> to
> try and reproduce the problem
> 
> Ticker 
> 
> On Tue, 2017-02-14 at 18:17 +0000, Mike Baggaley wrote:
> > Hi Ticker the data is postcode centres (points). As well as
> > geographic postcodes, the Royal Mail postcode data includes a
> > number
> > of postcodes that are centred on sorting offices (presumably for
> > box
> > numbers). There are quite a few at large sorting offices, but if
> > each
> > split should be able to accept 255 points, and it is exceeding 2000
> > splits before crashing, it looks like the splitting is not working 
> > -
> > there won't be more than 2000 * 255 postcodes at the same place. My
> > OSM file is rather large. I can see whether I can extract a map
> > centred on a sorting office.
> > 
> > Regards,
> > Mike
> > 
> > -----Original Message-----
> > From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk] 
> > Sent: 14 February 2017 17:52
> > To: mkgmap-dev at lists.mkgmap.org.uk
> > Subject: Re: [mkgmap-dev] RE Commit r3801: merge split-shape branch
> > 
> > Hi Mike
> > 
> > This is an area of code I've changed and it should be able to cope
> > with
> > many items at the same location without recursing.
> > 
> > Do you have some sample data? what are the items (point/line/shape
> > extended?)
> > 
> > Ticker
> > 
> > On Tue, 2017-02-14 at 17:40 +0000, Mike Baggaley wrote:
> > > Hi Gerd, since this change I am getting a
> > > java.lang.StackOverflowError crash
> > > caused by the code recursively attempting to split something
> > > which
> > > is
> > > unsplittable (assuming the split is based on location), as I have
> > > a
> > > large
> > > number of points at exactly the same location (from external data
> > > I
> > > add to
> > > the OSM data).
> > > 
> > > The offending line is at
> > > uk.me.parabola.mkgmap.build.MapSplitter.addAreasToList(MapSplitte
> > > r.
> > > ja
> > > va:187)
> > > . It is failing on my system with a depth of 2342. I suggest
> > > there
> > > needs to
> > > be a maximum depth after which it should give up trying to split.
> > > 
> > > 				} else if (mustSplit) { // can't reduce
> > > size, so force more subdivisions
> > > 					log.debug("splitting area by
> > > contents", area);
> > > 					MapArea[] sublist =
> > > area.split(1, 1,
> > > bounds, true);
> > > 					addAreasToList(sublist, alist,
> > > depth
> > > + 1);
> > > 					continue;
> > > 				}
> > > 
> > > I am unsure whether you would want a fixed depth limit, would
> > > want
> > > it
> > > to be
> > > set by parameter, would prefer to catch the error or think it
> > > would
> > > be
> > > possible to see whether the proposed area split has made any
> > > improvement,
> > > and give up if it hasn't.
> > > 
> > > Regards,
> > > Mike
> > > 
> > > 
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > 
> 