[mkgmap-dev] Problem with splitter: Failed to find a correct split

Fri Mar 25 06:38:50 GMT 2016

I have the same problem , with splitter for australia , or
australia-oceania

even when i have search limit at 60,000 or 90,000  or in between these
 Figures
I stil get the same response in splitter log file .

Stephen

On Fri, Mar 25, 2016 at 4:33 PM, Gerd Petermann <
GPetermann_muenchen at hotmail.com> wrote:

> Hi Patrik,
>
>
> good question. I also wondered if splitter can be changed to ignore wrong
> bounds
>
> tags. My thoughts:
>
> In your case the bbox is far too large, means it claims to contain data
> that is not there.
>
> The normal case is vice versa: The file contains some nodes outside of the
> bbox,
>
> maybe from ferry lines or other long ways, and my understanding is that we
> don't
>
> want to calculate the tiles based on those few nodes, since we might get a
> map with
>
> larger empty areas at the "border". No idea if that would cause more
> trouble.
>
> With typical data from geofabrik and the --no-trim option there will
> always be large
>
> empty areas as most countries are not rectangular ;-)
>
>
> Possible changes in the code:
>
> 1) add a --ignore-osm-bounds option and set the default  to false  (means
> one gets
>
> the same result like now when he uses the default)
>
> 2) add a --use-osm-bounds option and set the default  to false
>
> (means one might get a different result when using the default)
>
> 3) add code to check how the collected data matches the given bounds,
>
> use whatever is smaller. This might also be triggered by an option if
> needed.
>
>
> I'd like to get some feedback from others first.
>
>
> Gerd
>
>
> ------------------------------
> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> KeenOnKites <keenonkites at gmx.net>
> *Gesendet:* Freitag, 25. März 2016 00:13
> *An:* mkgmap-dev at lists.mkgmap.org.uk
> *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a
> correct split
>
> Gerd,
>
> I couldn't go to sleep without getting to the ground of this...
>
> I performed the following tests on linux, always with my standard
> --no-trim present:
>
>    - original osm.pbf file downloaded from geofabrik (with the
>    problematic bounds tag in it)
>    -> FAILURE
>    - converted with the help of osmconvert to normal osm format
>    -> FAILURE (which was to be expected, same file, just different
>    format)
>    - deleted the osm tag with a simple command like 'cat file.osm | grep
>    -v bounds > file-nobounds.osm' and ran splitter against it, always with the
>    same options
>    -> SUCCESS
>
> This leads me to the question why the bounds tag is read/used if it also
> works without and even gives less problem ?
>
>
> .... sorry for bombarding you (and the list), but I think it's
> nevertheless and interesting question.
>
>
> Cheers
> Patrik
>
> On 24.03.2016 23:51, KeenOnKites wrote:
>
> Gerd,
>
> I did dig a bit deeper... also as it rang a bell:
> we had quite a similar problem with the wrong bounding box in Alaska
> already October 2014. It was an illegal value of maxlon="180.0005" causing
> splitter to bail out
>
> When I convert the osm.pbf file with the help of osmconvert to osm and
> look at the first few lines I see a 'bounds' tag announcing the problematic
> bounding box (not illegal as in 2014, but still 'suboptimal'):
>
> <?xml version='1.0' encoding='UTF-8'?>
> <osm version="0.6" generator="osmconvert 0.8.2"
> timestamp="2016-03-23T20:13:02Z">
>         <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794"
> maxlon="179.9999"/>
>         <node id="27207079" lat="64.7487541" lon="-147.3242821"
> version="2"
> ...
>
>
> Getting the statistics via osmconvert with --out-statistics seems to read
> through the file and checks the 'real' bounding box:
>
> ...
> lon min: -180.0000000
> lon max: -122.5122525
> lat min: 48.6234931
> lat max: 71.6061501
> ...
>
>
> I'm now wondering if splitter get's confused by the existing but obviously
> strange bounds tag.
>
> According to the findings in 2014 splitter can handle files without the
> bounds tag and just gets the real boundings from all the elements in the
> file.
> I did not test to run it through splitter without the bounds tag as I'm
> having troubles converting the file properly on my windows... but I'll try
> that probably tomorrow again on linux (sort of late already today).
> The process would be
>
> osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to
> have same source file format)
> .... and then run the splitter and see what happens.
>
> But if I have a proper osm file with the 'problematic' bounds tag in it, I
> can also try to reproduce the problem with the osm file. If it is
> reproducible I just drop the tag and try it again.
>
> BTW: I've contacted geofabrik already via email
>
> Patrik
>
>
> On 24.03.2016 22:27, KeenOnKites wrote:
>
> Gerd,
>
> I'll play a bit more with this option and check what suits me best.
>
> Again thanks for the incredibly quick answer to my problem/question.
>
> Cheers
> Patrik
>
> On 24.03.2016 22:20, Gerd Petermann wrote:
>
> Hi Patrik,
>
>
> I think the wanted effect of the no-trim option is a rectangular map,
>
> which some people prefer, esp. on the PC.
>
>
> Gerd
>
> ------------------------------
> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von KeenOnKites
> <keenonkites at gmx.net> <keenonkites at gmx.net>
> *Gesendet:* Donnerstag, 24. März 2016 22:16
> *An:* mkgmap-dev at lists.mkgmap.org.uk
> *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a
> correct split
>
> Gerd,
>
> Sorry, your explanations came in during I was writing up the test results
> ...
>
> Think it's all clear so far.
>
> As we're creating lot of different maps I'm just wondering if I can/should
> drop the option --no-trim for all map building or if I would suddenly run
> into other/new problems...
>
> I'll contact geofabrik with the details.
>
> Many thanks and happy Easter.
> Patrik
>
> On 24.03.2016 22:07, Gerd Petermann wrote:
>
> Hi Patrik,
>
>
> I don't need the files, I downloaded the alaska file and tried some
> variants.
>
> See also my last post, send a few minutes ago.
>
>
> Gerd
>
>
> ------------------------------
> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Patrik Brunner
> <patrik.brunner at gmx.net> <patrik.brunner at gmx.net><patrik.brunner at gmx.net>
> <patrik.brunner at gmx.net>
> *Gesendet:* Donnerstag, 24. März 2016 22:03
> *An:* mkgmap-dev at lists.mkgmap.org.uk
> *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a
> correct split
>
> Gerd,
>
> Yes, alaska.osm.pbf is small.
>
> It works without --no-trim. And it also works with the polygon file that
> belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according
> to documentation, anyway would disable --no-trim option.
>
> Do you still need the resulting densities-out of the failure ? It's about
> 700 kb... if yes, how should I provide it ? Just attach it here ?
>
> You have to excuse my question, I'm not the crack: is this now a problem
> of the pbf file, or the splitter ? ... or just the way I try to use it ?
>
> Thanks already now for your help.
> Patrik
>
> On 24.03.2016 21:14, Gerd Petermann wrote:
>
>
> Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.
>
> The problem is here is that the bounding box spans from -180 to 180,
>
> but this box is most empty. I have to run splitter now to find the details.
>
> It works without --no-trim, probably also with an appropriate polygon file.
>
> Does that help?
>
>
> Gerd
> ------------------------------
> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann
> <GPetermann_muenchen at hotmail.com><GPetermann_muenchen at hotmail.com>
> <GPetermann_muenchen at hotmail.com>
> *Gesendet:* Donnerstag, 24. März 2016 21:01
> *An:* Development list for mkgmap
> *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a
> correct split
>
>
> Hi Patrik,
>
>
> please provide the complete log from splitter and the densities-out.txt
>
> Gerd
>
> ------------------------------
> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk>
> <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von KeenOnKites
> <keenonkites at gmx.net><keenonkites at gmx.net> <keenonkites at gmx.net>
> *Gesendet:* Donnerstag, 24. März 2016 20:25
> *An:* Development list for mkgmap
> *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct
> split
>
> Hello together,
>
> I'm running into a problem with the splitter (r435 aswell as r427) when
> splitting the US_ALASKA file downloadable from Geofabrik.
>
> The exception is:
>
> Warning: No solution found for partition (49.7900390625,-179.9560546875)
> to (73.828125,180.0) with 6'702'717 nodes
> uk.me.parabola.splitter.SplitFailedException: Failed to find a correct
> split
>         at
> uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152)
>         at
> uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196)
>         at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645)
>         at uk.me.parabola.splitter.Main.split(Main.java:258)
>         at uk.me.parabola.splitter.Main.start(Main.java:187)
>         at uk.me.parabola.splitter.Main.main(Main.java:157)
>
>
> The complete command line with the splitter call is:
>
> java -Xmx1536M -jar
> D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar
> --max-threads=2
> --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c
> ities15000.zip --no-trim
> --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea
> --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml
> --output-dir=D:/fzk/develop/fzk
> -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA
> D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf
>
>
> The pbf source file comes from:
>
> <http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf>
> http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf
>
>
> The osmconvert statistics about that file is:
>
> PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe
> .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf
> --out-statistics
> timestamp min: 2007-06-05T03:23:59Z
> timestamp max: 2016-03-23T05:41:43Z
> lon min: -180.0000000
> lon max: -122.5122525
> lat min: 48.6234931
> lat max: 71.6061501
> nodes: 4360214
> ways: 185550
> relations: 2245
> node id min: 27207079
> node id max: 4072166815
> way id min: 4708608
> way id max: 404980503
> relation id min: 13971
> relation id max: 6033189
> keyval pairs max: 310
> keyval pairs max object: relation 60189
> noderefs max: 2000
> noderefs max object: way 42394334
> relrefs max: 681
> relrefs max object: relation 3337277
> PS Freizeitkarte-Entwicklung>
>
> Interesting to mention:
> - splitter exception mentiones a complete different coverage area than
> osmconvert statistics.
> - the area is near -180 / +180... always complicated.
>
> Do I miss something ? All other pbf's I've tried are splitting properly
> without any problems. Do I need to change something in the arguments ? Or
> is it a simple problem of the actual pbf file ?
>
> Any ideas are very welcome....
>
> Cheers
> Patrik
>
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing listmkgmap-dev at lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing listmkgmap-dev at lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing listmkgmap-dev at lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing listmkgmap-dev at lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing listmkgmap-dev at lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20160325/1935d79f/attachment-0001.html>