<div dir="ltr">Oh yeah the best solution would be if mkgmap kinda could create 4 maps of Europe from all the Europe .img tiles with polygons/boxes created manually/then provided as input file..<div><br></div><div>Splitting before and creating those 4 maps from scratch takes a lot more of compilation time than the full Europe map alone.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 1 Dec 2021 at 18:07, Felix Hartmann <<a href="mailto:extremecarver@gmail.com">extremecarver@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">It's still crashing also with patch v2. <div>The full map of Europe serves one need - it is simpler than say creating 5 maps of Europe with an Overlap. But yes I will simply create the continent maps of Europe and Asia without split-name-index (Asia has not crashed yet - but is its growing quickly).</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 1 Dec 2021 at 17:44, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">yes it is some sort of overflow - if I do the map in two runs (1500 tiles approx) <div>with 65500* </div><div>and 65501* </div><div>it compiles fine.</div><div>That's why I use a pretty long exclusion list - but this is not enough. The index without split-name-index is about half the size. I think the first list of options only matters on the actual mkgmap creation run - and does not change anything anymore on the second step of creating the address index. Maybe I missed a couple - but most should be excluded.</div><div><br></div><div>--mdr7-del=pri;sec;ter;cw;min;unsf,uncl;living;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;bridge;swing;aqueduct;viaduct;"ntnl bndry";admin_lv=4;"ntnl park";weir;dam;waterfall;rapids;stream;river;drain;ditch;"huge river";"hugest river";"medium river";"small river";"wide river";"hugest canal";"huge canal";"medium canal";"small canal";"big canal";canal;snowpark;Skitour-descent;Skipiste;Skislope;illuminated;Nordic-Skiing;skitour;platter;ropetow;tbar;chairlift;goods_lift;runway;taxiway;apron;VFS0;VFS1;VFS2;VFS3;VFS4;VFS5;VFS6;VFS7;usf;cstrn;steps;parking_aisle;res;cob;cobstn;public_footway;public_footpath --mdr7-excl=pri;sec;ter;cw;min;unsf,uncl;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;"ntnl bndry"<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 1 Dec 2021 at 17:39, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Felix,<br>
<br>
your index is probably simply too large (> 2GB).<br>
The attached patch fixes another possible overflow, but I am not sure if such a file size is allowed for the index.<br>
<br>
Maybe try to remove specific items from the index (see --mdr7-excl, --mdr7-del)<br>
I think the reason for the huge size is your style which combines a lot of information in the road name, e.g. "102 Thw-Göttinger Weg T2 Pth Zentralalpenweg 02 Hr"<br>
or "137 Mindener Jubiläumsweg T2 Pth Mindener Jubiläumsweg Hr"<br>
(found these in omtb_austria_13.05.2021)<br>
All this information is stored in the PC index (a string table), and with --split-name-index lots of entries are produced for strings like "Pth" or "T2" or "Hr"<br>
<br>
Gerd<br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>> im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><br>
Gesendet: Mittwoch, 1. Dezember 2021 17:16<br>
An: Development list for mkgmap<br>
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract<br>
<br>
I found out that if I remove --split-name-index then it compiles fine. I'm just trying a minimal setup option wise and could look if it's a special .img tile or not.<br>
I could upload all map tiles somewhere for further analysis - or try some more things before doing that (12GB data or so).<br>
<br>
On Wed, 1 Dec 2021 at 13:08, Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>> wrote:<br>
Hi Gerd,<br>
<br>
here is the log - hope you can find out something with it - but I think it is missing the problem - there are only 3 lines before the crash - all else is many minutes before.<br>
<br>
2021/12/01 12:55:08 INFO (ImgFS): bs=512, whole size=-1789952000, hb=-14501, fb=-3481499, blocks=-3496000<br>
<br>
2021/12/01 12:55:08 INFO (ImgFS): bs=1024, whole size=-1786235904, hb=-3623, fb=-1740748, blocks=-1744371<br>
<br>
2021/12/01 12:55:08 INFO (ImgFS): Best block size: 512 sizeInBlocks=-3496000, reserved=-14501<br>
<br>
<br>
On Wed, 1 Dec 2021 at 08:57, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>> wrote:<br>
Hi Felix,<br>
<br>
hm, in that case one of the two factors is already negative.<br>
You can enable logging and run the combiner step again (no need to compile the tiles again)<br>
Add this to the log config:<br>
uk.me.parabola.imgfmt.sys.ImgFS.level=FINE<br>
<br>
BTW: The index contains not only address search data, also POI.<br>
<br>
Gerd<br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>> im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>><br>
Gesendet: Mittwoch, 1. Dezember 2021 02:27<br>
An: Development list for mkgmap<br>
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract<br>
<br>
Hi Gerd,<br>
<br>
thanks for the quick response. Any other way I can try to get more debug information?<br>
<br>
I tried with the patch but I still get:<br>
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500448.img<br>
Exception in thread "main" java.lang.IllegalArgumentException<br>
at java.base/<a href="http://sun.nio.ch" rel="noreferrer" target="_blank">sun.nio.ch</a><<a href="http://sun.nio.ch" rel="noreferrer" target="_blank">http://sun.nio.ch</a>>.FileChannelImpl.position(FileChannelImpl.java:355)<br>
at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)<br>
at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)<br>
at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)<br>
at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)<br>
at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)<br>
at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)<br>
at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)<br>
<br>
On Tue, 30 Nov 2021 at 14:48, Gerd Petermann <<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a><mailto:<a href="mailto:gpetermann_muenchen@hotmail.com" target="_blank">gpetermann_muenchen@hotmail.com</a>>>> wrote:<br>
Hi Felix,<br>
<br>
might be an overflow. The code line is<br>
file.position(param.getReservedDirectoryBlocks() * param.getBlockSize());<br>
<br>
so two ints are multiplied.<br>
<br>
Please try if the attached patch changes something.<br>
<br>
Gerd<br>
<br>
________________________________________<br>
Von: mkgmap-dev <<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk" target="_blank">mkgmap-dev-bounces@lists.mkgmap.org.uk</a>>>> im Auftrag von Felix Hartmann <<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a><mailto:<a href="mailto:extremecarver@gmail.com" target="_blank">extremecarver@gmail.com</a>>>><br>
Gesendet: Dienstag, 30. November 2021 13:23<br>
An: Development list for mkgmap<br>
Betreff: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract<br>
<br>
Using mkgmap version 4808 - when compiling Europe. It went through fine on all geofabrik country extracts.<br>
I then upgraded to mkgmap 4819 and yesterday downloaded a new europe extract (4 days newer) from geofabrik to see if I can reproduce it again - and the error message is identical.<br>
I'm going to run with default style now again - to see if it makes a difference.<br>
<br>
C:\openmtbmap\mkgmap.jar --max-jobs=11 --order-by-decreasing-area "--generate-sea" --code-page=65001 "--precomp-sea=E:\OpenMTBMap\osmpbf_geofabrik\sea.zip" "--style-file=C:\openmtbmap\openmtbmap_style" --add-boundary-nodes-at-admin-boundaries --fix-roundabout-direction --improve-overview --drive-on=detect --allow-reverse-merge --lower-case --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --add-pois-to-areas --pois-to-areas-placement=entrance=main;entrance=yes;building=entrance;barrier=entrance --simplify-lines=23:2.6,22:4.2,21:5.4,20:6,19:7,18:7.5,17:4.5,16:5,15:5,14:6 --simplify-polygons=23:3.6,22:7,21:6,20:9,17:5.4 --add-boundary-nodes-at-admin-boundaries=2 --cycle-map --ignore-fixme-values --housenumbers --road-name-config=C:\openmtbmap\openmtbmap_svn\roadNameConfig.txt --split-name-index --link-pois-to-ways --ignore-turn-restrictions --polygon-size-limits="24:12, 23:14, 22:14, 21:20, 20:20, 19:20, 18:20<br>
, 17:20, 16:20, 15:20, 14:20, 13:25" --description=omtb_eu --show-profiles=1 --location-autofill=bounds,is_in,nearest --bounds=E:\OpenMTBMap\osmpbf_geofabrik\bounds.zip --route --country-abbr=eu --country-name=europe --mapname=65500000 --family-id=6550 --product-id=1 --series-name=omtb_europe_29.11.2021_UC_LOCAL --family-name=mtb_eu_29.11.2021_UC_LOCAL --tdbfile --x-gmapi-minimal --overview-mapname=mapsetc --keep-going --area-name="europe_29.11.2021_UC_LOCAL_omtb" -c D:\openmtbmap\maps\template.europe E:\OpenMTBMap\contourlines20\europe\7*.img typeu.TYP 1>NUL<br>
<br>
<br>
Number of MapFailedExceptions: 0<br>
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500000.img<br>
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500001.img<br>
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500002.img<br>
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500003.img<br>
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500004.img<br>
................<br>
skipped<br>
.....................<br>
long list of contourline files:<br>
.........<br>
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500447.img<br>
WARNING (global): Input file E:\OpenMTBMap\contourlines20\europe\75500448.img has different code page 1252<br>
WARNING (global): Input file E:\OpenMTBMap\contourlines20\europe\75500448.img has different charset type 10<br>
WARNING (global): Input files have different code pages<br>
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500448.img<br>
.... 448 is the highest number contourlines file<br>
<br>
............And here comes the error<br>
it happens on writing the address index mdr file - mapset.img and mapset.tdb have already been created.<br>
<br>
Exception in thread "main" java.lang.IllegalArgumentException<br>
at java.base/<a href="http://sun.nio.ch" rel="noreferrer" target="_blank">sun.nio.ch</a><<a href="http://sun.nio.ch" rel="noreferrer" target="_blank">http://sun.nio.ch</a>><<a href="http://sun.nio.ch" rel="noreferrer" target="_blank">http://sun.nio.ch</a>>.FileChannelImpl.position(FileChannelImpl.java:355)<br>
at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)<br>
at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)<br>
at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)<br>
at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)<br>
at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)<br>
at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)<br>
at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)<br>
<br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>>><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><mailto:<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a>><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
<br>
<br>
--<br>
Felix Hartman - Openmtbmap.org & VeloMap.org<br>
<br>
_______________________________________________<br>
mkgmap-dev mailing list<br>
<a href="mailto:mkgmap-dev@lists.mkgmap.org.uk" target="_blank">mkgmap-dev@lists.mkgmap.org.uk</a><br>
<a href="https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev" rel="noreferrer" target="_blank">https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Felix Hartman - Openmtbmap.org & VeloMap.org<br></div><br></div></div></div></div></div></div>