[mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Wed Dec 1 17:07:09 GMT 2021

It's still crashing also with patch v2.
The full map of Europe serves one need - it is simpler than say creating 5
maps of Europe with an Overlap. But yes I will simply create the continent
maps of Europe and Asia without split-name-index (Asia has not crashed yet
- but is its growing quickly).

On Wed, 1 Dec 2021 at 17:44, Felix Hartmann <extremecarver at gmail.com> wrote:

> yes it is some sort of overflow - if I do the map in two runs (1500 tiles
> approx)
> with 65500*
> and 65501*
> it compiles fine.
> That's why I use a pretty long exclusion list - but this is not enough.
> The index without split-name-index is about half the size. I think the
> first list of options only matters on the actual mkgmap creation run - and
> does not change anything anymore on the second step of creating the address
> index. Maybe I missed a couple - but most should be excluded.
>
> --mdr7-del=pri;sec;ter;cw;min;unsf,uncl;living;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;bridge;swing;aqueduct;viaduct;"ntnl
> bndry";admin_lv=4;"ntnl
> park";weir;dam;waterfall;rapids;stream;river;drain;ditch;"huge
> river";"hugest river";"medium river";"small river";"wide river";"hugest
> canal";"huge canal";"medium canal";"small canal";"big
> canal";canal;snowpark;Skitour-descent;Skipiste;Skislope;illuminated;Nordic-Skiing;skitour;platter;ropetow;tbar;chairlift;goods_lift;runway;taxiway;apron;VFS0;VFS1;VFS2;VFS3;VFS4;VFS5;VFS6;VFS7;usf;cstrn;steps;parking_aisle;res;cob;cobstn;public_footway;public_footpath
> --mdr7-excl=pri;sec;ter;cw;min;unsf,uncl;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;"ntnl
> bndry"
>
> On Wed, 1 Dec 2021 at 17:39, Gerd Petermann <
> gpetermann_muenchen at hotmail.com> wrote:
>
>> Hi Felix,
>>
>> your index is probably simply too large (> 2GB).
>> The attached patch fixes another possible overflow, but I am not sure if
>> such a file size is allowed for the index.
>>
>> Maybe try to remove specific items from the index (see --mdr7-excl,
>> --mdr7-del)
>> I think the reason for the huge size is your style which combines a lot
>> of information in the road name, e.g. "102 Thw-Göttinger Weg T2 Pth
>> Zentralalpenweg 02 Hr"
>> or "137 Mindener Jubiläumsweg T2 Pth Mindener Jubiläumsweg Hr"
>> (found these in omtb_austria_13.05.2021)
>> All this information is stored in the PC index (a string table), and with
>> --split-name-index lots of entries are produced for strings like "Pth" or
>> "T2" or "Hr"
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
>> Felix Hartmann <extremecarver at gmail.com>
>> Gesendet: Mittwoch, 1. Dezember 2021 17:16
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Exception in thread "main"
>> java.lang.IllegalArgumentException . on Europe extract
>>
>> I found out that if I remove --split-name-index then it compiles fine.
>> I'm just trying a minimal setup option wise and could look if it's a
>> special .img tile or not.
>> I could upload all map tiles somewhere for further analysis - or try some
>> more things before doing that (12GB data or so).
>>
>> On Wed, 1 Dec 2021 at 13:08, Felix Hartmann <extremecarver at gmail.com
>> <mailto:extremecarver at gmail.com>> wrote:
>> Hi Gerd,
>>
>> here is the log - hope you can find out something with it - but I think
>> it is missing the problem - there are only 3 lines before the crash - all
>> else is many minutes before.
>>
>> 2021/12/01 12:55:08 INFO (ImgFS): bs=512, whole size=-1789952000,
>> hb=-14501, fb=-3481499, blocks=-3496000
>>
>> 2021/12/01 12:55:08 INFO (ImgFS): bs=1024, whole size=-1786235904,
>> hb=-3623, fb=-1740748, blocks=-1744371
>>
>> 2021/12/01 12:55:08 INFO (ImgFS): Best block size: 512
>> sizeInBlocks=-3496000, reserved=-14501
>>
>>
>> On Wed, 1 Dec 2021 at 08:57, Gerd Petermann <
>> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
>> wrote:
>> Hi Felix,
>>
>> hm, in that case one of the two factors is already negative.
>> You can enable logging and run the combiner step again (no need to
>> compile the tiles again)
>> Add this to the log config:
>> uk.me.parabola.imgfmt.sys.ImgFS.level=FINE
>>
>> BTW: The index contains not only address search data, also POI.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
>> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
>> Gesendet: Mittwoch, 1. Dezember 2021 02:27
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Exception in thread "main"
>> java.lang.IllegalArgumentException . on Europe extract
>>
>> Hi Gerd,
>>
>> thanks for the quick response. Any other way I can try to get more debug
>> information?
>>
>> I tried with the patch but I still get:
>> gmapi-minimal: Skipping file
>> E:\OpenMTBMap\contourlines20\europe\75500448.img
>> Exception in thread "main" java.lang.IllegalArgumentException
>>         at java.base/sun.nio.ch<http://sun.nio.ch
>> >.FileChannelImpl.position(FileChannelImpl.java:355)
>>         at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
>>         at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
>>         at
>> uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
>>         at
>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
>>
>> On Tue, 30 Nov 2021 at 14:48, Gerd Petermann <
>> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
>> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
>> gpetermann_muenchen at hotmail.com>>> wrote:
>> Hi Felix,
>>
>> might be an overflow. The code line is
>> file.position(param.getReservedDirectoryBlocks() * param.getBlockSize());
>>
>> so two ints are multiplied.
>>
>> Please try if the attached patch changes something.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
>> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>> mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <
>> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
>> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
>> Gesendet: Dienstag, 30. November 2021 13:23
>> An: Development list for mkgmap
>> Betreff: [mkgmap-dev] Exception in thread "main"
>> java.lang.IllegalArgumentException . on Europe extract
>>
>> Using mkgmap version 4808 - when compiling Europe. It went through fine
>> on all geofabrik country extracts.
>> I then upgraded to mkgmap 4819 and yesterday downloaded a new europe
>> extract (4 days newer) from geofabrik to see if I can reproduce it again -
>> and the error message is identical.
>> I'm going to run with default style now again - to see if it makes a
>> difference.
>>
>> C:\openmtbmap\mkgmap.jar --max-jobs=11 --order-by-decreasing-area
>> "--generate-sea" --code-page=65001
>> "--precomp-sea=E:\OpenMTBMap\osmpbf_geofabrik\sea.zip"
>> "--style-file=C:\openmtbmap\openmtbmap_style"
>> --add-boundary-nodes-at-admin-boundaries --fix-roundabout-direction
>> --improve-overview --drive-on=detect --allow-reverse-merge --lower-case
>> --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18"
>> --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12"
>> --add-pois-to-areas
>> --pois-to-areas-placement=entrance=main;entrance=yes;building=entrance;barrier=entrance
>> --simplify-lines=23:2.6,22:4.2,21:5.4,20:6,19:7,18:7.5,17:4.5,16:5,15:5,14:6
>> --simplify-polygons=23:3.6,22:7,21:6,20:9,17:5.4
>> --add-boundary-nodes-at-admin-boundaries=2 --cycle-map
>> --ignore-fixme-values --housenumbers
>> --road-name-config=C:\openmtbmap\openmtbmap_svn\roadNameConfig.txt
>> --split-name-index --link-pois-to-ways --ignore-turn-restrictions
>> --polygon-size-limits="24:12, 23:14, 22:14, 21:20, 20:20, 19:20, 18:20
>>  , 17:20, 16:20, 15:20, 14:20, 13:25" --description=omtb_eu
>> --show-profiles=1  --location-autofill=bounds,is_in,nearest
>> --bounds=E:\OpenMTBMap\osmpbf_geofabrik\bounds.zip --route
>> --country-abbr=eu --country-name=europe --mapname=65500000 --family-id=6550
>> --product-id=1 --series-name=omtb_europe_29.11.2021_UC_LOCAL
>> --family-name=mtb_eu_29.11.2021_UC_LOCAL --tdbfile --x-gmapi-minimal
>> --overview-mapname=mapsetc --keep-going
>> --area-name="europe_29.11.2021_UC_LOCAL_omtb" -c
>> D:\openmtbmap\maps\template.europe
>> E:\OpenMTBMap\contourlines20\europe\7*.img typeu.TYP  1>NUL
>>
>>
>> Number of MapFailedExceptions: 0
>> gmapi-minimal: Writing freshly compiled file
>> C:\openmtbmap\maps\.\65500000.img
>> gmapi-minimal: Writing freshly compiled file
>> C:\openmtbmap\maps\.\65500001.img
>> gmapi-minimal: Writing freshly compiled file
>> C:\openmtbmap\maps\.\65500002.img
>> gmapi-minimal: Writing freshly compiled file
>> C:\openmtbmap\maps\.\65500003.img
>> gmapi-minimal: Writing freshly compiled file
>> C:\openmtbmap\maps\.\65500004.img
>> ................
>> skipped
>> .....................
>> long list of contourline files:
>> .........
>> gmapi-minimal: Skipping file
>> E:\OpenMTBMap\contourlines20\europe\75500447.img
>> WARNING (global): Input file
>> E:\OpenMTBMap\contourlines20\europe\75500448.img has different code page
>> 1252
>> WARNING (global): Input file
>> E:\OpenMTBMap\contourlines20\europe\75500448.img has different charset type
>> 10
>> WARNING (global): Input files have different code pages
>> gmapi-minimal: Skipping file
>> E:\OpenMTBMap\contourlines20\europe\75500448.img
>> .... 448 is the highest number contourlines file
>>
>> ............And here comes the error
>> it happens on writing the address index mdr file - mapset.img and
>> mapset.tdb have already been created.
>>
>> Exception in thread "main" java.lang.IllegalArgumentException
>>         at java.base/sun.nio.ch<http://sun.nio.ch><http://sun.nio.ch
>> >.FileChannelImpl.position(FileChannelImpl.java:355)
>>         at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
>>         at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
>>         at
>> uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
>>         at
>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
>>
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
>> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
>> mkgmap-dev at lists.mkgmap.org.uk>>
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20211201/57847610/attachment-0001.html>