[mkgmap-dev] [PATCH v5]splitter memory usage: Exception in thread "main" java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ShortFunction

Thu Nov 17 15:29:52 GMT 2011

Seems to be still some fastutil.jar problem around? I tried to use the 
splitter.jar from the mkgmap download page WITHOUT the modified 
fastutil.jar (latest as of today):
Exception in thread "main" java.lang.NoClassDefFoundError: 
it/unimi/dsi/fastutil/longs/Long2ShortFunction

It does work however if I use the modified fastutil.jar.

Just a quick speed comparison (i have 8GB RAM on my server, so RAM is 
abundant):
splitter trunk Austria: 40 seconds (however with modified fastutil.jar, 
don't think it matters on speed however).
splitter_patched with optimize-mem=33 seconds
splitter _patched without optimime-mem=34 seconds

I am right now doing some more tries on other countries (bigger ones, as 
it doesn't matter that much on too small countries where it is anyhow fast).

Here is the complete log of it failing:

c:\OpenMTBMap\maps>java -Xmx6600m -jar c:\openmtbmap\splitter.jar 
--max-nodes=1250000 --output=pbf --max-areas=128 
--geonames-file=cities15000 --description=austria --cache=cache 
--mapid=63650000 aust
ria.osm.pbf
cache=cache
description=austria
geonames-file=cities15000
legacy-mode=false
mapid=63650000
max-areas=128
max-nodes=1250000
max-threads=4 (auto)
mixed=false
no-trim=false
optimize-mem=false
output=pbf
output-dir=
overlap=2000
resolution=13
split-file=
status-freq=120
write-kml=
Elapsed time: 0s   Memory: Current 122MB (1MB used, 121MB free) Max 5866MB
Time started: Thu Nov 17 16:07:34 CET 2011
Map is being split for resolution 13:
  - area boundaries are aligned to 0x800 map units
  - areas are multiples of 0x1000 map units wide and high
Processing austria.osm.pbf
Bounding box 9.52678 46.36851 17.16273 49.02403
10'000'000 nodes processed... id=1174172078
1'000'000 ways processed... id=110462196
in 1 file
Time: Thu Nov 17 16:07:42 CET 2011
Exact map coverage is (46.36850595474243,9.526777267456055) to 
(49.0240216255188,17.162725925445557)
Trimmed and rounded map coverage is (46.40625,9.4921875) to 
(49.04296875,17.138671875)
Splitting nodes into areas containing a maximum of 1'250'000 nodes each...
Area (46.4501953125,9.4921875) to (47.2412109375,10.8984375) contains 
540'283 nodes. DONE!
Area (47.2412109375,9.4921875) to (47.7685546875,10.8984375) contains 
737'377 nodes. DONE!
Area (46.7138671875,10.8984375) to (47.2412109375,12.216796875) contains 
525'664 nodes. DONE!
Area (47.2412109375,10.8984375) to (48.1201171875,12.216796875) contains 
1'125'411 nodes. DONE!
Area (46.40625,12.216796875) to (47.4609375,13.0078125) contains 639'704 
nodes. DONE!
Area (46.40625,13.0078125) to (47.4609375,13.88671875) contains 
1'106'723 nodes. DONE!
Area (47.4609375,12.216796875) to (47.8125,13.88671875) contains 
1'112'114 nodes. DONE!
Area (47.8125,12.392578125) to (48.955078125,13.88671875) contains 
1'058'689 nodes. DONE!
Area (46.40625,13.88671875) to (47.021484375,15.205078125) contains 
754'775 nodes. DONE!
Area (47.021484375,13.88671875) to (47.8125,15.205078125) contains 
801'876 nodes. DONE!
Area (46.494140625,15.205078125) to (47.197265625,16.962890625) contains 
933'138 nodes. DONE!
Area (47.197265625,15.205078125) to (47.8125,15.908203125) contains 
568'885 nodes. DONE!
Area (47.197265625,15.908203125) to (47.8125,17.138671875) contains 
756'400 nodes. DONE!
Area (47.8125,13.88671875) to (48.1640625,15.205078125) contains 708'241 
nodes. DONE!
Area (48.1640625,13.88671875) to (48.33984375,15.205078125) contains 
1'221'308 nodes. DONE!
Area (48.33984375,13.88671875) to (49.04296875,15.205078125) contains 
768'745 nodes. DONE!
Area (47.8125,15.205078125) to (49.04296875,15.99609375) contains 
1'244'722 nodes. DONE!
Area (47.8125,15.99609375) to (48.1640625,17.138671875) contains 550'724 
nodes. DONE!
Area (48.1640625,15.99609375) to (49.04296875,17.138671875) contains 
1'039'634 nodes. DONE!
19 areas:
Area 63650000 covers (0x211000,0xad000) to (0x219000,0xc1000) AT-Graz
Area 63650001 covers (0x220000,0xad000) to (0x22e000,0xb6000) AT-Sankt 
Polten
Area 63650002 covers (0x219000,0xb5000) to (0x220000,0xc3000) HU-Szombathely
Area 63650003 covers (0x220000,0x9e000) to (0x224000,0xad000) AT-Steyr
Area 63650004 covers (0x213800,0x7c000) to (0x219800,0x8b000) IT-Bressanone
Area 63650005 covers (0x210000,0x9e000) to (0x217000,0xad000) 
AT-Klagenfurt am Woerthersee
Area 63650006 covers (0x21c000,0x8b000) to (0x220000,0x9e000) AT-Salzburg
Area 63650007 covers (0x224000,0xb6000) to (0x22e000,0xc3000) AT-Vienna
Area 63650008 covers (0x210800,0x6c000) to (0x219800,0x7c000) CH-Chur
Area 63650009 covers (0x219000,0xad000) to (0x220000,0xb5000) AT-Kapfenberg
Area 63650010 covers (0x210000,0x8b000) to (0x21c000,0x94000) 
AT-Saalfelden am Steinernen Meer
Area 63650011 covers (0x226000,0x9e000) to (0x22e000,0xad000) CZ-Ceske 
Budejovice
Area 63650012 covers (0x219800,0x7c000) to (0x223800,0x8b000) AT-Innsbruck
Area 63650013 covers (0x217000,0x9e000) to (0x220000,0xad000) AT-Leoben
Area 63650014 covers (0x220000,0x8d000) to (0x22d000,0x9e000) DE-Passau
Area 63650015 covers (0x220000,0xb6000) to (0x224000,0xc3000) SK-Bratislava
Area 63650016 covers (0x219800,0x6c000) to (0x21f800,0x7c000) DE-Kempten 
(Allgaeu)
Area 63650017 covers (0x224000,0x9e000) to (0x226000,0xad000) AT-Linz
Area 63650018 covers (0x210000,0x94000) to (0x21c000,0x9e000) AT-Villach
Writing out split osm files Thu Nov 17 16:07:42 CET 2011
Processing 19 areas in a single pass
Area 63650000 covers (0x211000,0xad000) to (0x219000,0xc1000) AT-Graz
Area 63650001 covers (0x220000,0xad000) to (0x22e000,0xb6000) AT-Sankt 
Polten
Area 63650002 covers (0x219000,0xb5000) to (0x220000,0xc3000) HU-Szombathely
Area 63650003 covers (0x220000,0x9e000) to (0x224000,0xad000) AT-Steyr
Area 63650004 covers (0x213800,0x7c000) to (0x219800,0x8b000) IT-Bressanone
Area 63650005 covers (0x210000,0x9e000) to (0x217000,0xad000) 
AT-Klagenfurt am Woerthersee
Area 63650006 covers (0x21c000,0x8b000) to (0x220000,0x9e000) AT-Salzburg
Area 63650007 covers (0x224000,0xb6000) to (0x22e000,0xc3000) AT-Vienna
Area 63650008 covers (0x210800,0x6c000) to (0x219800,0x7c000) CH-Chur
Area 63650009 covers (0x219000,0xad000) to (0x220000,0xb5000) AT-Kapfenberg
Area 63650010 covers (0x210000,0x8b000) to (0x21c000,0x94000) 
AT-Saalfelden am Steinernen Meer
Area 63650011 covers (0x226000,0x9e000) to (0x22e000,0xad000) CZ-Ceske 
Budejovice
Area 63650012 covers (0x219800,0x7c000) to (0x223800,0x8b000) AT-Innsbruck
Area 63650013 covers (0x217000,0x9e000) to (0x220000,0xad000) AT-Leoben
Area 63650014 covers (0x220000,0x8d000) to (0x22d000,0x9e000) DE-Passau
Area 63650015 covers (0x220000,0xb6000) to (0x224000,0xc3000) SK-Bratislava
Area 63650016 covers (0x219800,0x6c000) to (0x21f800,0x7c000) DE-Kempten 
(Allgaeu)
Area 63650017 covers (0x224000,0x9e000) to (0x226000,0xad000) AT-Linz
Area 63650018 covers (0x210000,0x94000) to (0x21c000,0x9e000) AT-Villach
Writing out split osm files Thu Nov 17 16:07:42 CET 2011
Processing 19 areas in a single pass
(46.494140625,15.205078125) to (47.197265625,16.962890625)
(47.8125,15.205078125) to (49.04296875,15.99609375)
(47.197265625,15.908203125) to (47.8125,17.138671875)
(47.8125,13.88671875) to (48.1640625,15.205078125)
(46.7138671875,10.8984375) to (47.2412109375,12.216796875)
(46.40625,13.88671875) to (47.021484375,15.205078125)
(47.4609375,12.216796875) to (47.8125,13.88671875)
(48.1640625,15.99609375) to (49.04296875,17.138671875)
(46.4501953125,9.4921875) to (47.2412109375,10.8984375)
(47.197265625,15.205078125) to (47.8125,15.908203125)
(46.40625,12.216796875) to (47.4609375,13.0078125)
(48.33984375,13.88671875) to (49.04296875,15.205078125)
(47.2412109375,10.8984375) to (48.1201171875,12.216796875)
(47.021484375,13.88671875) to (47.8125,15.205078125)
(47.8125,12.392578125) to (48.955078125,13.88671875)
(47.8125,15.99609375) to (48.1640625,17.138671875)
(47.2412109375,9.4921875) to (47.7685546875,10.8984375)
(48.1640625,13.88671875) to (48.33984375,15.205078125)
(46.40625,13.0078125) to (47.4609375,13.88671875)
Starting pass 1 of 1, processing 19 areas (63650000 to 63650018)
Exception in thread "main" java.lang.NoClassDefFoundError: 
it/unimi/dsi/fastutil/longs/Long2ShortFunction
         at java.lang.ClassLoader.defineClass1(Native Method)
         at java.lang.ClassLoader.defineClass(Unknown Source)
         at java.security.SecureClassLoader.defineClass(Unknown Source)
         at java.net.URLClassLoader.defineClass(Unknown Source)
         at java.net.URLClassLoader.access$100(Unknown Source)
         at java.net.URLClassLoader$1.run(Unknown Source)
         at java.net.URLClassLoader$1.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         at 
uk.me.parabola.splitter.SplitProcessor.<init>(SplitProcessor.java:87)
         at uk.me.parabola.splitter.Main.writeAreas(Main.java:371)
         at uk.me.parabola.splitter.Main.split(Main.java:193)
         at uk.me.parabola.splitter.Main.start(Main.java:121)
         at uk.me.parabola.splitter.Main.main(Main.java:110)
Caused by: java.lang.ClassNotFoundException: 
it.unimi.dsi.fastutil.longs.Long2ShortFunction
         at java.net.URLClassLoader$1.run(Unknown Source)
         at java.net.URLClassLoader$1.run(Unknown Source)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
         at java.lang.ClassLoader.loadClass(Unknown Source)
         ... 17 more