[mkgmap-dev] Error in memory calculation

Wed Feb 5 21:33:00 GMT 2020

Hi Manfred,

the calculation is only done when you don't use the --max-jobs option and when more than two input files are processed.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von DD8KQ <dd8kq at gmx.de>
Gesendet: Mittwoch, 5. Februar 2020 21:49
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] Error in memory calculation

Hi Gerd

just did a small map job but on the console there is no print about
number of max jobs. Is there a special log flag to get this print out ?
What i've seen in the task manager while building the map, all 8 cpu's
are used, but i don't know anything about the number of jobs ...

Am 05.02.2020 um 21:35 schrieb Gerd Petermann:
> Hi Manfred,
>
> mkgmap contains code to estimate a suitable value for max-jobs if this parameter isn't set. This code fails and the result is that it believes that a first thread needed all available memory. This doesn't happen when the -Xms parameter in my sample call is removed, and it also doesn't happen with JRE 1.8 (both from OpenJDK and Oracle). I've not yet started to look at the details returned in class java.lang.management.MemoryPoolMXBean
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von DD8KQ <dd8kq at gmx.de>
> Gesendet: Mittwoch, 5. Februar 2020 21:16
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] Error in memory calculation
>
> Hi Gerd
>
> i'm not sure what you are looking for. In my build environment i'm using
> java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable
> my command line to build is like this :
>
> java -Xmx30000m -XX:-UseGCOverheadLimit -ea -jar
> -Dlog.config=logging.properties "c:\Program Files
> (x86)\KartenTools\mkgmap\mkgmap.jar"  ....
>
> and depending on which map i want to build, the occupied memory runs up
> nearly to 30Gb
>
> Am 05.02.2020 um 14:05 schrieb Ticker Berkin:
>> Hi Gerd
>>
>> I don't have a 64bit environment or that much memory. In my normal
>> environment, I get:
>>
>> $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \
>>     dummy.osm dummy.osm dummy.osm
>> Time started: Wed Feb 05 12:56:28 GMT 2020
>> Setting max-jobs to 4
>> Number of MapFailedExceptions: 0
>> Number of ExitExceptions: 0
>> Time finished: Wed Feb 05 12:56:29 GMT 2020
>> Total time taken: 1 second
>>
>> -Xmx1540M is the maximum value that allows Java to start up on my
>> machine.
>>
>> With a newer version I loaded in October, but it didn't have any
>> obvious advantages:
>>
>> $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \
>>     -jar ../mkgmap.trunk/mkgmap.jar \
>>     dummy.osm dummy.osm dummy.osm
>> Time started: Wed Feb 05 12:58:29 GMT 2020
>> Setting max-jobs to 4
>> Number of MapFailedExceptions: 0
>> Number of ExitExceptions: 0
>> Time finished: Wed Feb 05 12:58:30 GMT 2020
>> Total time taken: 1 second
>>
>> Ticker
>>
>> On Fri, 2020-01-31 at 16:15 +0000, Gerd Petermann wrote:
>>> please try this command with openjdk 11 or 13 64Bit:
>>> java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm
>>> f:\osm\dummy.osm
>>> This is what I get:
>>> Time started: Fri Jan 31 17:12:53 CET 2020
>>> Setting max-jobs to 1
>>> Number of MapFailedExceptions: 0
>>> To reduce the run time, consider increasing the amnount of memory
>>> available for use by mkgmap by using the Java -Xmx flag to set the
>>> memory to more than 41000 MB, providing this is less than the amount
>>> of physical memory installed.
>>> Number of ExitExceptions: 0
>>> Time finished: Fri Jan 31 17:12:54 CET 2020
>>> Total time taken: 1 second
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> --
>
> #####################################################
>
>        Viele Grüße und 73 de Manfred Haiduk, DD8KQ
>        e-mail mhaiduk at t-online.de dd8kq at gmx.de
>
> #####################################################
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--

#####################################################

      Viele Grüße und 73 de Manfred Haiduk, DD8KQ
      e-mail mhaiduk at t-online.de dd8kq at gmx.de

#####################################################

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev