[mkgmap-dev] Patch to reduce memory usage by interning strings.

Fri Apr 2 04:47:59 BST 2010

On Wed, 31 Mar 2010 23:58:45 +0200, WanMil <wmgcnfg at web.de> writes:

> Am 31.03.2010 22:10, schrieb Scott A Crosby:
>> On Wed, 31 Mar 2010 21:13:49 +0200, WanMil<wmgcnfg at web.de>  writes:
>>

> my patch interned all keys and additionally the values of a limited
> number of keys. Maybe it's not necessary to limit the interning of
> values. So I have attached the very simple but hopefully very
> effective patch regarding the memory footprint of mkgmap.

My opinion is that its simpler and more robust to intern or
pseudointern every tag value. The bad tile had a lot of duplicate
values, what if those tags were not on your list of the 'only intern
value strings for some keys'?

> Regarding your patch: I don't understand the function of the
> FuzzyIntern class. You build a HashMap from (uninterned) Strings to
> the interned String. Then you are looking up new strings in this
> HashMap and use the interned variant. Where's the difference to the
> (hopefully) very performance optimized intern() method?

Note that this code is not actually interning any strings in with
String.intern(). Call it psuedointerning. The purpose of FuzzyIntern
was when I believed that String.intern() interned forever, which I
considered very undesirable. I wanted semantics that would remove
(most) duplicate strings in memory without forcing those strings to
remain in RAM forever.

>> String intern does not intern forever

> I didn't know that. Do you have any link where this is specified?

A google search 'java intern weak reference' seems to indicate that
Java since version 1.2 uses a weak reference table for string intern,
which means that they can be removed. That search offers alternative
implementation ideas, such as using a real weak reference hashtable.

Scott