[mkgmap-dev] Patch to reduce memory usage by interning strings.

Thu Apr 1 12:10:22 BST 2010

On 31/03/10 14:56, Scott A Crosby wrote:
> I noticed that mkgmap does not intern any strings. In particular, this

This is true.  There is a pending patch that deals with excessive
memory use in a slightly different way which is on the style-speed
branch, with a pre-built one available at the bottom of the mkgmap
download page.

I drops all tags that are not used by the current style and as an
extra feature it ensures that there is only copy of all the strings
on the key side.

Could you try it out please. It may be worth also interning the values
but when I was looking at it there was much less benefit for the maps
I was looking at (but it might vary with the input).
I'd be happy to apply a patch that interned the values too if there
was a decent memory saving without a significant performance drop.

I think the point that Chris brought up applies mostly in cases where
there is a lot of contended access between threads.  If that is the
case then it won't matter much for us as reading the input files is
currently single threaded.

I can't recall why reading maps *is* single threaded. I'm not sure
there is really a problem and if there is I am sure it can be fixed
easily.  Does anyone know or can find out?

..Steve