[mkgmap-dev] New assertion, now with code-page=632 and Japan tile

Wed Nov 17 14:37:31 GMT 2021

Hi Gerd

My description didn't quite mean what I hoped it did - sorry. I was
thinking that there would be a single attempt at encoding the whole
string, and if that fails, start again but char-by-char.

But, assuming result.length() works and charBuffer.get() and
outBuff.put() maintain positions used by main encoder, within the loop
the failed component needs to be processed input char-by-char,
transliterated (if no change replaced by "?") and encoded with another
encoder.

Any variable length nature of the output charset shouldn't be a
problem. The variable length input UTF-16 will need care.

Ticker

On Wed, 2021-11-17 at 11:16 +0000, Gerd Petermann wrote:
> Hi Ticker,
> 
> remember that cs932 is a double-byte character set.
> With your code only a few unmappable utf-16 characters are replaced,
> for the rest one of cs932 is used, but without any good reason. The
> result is typically garbage.
> 
> I've modified the patch to replace any  unmappable character that was
> not transliterated by '?' .
> I've also attached a debug version that shows what goes on.
> A possible change in SparseTransliterator would be to add a mapping
> for the MATH MINUS, the other FULLWIDTH digits are supported in
> cs932.
> 
> Gerd