[mkgmap-dev] New assertion, now with code-page=632 and Japan tile

Wed Nov 17 16:00:54 GMT 2021

Hi Ticker,

result.length() works and most times returns 1, sometimes higher values for unicode characters which cannot be represented by a single char.

OK to commit v2?
Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
Gesendet: Mittwoch, 17. November 2021 15:37
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] New assertion, now with code-page=632 and Japan tile

Hi Gerd

My description didn't quite mean what I hoped it did - sorry. I was
thinking that there would be a single attempt at encoding the whole
string, and if that fails, start again but char-by-char.

But, assuming result.length() works and charBuffer.get() and
outBuff.put() maintain positions used by main encoder, within the loop
the failed component needs to be processed input char-by-char,
transliterated (if no change replaced by "?") and encoded with another
encoder.

Any variable length nature of the output charset shouldn't be a
problem. The variable length input UTF-16 will need care.

Ticker

On Wed, 2021-11-17 at 11:16 +0000, Gerd Petermann wrote:
> Hi Ticker,
>
> remember that cs932 is a double-byte character set.
> With your code only a few unmappable utf-16 characters are replaced,
> for the rest one of cs932 is used, but without any good reason. The
> result is typically garbage.
>
> I've modified the patch to replace any  unmappable character that was
> not transliterated by '?' .
> I've also attached a debug version that shows what goes on.
> A possible change in SparseTransliterator would be to add a mapping
> for the MATH MINUS, the other FULLWIDTH digits are supported in
> cs932.
>
> Gerd

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev