svavar at kjarrval.is
Tue Aug 26 09:40:29 UTC 2008
Tim Kientzle wrote:
>> Going to UTF-8 might fix some of the character issues
>> but we would be in the same shoes when it comes to characters
>> which are in -16 and -32 but not in -8.
> You need to read the Unicode/ISO10646 standards again;
> you do not understand them.
You are right, I do not understand them. As I mentioned, I am not a
Unicode expert and I have never claimed to be one.
> There are no characters in UTF-32 that are not in UTF-8.
> UTF-32, UTF-16, and UTF-8 all use exactly the same characters.
> UTF-8 encodes Unicode characters from U+000000 to U+10FFFF, using 1 to
> 4 bytes per character.
> UTF-16 encodes Unicode characters from U+000000 to U+10FFFF, using 2
> to 4 bytes per character.
> UTF-32 encodes Unicode characters from U+000000 to U+10FFFF, using 4
> bytes per character.
> Practically speaking, UTF-8 is a bit more convenient for file
> storage and transmission (including terminal support), UTF-16
> or UTF-32 can be slightly more convenient for internal
> string manipulation. But all three encodings use exactly
> the same characters.
> Tim Kientzle
I cannot confirm you are 100% right because I am not an expert in
Unicode. However, after some reading, I can see there is no "character
loss" by using one form of Unicode than the other. Therefore, I stand
corrected on that issue. I still think there should be support for
UTF-16 and UTF-32 in FreeBSD in general but it is outside the scope of
the topic (Unicode in syscons).
Tz-Huan Huang wrote:
> How do you define ``support''?
> If you mean software-level support, vim supports UTF-16, firefox
> supports UTF-16/UTF-32, perl supports UTF-16/UTF-32, etc.
> If you mean system-level support, there are two cases:
> 1. The system internal text representation is still in UTF-8, just add
> support for terminal, stdin/stdout/stderr, etc. I think it's not so
> hard (I might be
> wrong because I don't know terminal at all) but I don't see any reason to set
> locale to UTF-16 or UTF-32.
> 2. The system internal text representation is changed to UTF-16 or UTF-32.
> This is another story and I have no comment on it.
By support I meant full handling of Unicode characters which meant both
1 and 2. Although, in connection to my discovery above, I think it is
better if the internal handling is (continued to be) done in UTF-8.
Með kveðju / With regards,
Svavar Kjarrval (svavar at kjarrval.is)
More information about the freebsd-current