Unicode-based FreeBSD

[Alexander Churanov]

Svavar,

I am trying to understand you.

2008/8/26 Svavar Lúthersson <svavar at kjarrval.is>

> The Icelandic alphabet works in editors like pico but I have not found
> another editor where it actually displays the characters correctly. I
> checked edit, vi and vipw to be sure. It might be a configuration problem or
> a lack of it but it's better for the user experience if it works
> out-of-the-box. It should be enough to configure it in one place and it
> should work "everywhere".
>
Hmm. A minute ago I've pressed Ctrl-Alt-F3, switched to syscons console,
started "emacs /tmp/test", where "test" was written in russian, typed some
russian text into, closed the editor and then started "cat /tmp/test". No
problems. I still can not understand what's the difference between
ISO-8859-1 and KOI8-R from the implementation point of view. It seems that I
need to try to configure a system for Icelandic. I'll do that tomorrow on a
dedicated box. I promise to help you with configuration in case It's at all
possible.


> The primary problem of the character support in syscons is displaying
> specialised characters on the screen/tty. When I use the special Icelandic
> characters in UTF-8, each character is displayed as "??" which is very
> confusing to see if there are 2 or more in a row...

This is exactly what I am trying to solve, examining opinions on this list
at the same time.

> Of course there are certain problems with changing the filenames between
> languages like Russian and Icelandic since the normal keyboard only has
> about 100 keys and cannot possible contain all the characters in the Unicode
> specification.

There are special Input Methods for the rest of Unicode (more than 200K code
points currently assigned).


> It however should not stop me from reading the filenames in the language
> they were written. As for writing characters in other languages, the
> "Windows approach" steps in the right direction by enabling me to change the
> input language and therefore type in characters I would not otherwise be
> able to with the Icelandic keyboard. If the characters are translated to
> Unicode, it should not matter what keyboard layout is used. As for how it
> would be carried out in FreeBSD, I will leave it up to the developers.

For switching I use CapsLock when in plain syscons console and  Alt+Shift
when in X. By the way, how Windows displays non-ASCII characters in plain
text console? I'll wonder if better than suggested by me.

> The aforementioned is why I am suggesting that the system should be moved
> directly to UTF-32. If it is moved to UTF-8 and there is a need in the
> future for UTF-16 or -32, the conversion process has to start again.


I'm sure that it is not necessary. Again, all UTFs encode THE SAME SET. But
UTF-32 is better for single characters. And UTF-8 is better for UNIX-like
systems.


> Like I mentioned in my former answer, the program writers do not write
> Unicode compatible programs because there is almost no Unicode support and
> the FreeBSD developers see little reason to speed up Unicode implementation
> because there are so few programs Unicode compatible. Therefore I think that
> FreeBSD should implement a Unicode support policy and move straight to
> UTF-32 and make it the FreeBSD default. I am not pretending that this
> project will be easy, painless and quick but it is better done sooner than
> later. Said policy could begin by announcing an active plan for Unicode
> support and suggest that every new FreeBSD project should support Unicode.
> At the same time it should suggest the same to other developers which write
> software for FreeBSD. When the time is right or after further steps, the
> FreeBSD Foundation should announce that after version X, Unicode will be
> default charset. At that time, the software which has Unicode support will
> (I hope) work flawlessly with Unicode characters. When UTF-32 would be fully
> supported in FreeBSD, the developers could wait for the end of the support
> cycle for the first version with full UTF-32 support and then make it the
> default in the versions to come. That way the backward compatibility would
> be great and for all supported versions of FreeBSD.

Probably, this is useful, but I'm sure that this is out of scope of my
little project. I do not have enough power to enforce such a policy.

> Með kveðju / With regards,
> Svavar Kjarrval (svavar at kjarrval.is)
> s. 863-9900
>
Alexander Churanov