CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo]

[Andrey Chernov]

On Wed, Jun 18, 2008 at 10:22:31AM +0200, Dag-Erling Sm??rgrav wrote:
> I think part of the problem is that there aren't enough people who truly
> understand localization.  I think I understand most of it, but I'm
> pretty sure I *don't* understand how collation works, or is supposed to
> work.  Amongst other things, I don't understand how (or whether) it
> handles cases like "aa" and "??", which are considered the same letter in
> Norwegian.

Single byte locales collation works through strcoll() via chains, i.e. 
seek all chains starting with given letter. Multibyte locales collation 
currently is not implemented and can't be properly implemented under 
existen single byte framework (it will consume resourses badly in that 
case). I know semi-hacking attempts to implement multibyte collattion via 
single byte one, but all they are only for small ASCII + national alphabet 
subset, rest of Unicode left unsorted.

> Perhaps you could create a Localization page on wiki.freebsd.org which
> addresses these issues, or at least points to relevant resources?

IMHO single byte collating will be obsolete soon when Unicode collation 
will be implemented as SoC project, we needs something like ICU library 
which performs as described below, i.e. unified sorting for all possible 
chars:
http://unicode.org/reports/tr10/

-- 
http://ache.pp.ru/