docs/156081: troff falls with troff.core with UTF-8 man with incorrect symbols displaying for kernel UTF-8

[Tim]

>Number:         156081
>Category:       docs
>Synopsis:       troff falls with troff.core with UTF-8 man with incorrect symbols displaying for kernel UTF-8
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-doc
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          doc-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Mar 31 08:00:20 UTC 2011
>Closed-Date:
>Last-Modified:
>Originator:     Tim
>Release:        8.2-STABLE
>Organization:
>Environment:
FreeBSD timfree.ibmc.msk.ru 8.2-STABLE FreeBSD 8.2-STABLE #0: Wed Mar 30 16:09:42 MSD 2011     root at timfree.ibmc.msk.ru:/usr/obj/usr/src/sys/KERN_2011_03_30  amd64
>Description:
man pages for UTF-8 encoding at least for russian language cause troff falling with troff.core, and all text is in incorrect character set for kernel with embeded UTF-8 and UTF-8 locale specified (with xterm tty, with LANG, etc.)
>How-To-Repeat:
1. make kernel UTF-8 and UTF-8 locale
2. install vim
3. man vim
>Fix:
*roff system not understand standart unicode symbols, but man send to *roff exactly UTF-8 without any transformations:

Principles of man -d for FreeBSD (with vim as example):
-----------------------
zcat /usr/local/man/ru.UTF-8/man1/vim.1.gz |\
tbl |\
groff -S -Wall -mtty-char -man -Tutf8 -dlocale=ru.UTF-8 |\
col | less
----------------------

In Linux there is preconv between zcat and *roff:

Principles of man -d vim for Ubuntu (as Linux example):
-------------------------------
zcat /usr/share/man/ru/man1/vim.1.gz |\
preconv -e UTF-8 |\
 tbl |\
nroff -mandoc -rLL=97n | less
-------------------------

preconv make \[uXXXX] char sequences for any UTF symbol and these sequences understand *roff. In FreeBSD such preconv is absent. And this cause title effect.
Usage iconv such a way:
zcat vim.1.gz |iconv -f utf-8 -t koi8-r |nroff -mandoc -Tkoi8-r |iconv -t utf-8 -f koi8-r|less
is workaround of this problem, while nroff work correctly with koi8.

>Release-Note:
>Audit-Trail:
>Unformatted: