csh Cannot open /etc/termcap after starting "screen"
Jeremy Chadwick
freebsd at jdc.parodius.com
Sun Jun 19 06:08:44 UTC 2011
On Sat, Jun 18, 2011 at 10:14:32PM +0200, Stefan `Sec` Zehl wrote:
> Hi,
>
> On Thu, Jun 16, 2011 at 13:15 -0700, Jeremy Chadwick wrote:
> > Example: run mutt from within GNU screen while connected to
> > the system with PuTTY, then copy some of the terminal content and paste
> > it somewhere. Wow, look at all those extraneous spaces at the end of
> > lines, which you now gloriously have to manually remove.
>
> While I don't want to stand in the way of your rant, this is actually a
> bug/problem of mutt. -- mutt is really printing spaces there, so it is
> (IMHO) correct that copy&paste copies spaces.
mutt is not printing spaces. GNU screen is doing it, and I'm going to
show you hard evidence of it. I've done this analysis many times over
the years, so doing it once again won't hurt. It never ceases to amaze
me how GNU screen advocates "insist it's a problem with <app>" when that
simply isn't the case.
Let's take a look at *exactly* what mutt is sending to stdout (fd 1) in
a terminal that's 132x43 in size, using TERM=xterm, thus without GNU
screen. PuTTY is my terminal emulator. We're looking at the I/O that
comes across fd 1 (stdout) from mutt. mutt on this system is build with
ncurses (base system version, not ports)
The test procedure:
ktrace -ti /usr/local/bin/mutt -f =spam
<hit q>
kdump | less
<look through output for relevant data>
Firstly, a screen shot of what shows up in PuTTY, just so you have some
visual context:
http://www.malkavian.com/~jdc/mutt.png
And across stdout, here's what we see:
29053 mutt GIO fd 1 wrote 340 bytes
0x0000 0d1b 5b33 376d 1b5b 4a1b 5b48 1b5b 3337 6d1b 5b34 346d 1b5b 316d 2d2d 2d4d 7574 |..[37m.[J.[H.[37m.[44m.[1m---Mut|
0x0020 743a 203d 7370 616d 205b 4d73 6773 3a31 204e 6577 3a31 2049 6e63 3a33 2032 2e38 |t: =spam [Msgs:1 New:1 Inc:3 2.8|
0x0040 4b5d 2d2d 2d28 7468 7265 6164 732f 6461 7465 292d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d |K]---(threads/date)-------------|
0x0060 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d |--------------------------------|
0x0080 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2861 6c6c 292d 2d2d 1b5b |----------------------(all)---.[|
0x00a0 323b 3148 1b5b 3337 6d1b 5b34 366d 2020 2031 204e 202b 2030 362f 3138 2031 363a |2;1H.[37m.[46m 1 N + 06/18 16:|
0x00c0 3436 2020 4f72 6465 7220 4e6f 7469 6669 6572 2020 2020 2020 2830 2e34 4b29 205b |46 Order Notifier (0.4K) [|
0x00e0 7370 616d 5d20 4865 6c6c 6f1b 5b4b 0d1b 5b34 3042 1b5b 3337 6d1b 5b34 346d 713a |spam] Hello.[K..[40B.[37m.[44mq:|
0x0100 5175 6974 2020 643a 4465 6c20 2075 3a55 6e64 656c 2020 733a 5361 7665 2020 6d3a |Quit d:Del u:Undel s:Save m:|
0x0120 4d61 696c 2020 723a 5265 706c 7920 2067 3a47 726f 7570 2020 3f3a 4865 6c70 1b5b |Mail r:Reply g:Group ?:Help.[|
0x0140 4b1b 5b32 3b31 3332 481b 5b33 393b 3439 6d1b 5b6d |K.[2;132H.[39;49m.[m|
Here's the xterm escape sequence decoding chart -- we'll be caring about
<esc>[ a lot, which is referred to as "CSI" in the docs.
http://invisible-island.net/xterm/ctlseqs/ctlseqs.html
Let's decode the above output and turn it into something human-readable:
<esc>[37m = set foreground text colour to white
<esc>[J = clear from cursor to end of terminal
<esc>[H = move cursor to top left corner of terminal
<esc>[37m = set foreground text colour to white
<esc>[44m = set background colour to blue
<esc>[1m = turn on bold/bright
---Mutt: =spam [Msgs:1 New:1 Inc:3 2.8K]---(threads/date)-------------------------------------------------------------------(all)---
<esc>[2;1H = move cursor to row 2 column 1
<esc>[37m = set foreground text colour to white
<esc>[46m = set background colour to cyan
1 N + 06/18 16:46 Order Notifier (0.4K) [spam] Hello
<esc>[K = erase from cursor position to end of line
<newline>
<esc>[40B = move cursor 40 lines downward
<esc>[37m = set foreground text colour to white
<esc>[44m = set background colour to blue
q:Quit d:Del u:Undel s:Save m:Mail r:Reply g:Group ?:Help
<esc>[K = erase from cursor position to end of line
<esc>[2;132H = move cursor to row 2, column 132
<esc>[39;49m = set foreground text colour and background to default
<esc>[m = turn off bold/bright
Given this analysis, we can see where GNU screen is making some stupid
assumptions. Let's focus on the part that's causing the nonsense:
1 N + 06/18 16:46 Order Notifier (0.4K) [spam] Hello
^
cursor is now here --------^
This is followed by <esc>[K, which is used to erase all content from the
cursor position to the end of the line. "Erase all content" does not
mean "print spaces" -- you can see quite clearly in the I/O to stdout
that no "padding spaces" are printed. It's a terminal escape sequence
which is interpreted by xterm (PuTTY in my case) to do what it's told.
It's important to note that the foreground colour at this point is set
to bolded white (not the default), and the background colour is set to
cyan (not the default). An erase-line command in this situation results
in something called BCE -- you know, the thing I talked about in my mail
that you responded to? Please read about it:
http://invisible-island.net/xterm/xterm.faq.html#xterm_terminfo
Like I said in my mail, GNU screen does stupid things when it comes to
BCE. It seems to think that because from-cursor-to-EOL is being erased
with a non-default background colour, that that means "hey I better
'emulate' this by printing spaces". And I'm about to show you that's
completely the case.
Let's run the exact same test procedure but within GNU screen. This is
stock GNU screen; literally I just installed it from ports, ran it, then
ran the same test procedure. Once in screen, TERM=screen (obviously).
Lo and behold, what do we see?
32547 mutt GIO fd 1 wrote 495 bytes
0x0000 1b5b 481b 5b33 376d 1b5b 3434 6d1b 5b31 6d2d 2d2d 4d75 7474 3a20 3d73 7061 6d20 |.[H.[37m.[44m.[1m---Mutt: =spam |
0x0020 5b4d 7367 733a 3120 4e65 773a 3120 496e 633a 3320 322e 384b 5d2d 2d2d 2874 6872 |[Msgs:1 New:1 Inc:3 2.8K]---(thr|
0x0040 6561 6473 2f64 6174 6529 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d |eads/date)----------------------|
0x0060 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d |--------------------------------|
0x0080 2d2d 2d2d 2d2d 2d2d 2d2d 2d2d 2d28 616c 6c29 2d2d 2d1b 5b32 3b31 481b 5b33 376d |-------------(all)---.[2;1H.[37m|
0x00a0 1b5b 3436 6d20 2020 3120 4e20 2b20 3036 2f31 3820 3136 3a34 3620 204f 7264 6572 |.[46m 1 N + 06/18 16:46 Order|
0x00c0 204e 6f74 6966 6965 7220 2020 2020 2028 302e 344b 2920 5b73 7061 6d5d 2048 656c | Notifier (0.4K) [spam] Hel|
0x00e0 6c6f 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 |lo |
0x0100 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 | |
0x0120 2020 2020 2020 2020 201b 5b34 323b 3148 1b5b 3337 6d1b 5b34 346d 713a 5175 6974 | .[42;1H.[37m.[44mq:Quit|
0x0140 2020 643a 4465 6c20 2075 3a55 6e64 656c 2020 733a 5361 7665 2020 6d3a 4d61 696c | d:Del u:Undel s:Save m:Mail|
0x0160 2020 723a 5265 706c 7920 2067 3a47 726f 7570 2020 3f3a 4865 6c70 2020 2020 2020 | r:Reply g:Group ?:Help |
0x0180 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 | |
0x01a0 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 2020 1b5b | .[|
0x01c0 3433 3b31 481b 5b6d 1b5b 3337 6d20 2020 2020 2020 2020 2020 2020 2020 2020 201b |43;1H.[m.[37m .|
0x01e0 5b32 3b31 3332 481b 5b33 396d 1b5b 6d |[2;132H.[39m.[m|
Look at all those spaces to fill the entire width of the terminal
window. I don't even need to do the analysis, it's quite obvious what's
going on.
I also want to point out "defbce on" in .screenrc also did not solve
this problem; the output is the exact same as you see in the above text
(extraneous spaces).
Why does this happen? Like I said in my previous mail: GNU screen tries
to **convert** between terminal types (xterm <--> screen in this case).
This concept of GNU screen stems from environments where you had an
administrator wanting a "consistent terminal" regardless of what actual
**attached** terminal type he was using.
For example, admin is on a Linux console (TERM=cons25), runs screen,
uses things for a while, then detaches. He walks across the building
into the datacenter, and needs to start working on something local to
the servers. The only terminal he has in the datacenter is, say, a real
DEC VT100. He can connect to whatever, run "screen -r", and he'll still
(generally) "see" the same output he did before he detached (sans colour
and all that of course).
tmux doesn't behave this way. It doesn't try to do "terminal emulation"
or "terminal translation". The idea there is that people are probably
using, consistently, the same terminal software (PuTTY, xterm, rxvt,
etc.) rather than moving between terminal types often.
The above analysis/details speak for themselves. As such, please do not
tell me "mutt is causing this problem" -- as I said in the beginning,
GNU screen is causing the problem.
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
More information about the freebsd-stable
mailing list