Jexec and access to tty

[Paul Schenkeveld]

On Thu, Aug 11, 2011 at 02:34:01PM +0200, joris dedieu wrote:
> 2011/8/11 joris dedieu <joris.dedieu at gmail.com>:
> > 2011/8/10 joris dedieu <joris.dedieu at gmail.com>:
> >> 2011/8/9 Paul Schenkeveld <freebsd at psconsult.nl>:
> >>> Hi,
> >>>
> >>> There have been several threads about this issue, some people have come
> >>> up with work arounds but I think that the issue is more fundamental,
> >>> that's why I wanted to start this new thread.
> >>>
> >>> When using jexec to do interactive work inside an existing jail, people
> >>> find out that they no longer have access to their tty device.  As a
> >>> result, programs requiring input of passwords or passphrases behave
> >>> unexpectedly in one of several ways.
> >>>
> >>> Ssh says "Host key verification failed." and refuses to log in to
> >>> another system (unless pubkey authentication is user in combination with
> >>> an agent of course).  Some programs fall back to using stdin/stdout
> >>> and echo the password as it is typed (the mysql clients are popular
> >>> examples).
> >>>
> >>> Work-arounds that have been suggested are
> >>>  1. Run a sshd inside the jail and log in using ssh
> >>>  2. Start tmux inside the jail so you get a new pseudo tty slave inside
> >>>    the jail.  People trying screen find that it won't work unlike tmux.
> >>>  3. I tried using 'script -q /dev/null' inside the jail because it is
> >>>    part op the base system and it doesn't change your terminal type
> >>>    and interpret keyboard input and screen output.  I found out that I
> >>>    failed when I resized my window :-(
> >>
> >> An other way is to use chroot(5) to enter the jail.
> >> Maybe chroot /jail/root login -f $USER should be acceptable in some situations.
> >>
> >>>
> >>> I don't like 1 on a machine with many jails, especially if some of them
> >>> share the same IP address (e.g. sometimes I have to run a mail server on
> >>> the same IP adress as a webserver but in a distinct jail).
> >>>
> >>> 2 is not ideal either because tmux emulates a different terminal on
> >>> the inside than the terminal on the outside that it runs on.
> >>>
> >>> 3 is really a kludge and causes problems when you resize your window.
> >>>
> >>> I thought that I found a solution by rewriting jexec such that it will
> >>> open a pseudo tty and does the passing of data between the jailed pts
> >>> and the tty from where jexec was started but that's not going to work as
> >>> the pseudo tty most be opened by the child process inside the jail but
> >>> the parent outside the jail must have access to the master side of the
> >>> pseudo tty.
> >>>
> >>> So far we are still talking about work-arounds.  Why not look at the
> >>> root cause.  Unfortunately I'm not familiar with kernel sources so if
> >>> I'm wrong, please forgive me, I write this with the best intentions.
> >>>
> >>> The root cause of th problem appears to be that pseudo ttys opened
> >>> outside a jail are not visible nor accessible inside a jail, pseudo ttys
> >>> created inside a jail are visible and accessible though.
> >>
> >> As far as I understand, sys/fs/devfs/devfs_vnops.c uses
> >> prison_check(9) too see if an item as been build in the same jail or
> >> in a child.
> >> The tty exists inside the jail but you can't use it (and so you can't
> >> escape the jail).
> >>
> >>>
> >>> Would it be conceivable that by using jexec the controlling tty of jexec
> >>> magically becomes visible and accessible inside the jail?  Preferrable
> >>> only until jexec dies.
> >>
> >> I'm not sure. The only way should be  to temporary disable the check
> >> with a  variable. But it will also brake jail security during jexec
> >> excution.
> >> Using chroot(2) instead of jail(2) should be an option (but it's  non
> >> trivial to affect jail context for all other subsystems).
> >>
> >> I think the only right way is to open a new tty while entering the
> >> jail (this is what tmux or a jailed ssh does).
> >> But it should be difficult to make things like echo ps | jexec 1 sh works.
> >
> > I gave this way a try and quickly produced a dirty patch. I did not
> > yet investigate on the right way, but I already know that I'm wrong on
> > several items so sorry for your eyes :)
> >
> > Basically this patch  introduced  -t option that forks jexec inside
> > the jail, open a new pts and tries to establish a communication with
> > the old one.
> > It mostly works has expected but breaks some console inputs like
> > control commands, completion and so on.
> > Before spending more time  trying to produce something acceptable,
> > please let  me know if it should be the expected feature or sounds
> > like a nasty workaround.

In fact this is what I thought of writing some time ago but you open the
pseudo tty before jailing the child so the problem remains that the slave
side of the pseudo tty cannot be reopened from within the jail.  In
other words, if tty(1) returns "/dev/pts/1" while inside the jail, the
command "echo foo > /dev/pts/1" replies "cannot create /dev/pts/1: No
such file or directory".

If you open the pseudo tty after the child process is jailed, the parent
process has no access to the master side of the pseudo tty.

Perhaps having a UNIX domain socket between parent and child and send the
file desciptor of the master side of the pseudo tty back from child to
parent could work but we're still looking at a work-around and I believe
it would be fixable in devfs because the child of jexec which is in the
jail has the slave side of the pseudo tty as controlling terminal so that
could be checked in devfs to decide whether /dev/pts/X should be visible
and accessible to a process inside the jail.

Wish I were familiar with FreeBSD kernel code :-(

Kind regards,

Paul Schenkeveld