git: 495bc1924edf - main - pty: preserve O_NONBLOCK when opening /dev/ptmx

The branch main has been updated by kib:

URL: https://cgit.FreeBSD.org/src/commit/?id=495bc1924edfd474c479df61ec35bdf6a83a8d17

commit 495bc1924edfd474c479df61ec35bdf6a83a8d17
Author:     Vladimir Grebenshchikov <vova@zote.me>
AuthorDate: 2026-10-12 18:31:30 +0000
Commit:     Konstantin Belousov <kib@FreeBSD.org>
CommitDate: 2026-10-16 13:34:20 +0000

    pty: preserve O_NONBLOCK when opening /dev/ptmx
    
    ptmx_fdopen() masks the supplied file flags to FREAD | FWRITE before
    passing them to pts_alloc(). This discards FNONBLOCK even when the
    caller explicitly requests O_NONBLOCK.
    
    As a result, open("/dev/ptmx", O_RDWR | O_NOCTTY | O_NONBLOCK) succeeds,
    but the returned file is blocking. F_GETFL reports O_NONBLOCK cleared,
    and a subsequent read can block instead of returning EAGAIN when no
    data is available.
    
    This causes Linux xfce4-terminal/VTE running under Linuxulator to
    freeze. VTE opens the PTY master with O_NONBLOCK and relies on
    nonblocking reads while draining terminal output. Once the available
    data has been consumed, the main UI thread blocks in ptsdev_read(),
    leaving the window unresponsive even though the child shell has
    started successfully.
    
    Include FNONBLOCK in the flags passed to pts_alloc(). Blocking opens
    retain their existing behavior. No change to posix_openpt(2) flag
    validation is required.
    
    The flag loss also reproduces with a native FreeBSD program, so the
    problem is in the shared /dev/ptmx implementation rather than Linux
    syscall flag translation.
    
    Validation:
    - Before the fix, a native FreeBSD reproducer observes O_NONBLOCK
      cleared after opening /dev/ptmx with that flag.
    - Explicitly setting O_NONBLOCK with F_SETFL works, confirming that
      nonblocking PTY operation itself is supported.
    - After the fix, F_GETFL preserves O_NONBLOCK when requested and
      leaves it cleared for a blocking open.
    - xfce4-terminal in the Linux jail displays the shell prompt and
      accepts input.
    
    Reviewed by:    kib
    MFC after:      1 week
    Differential revision:  https://reviews.freebsd.org/D60700
---
 sys/dev/pty/pty.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sys/dev/pty/pty.c b/sys/dev/pty/pty.c
index 3fb5e6b24184..abbf8201f1cb 100644
--- a/sys/dev/pty/pty.c
+++ b/sys/dev/pty/pty.c
@@ -129,7 +129,7 @@ ptmx_fdopen(struct cdev *dev __unused, int fflags, struct thread *td,
     struct file *fp)
 {
 
-	return (pts_alloc(fflags & (FREAD|FWRITE), td, fp));
+	return (pts_alloc(fflags & (FREAD | FWRITE | FNONBLOCK), td, fp));
 }
 
 static struct cdevsw ptmx_cdevsw = {