From nobody Mon Nov 16 14:45:57 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jSrJk25sBz70nD4 for ; Mon, 16 Nov 2026 14:45:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jSrJk0L22z4MJ3 for ; Mon, 16 Nov 2026 14:45:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794840358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=DnIGKa8dGNX3LhCzcVMFslm8cEeGzKV2NZxI3UJvYkQ=; b=VWvkEcK2KucDT9FW2hRtvJlW9jD0EP36w/5Z19b/d1afdYd4KDmVhVC1TFBGxsmrYq7WvI UmDI0d/Yfh/0iny/GPlPtXP/9WVi2emoGdS/SlN86gAsaLygRD6FfL4Z3mq3j135b1ls3N fcziW6J/xfeGjmNfpeMyzEKJmMe5pH3wrJKAQlSWFkuUsERLN90RNK6Fq7oOJ99qRHWsrR ZBAN67hRRCrsXY72vakrjV1m8X0it1uRv1Kc/qR0OdW1vWyba/jdSwnV/0MiaK5axd/nlY 9qUQLLOppyhpkE7iXgUT6IlzEmJHEMS0jRzRgwYPEQPx0Ffb5zUyp9eE2Ec2eA== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1794840358; b=YmPGr4lOH76rzO69Irv1+nG7IPJgi0Q3ovlz80PokNupI3mudADn7DABtHDKbp4u2Pjk92 1/4n2cBUnxIt+IwnahFfm+TQq4u5nFnBWEAR4PZk/Ltit16J9V6WwHCX1C49KlgyHxaNXx +n6WClaBBcadMCArR3tXbwqQAcYctmDKLYYnZ+3bTIHlwYw0ss/+P5os87+SyvIepMDx/q koLHHN0zhChfWuOapcsfK2Wnai22rCz2AYSiQsIfbAjwlTJh/veK+1QUWX27j3N6Q98RJY HQDN5XCUA4/ymTJJmHAo+YA76mL3pIJ0ZedoaIeoiZbzssXkMrMlCN9X0E4xVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1794840358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=DnIGKa8dGNX3LhCzcVMFslm8cEeGzKV2NZxI3UJvYkQ=; b=WSTcO4/F0JceMZU/TRYxHaqW10nAHpaD1X6iqgBfEScgyGh/jh81HD7LtzjnxWlJo6Dl+4 Li+ZDsEKH9EJu8V55rKKdUTer2RcB97zqU9fdNtr8PDZzKF+eu8zKbSBG3ancyQ6pY+pXy RqNn+4CvOEcWDKkLvgFtzxRJAoqHWjgXnSI42vBm/EVlx8QOG2pdXzdKAmZN0haPl2Civu M3eOWumJ/OavQnFMEdPra7F/ciBJ/xMV8Zgirr7BGqngPWUgOrwvGbdHZaqH7leFiRuFs2 2uU0aLFyaUki3u+Ue+2UJ1k3NIHhXjwe8WpAONgo9Fp5AsH3XAfWcQ/DzUeciA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jSrJk00FTzykF for ; Mon, 16 Nov 2026 14:45:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id f37 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 16 Nov 2026 14:45:57 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Dag-Erling=?utf-8?Q? Sm=C3=B8rg?=rav Subject: git: 471d0cb24c6d - main - pax: Use copy_file_range List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 471d0cb24c6d1252f33dd1cdb6c907dd3b6b5ea4 Auto-Submitted: auto-generated Date: Mon, 16 Nov 2026 14:45:57 +0000 Message-Id: <6afb1725.f37.5aaeb2f1@gitrepo.freebsd.org> The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=471d0cb24c6d1252f33dd1cdb6c907dd3b6b5ea4 commit 471d0cb24c6d1252f33dd1cdb6c907dd3b6b5ea4 Author: Dag-Erling Smørgrav AuthorDate: 2026-11-16 14:44:24 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2026-11-16 14:44:24 +0000 pax: Use copy_file_range Use copy_file_range(2) instead of our own inefficient and unreliable attempt at detecting holes. If copy_file_range(2) is not available, which in theory should be never, we fall back to a simple read(2) / write(2) loop which makes no attempt to copy holes. This should speed up pax copy mode and cpio passthrough mode significantly. This includes refactoring much of the existing I/O code to use the size_t and ssize_t instead of int where appropriate. MFC after: 1 week Sponsored by: Klara, Inc. Reviewed by: obiwac Differential Revision: https://reviews.freebsd.org/D58470 --- bin/pax/ar_io.c | 35 ++++++------ bin/pax/buf_subs.c | 163 +++++++++++++++++++++++++++++------------------------ bin/pax/extern.h | 12 ++-- 3 files changed, 114 insertions(+), 96 deletions(-) diff --git a/bin/pax/ar_io.c b/bin/pax/ar_io.c index c595870aaa5e..686c211b301f 100644 --- a/bin/pax/ar_io.c +++ b/bin/pax/ar_io.c @@ -484,8 +484,8 @@ ar_app_ok(void) if (!invld_rec) return(0); - paxwarn(1,"Cannot append, device record size %d does not support %s spec", - rdblksz, argv0); + paxwarn(1, "Cannot append, device record size %zu " + "does not support %s spec", rdblksz, argv0); return(-1); } @@ -498,10 +498,10 @@ ar_app_ok(void) * Number of bytes in buffer. 0 for end of file, -1 for a read error. */ -int -ar_read(char *buf, int cnt) +ssize_t +ar_read(char *buf, size_t cnt) { - int res = 0; + ssize_t res = 0; /* * if last i/o was in error, no more reads until reset or new volume @@ -523,7 +523,7 @@ ar_read(char *buf, int cnt) * manner. */ io_ok = 1; - if (res != rdblksz) { + if ((size_t)res != rdblksz) { /* * Record size changed. If this happens on * any record after the first, we probably have @@ -581,8 +581,8 @@ ar_read(char *buf, int cnt) * error in the archive occurred. */ -int -ar_write(char *buf, int bsz) +ssize_t +ar_write(char *buf, size_t bsz) { int res; off_t cpos; @@ -594,20 +594,21 @@ ar_write(char *buf, int bsz) if (lstrval <= 0) return(lstrval); - if ((res = write(arfd, buf, bsz)) == bsz) { - wr_trail = 1; - io_ok = 1; - return(bsz); + if ((res = write(arfd, buf, bsz)) < 0) { + lstrval = -1; + } else { + lstrval = 0; + if ((size_t)res == bsz) { + wr_trail = 1; + io_ok = 1; + return(bsz); + } } + /* * write broke, see what we can do with it. We try to send any partial * writes that may violate pax spec to the next archive volume. */ - if (res < 0) - lstrval = res; - else - lstrval = 0; - switch (artyp) { case ISREG: if ((res > 0) && (res % BLKMULT)) { diff --git a/bin/pax/buf_subs.c b/bin/pax/buf_subs.c index 7f9b31881d56..0ccd8e77af99 100644 --- a/bin/pax/buf_subs.c +++ b/bin/pax/buf_subs.c @@ -35,10 +35,13 @@ #include #include + #include -#include +#include #include #include +#include + #include "pax.h" #include "extern.h" @@ -54,14 +57,14 @@ * limit on blocking size is removed (though that will violate pax spec) * MAXBLK define and tests will also need to be updated. */ -static char bufmem[MAXBLK+BLKMULT]; /* i/o buffer + pushback id space */ +static char bufmem[MAXBLK + BLKMULT]; /* i/o buffer + pushback id space */ static char *buf; /* normal start of i/o buffer */ static char *bufend; /* end or last char in i/o buffer */ static char *bufpt; /* read/write point in i/o buffer */ -int blksz = MAXBLK; /* block input/output size in bytes */ -int wrblksz; /* user spec output size in bytes */ +size_t blksz = MAXBLK; /* block input/output size in bytes */ +size_t wrblksz; /* user spec output size in bytes */ int maxflt = MAXFLT; /* MAX consecutive media errors */ -int rdblksz; /* first read blksize (tapes only) */ +size_t rdblksz; /* first read blksize (tapes only) */ off_t wrlimit; /* # of bytes written per archive vol */ off_t wrcnt; /* # of bytes written on current vol */ off_t rdcnt; /* # of bytes read on current vol */ @@ -87,18 +90,18 @@ wr_start(void) if (!wrblksz) wrblksz = frmt->bsz; if (wrblksz > MAXBLK) { - paxwarn(1, "Write block size of %d too large, maximum is: %d", - wrblksz, MAXBLK); + paxwarn(1, "Write block size of %zu too large, maximum is: %d", + wrblksz, MAXBLK); return(-1); } if (wrblksz % BLKMULT) { - paxwarn(1, "Write block size of %d is not a %d byte multiple", + paxwarn(1, "Write block size of %zu is not a %d byte multiple", wrblksz, BLKMULT); return(-1); } if (wrblksz > MAXBLK_POSIX) { - paxwarn(0, "Write block size of %d larger than POSIX max %d, archive may not be portable", - wrblksz, MAXBLK_POSIX); + paxwarn(0, "Write block size of %zu larger than POSIX max %d, " + "archive may not be portable", wrblksz, MAXBLK_POSIX); return(-1); } @@ -132,13 +135,13 @@ rd_start(void) buf = &(bufmem[BLKMULT]); if ((act == APPND) && wrblksz) { if (wrblksz > MAXBLK) { - paxwarn(1,"Write block size %d too large, maximum is: %d", - wrblksz, MAXBLK); + paxwarn(1, "Write block size %zu too large, " + "maximum is: %d", wrblksz, MAXBLK); return(-1); } if (wrblksz % BLKMULT) { - paxwarn(1, "Write block size %d is not a %d byte multiple", - wrblksz, BLKMULT); + paxwarn(1, "Write block size %zu is not a " + "%d byte multiple", wrblksz, BLKMULT); return(-1); } } @@ -735,82 +738,95 @@ rd_wrfile(ARCHD *arcn, int ofd, off_t *left) return(0); } +/* + * Copy up to len (limited to SSIZE_MAX) bytes from infd to outfd with + * similar semantics to copy_file_range(2), for use when the latter does + * not work. Just like copy_file_range(2), this may terminate early for a + * variety of reasons, and should therefore be called in a loop until it + * returns zero or an error. + */ + +static ssize_t +copy_file_fallback(int infd, int outfd, size_t len) +{ + char *pos; + ssize_t rlen = 0, wlen = 0; + size_t total = 0; + + if (len > SSIZE_MAX) + len = SSIZE_MAX; + if (len == 0) + return (0); + while (total < len) { + if ((rlen = read(infd, buf, MIN(blksz, len))) <= 0) + break; + for (pos = buf; rlen > 0; pos += wlen, rlen -= wlen) + if ((wlen = write(outfd, pos, rlen)) < 0) + break; + total += pos - buf; + if (wlen < 0) + break; + } + /* report a partial success as success */ + if (total > 0) + return (total); + if (rlen < 0 || wlen < 0) + return (-1); + return (0); +} + /* * cp_file() - * copy the contents of one file to another. used during -rw phase of pax - * just as in rd_wrfile() we use a special write function to write the - * destination file so we can properly copy files with holes. + * copy the contents of one file to another, used during -rw phase of pax. */ void cp_file(ARCHD *arcn, int fd1, int fd2) { - int cnt; - off_t cpcnt = 0L; - int res = 0; - char *fnm = arcn->name; - int no_hole = 0; - int isem = 1; - int rem; - int sz = MINFBSZ; struct stat sb; + ssize_t res, total = 0; /* - * check for holes in the source file. If none, we will use regular - * write instead of file write. - */ - if (((off_t)(arcn->sb.st_blocks * BLKMULT)) >= arcn->sb.st_size) - ++no_hole; - - /* - * pass the blocksize of the file being written to the write routine, - * if the size is zero, use the default MINFBSZ + * Try copy_file_range(2) first. On FreeBSD, it works in almost + * all situations, even across file systems, and even when the + * underlying file system does not implement it. */ - if (fstat(fd2, &sb) == 0) { - if (sb.st_blksize > 0) - sz = sb.st_blksize; - } else - syswarn(0,errno,"Unable to obtain block size for file %s",fnm); - rem = sz; + for (;;) { + res = copy_file_range(fd1, NULL, fd2, NULL, SSIZE_MAX, 0); + if (res <= 0) + break; + total += res; + } /* - * read the source file and copy to destination file until EOF + * Fall back to a standard copy loop. */ - for(;;) { - if ((cnt = read(fd1, buf, blksz)) <= 0) - break; - if (no_hole) - res = write(fd2, buf, cnt); - else - res = file_write(fd2, buf, cnt, &rem, &isem, sz, fnm); - if (res != cnt) - break; - cpcnt += cnt; + if (res < 0 && errno == EINVAL) { + for (;;) { + res = copy_file_fallback(fd1, fd2, SSIZE_MAX); + if (res <= 0) + break; + total += res; + } } /* * check to make sure the copy is valid. */ - if (res < 0) + if (res < 0) { syswarn(1, errno, "Failed write during copy of %s to %s", - arcn->org_name, arcn->name); - else if (cpcnt != arcn->sb.st_size) + arcn->org_name, arcn->name); + } else if (total != arcn->sb.st_size) { paxwarn(1, "File %s changed size during copy to %s", - arcn->org_name, arcn->name); - else if (fstat(fd1, &sb) < 0) + arcn->org_name, arcn->name); + } else if (fstat(fd1, &sb) < 0) { syswarn(1, errno, "Failed stat of %s", arcn->org_name); - else if (arcn->sb.st_mtime != sb.st_mtime) + } else if (arcn->sb.st_mtime != sb.st_mtime || + arcn->sb.st_size != sb.st_size) { paxwarn(1, "File %s was modified during copy to %s", - arcn->org_name, arcn->name); + arcn->org_name, arcn->name); + } - /* - * if the last block has a file hole (all zero), we must make sure this - * gets updated in the file. We force the last block of zeros to be - * written. just closing with the file offset moved forward may not put - * a hole at the end of the file. - */ - if (!no_hole && isem && (arcn->sb.st_size > 0L)) - file_flush(fd2, fnm, isem); return; } @@ -872,11 +888,11 @@ buf_fill(void) */ int -buf_flush(int bufcnt) +buf_flush(size_t bufcnt) { - int cnt; - int push = 0; - int totcnt = 0; + ssize_t cnt; + size_t push = 0; + size_t totcnt = 0; /* * if we have reached the user specified byte count for each archive @@ -916,8 +932,9 @@ buf_flush(int bufcnt) /* * write a block and check if it all went out ok */ - cnt = ar_write(buf, blksz); - if (cnt == blksz) { + if ((cnt = ar_write(buf, blksz)) < 0) { + break; + } else if ((size_t)cnt == blksz) { /* * the write went ok */ @@ -937,7 +954,7 @@ buf_flush(int bufcnt) } else bufpt = buf; return(totcnt); - } else if (cnt > 0) { + } else { /* * Oh drat we got a partial write! * if format doesn't care about alignment let it go, diff --git a/bin/pax/extern.h b/bin/pax/extern.h index 26e5f6b6151b..fa203b73f58c 100644 --- a/bin/pax/extern.h +++ b/bin/pax/extern.h @@ -49,8 +49,8 @@ void ar_close(void); void ar_drain(void); int ar_set_wr(void); int ar_app_ok(void); -int ar_read(char *, int); -int ar_write(char *, int); +ssize_t ar_read(char *, size_t); +ssize_t ar_write(char *, size_t); int ar_rdsync(void); int ar_fow(off_t, off_t *); int ar_rev(off_t ); @@ -69,10 +69,10 @@ void copy(void); /* * buf_subs.c */ -extern int blksz; -extern int wrblksz; +extern size_t blksz; +extern size_t wrblksz; extern int maxflt; -extern int rdblksz; +extern size_t rdblksz; extern off_t wrlimit; extern off_t rdcnt; extern off_t wrcnt; @@ -91,7 +91,7 @@ int wr_rdfile(ARCHD *, int, off_t *); int rd_wrfile(ARCHD *, int, off_t *); void cp_file(ARCHD *, int, int); int buf_fill(void); -int buf_flush(int); +int buf_flush(size_t); /* * cache.c