cp(1) of large files is causing 100% CPU utilization and poor transfer
Alan Somers
asomers at freebsd.org
Sat Jan 2 16:06:38 UTC 2021
On Sat, Jan 2, 2021 at 9:02 AM Matthias Apitz <guru at unixarea.de> wrote:
> El día sábado, enero 02, 2021 a las 08:42:01a. m. -0700, Alan Somers
> escribió:
>
> > > # dd if=guru-20210102.tar.gz
> of=/mnt/AcerC720/backups/guru-20210102.tar.gz
> > > bs=8m
> > > 4601+1 records in
> > > 4601+1 records out
> > > 38603862368 bytes transferred in 506.778929 secs (76174956 bytes/sec)
> > > # ls -lh guru-20210102.tar.gz
> > > -r-------- 1 root wheel 36G 2 ene. 12:19 guru-20210102.tar.gz
> > >
> > > When I use cp(1) what I normaly do the transfer is very poor and causes
> > > 100% CPU itilization:
> > >
> > > # cp -p guru-20210102.tar.gz /mnt/AcerC720/backups/xxx
> > > ^C
> > >
> > > killed after 1 minute; transfered in 1 minute:
> > >
> > > # ls -lh /mnt/AcerC720/backups/xxx
> > > -r-------- 1 root wheel 168M 2 ene. 15:34
> /mnt/AcerC720/backups/xxx
> > >
> > > 168*1024*1024/60 = 2936012 bytes/sec ./. 76174956 bytes/sec
> > >
> > > Trussing the cp(1) process shows these sys calls:
> > >
> > > # truss -p 37655
> > > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000)
> > > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000)
> > > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000)
> > > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000)
> > > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000)
> > >
> > > The problem is unrelated to the external disk, also a copy
> > > in the local file system shows the same transfer speed of 168M per
> > > minute.
>
> > Not an issue I've heard of before. Could you please describe how your
> USB
> > and local disk are formatted? Also, where is the source file stored? It
> > could be that the source file's file system has a very slow
> implementation
> > of FIOSEEKDATA/FIOSEEKHOLE.
> > -Alan
>
>
> As I said, it can be reproduced using only the local file system. This
> was setup recently on a SSD:
>
> # dmesg | grep ada0
> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
> ada0: <TS512GMTS430S R0906A> ACS-2 ATA SATA 3.x device
> ada0: Serial Number F995890846
> ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 1024bytes)
> ada0: Command Queueing enabled
> ada0: 488386MB (1000215216 512 byte sectors)
>
> and by this procedure:
>
> # gpart create -s gpt ada0
> # gpart add -t freebsd-boot -s 512k -a4k -l ssdboot ada0
> # gpart bootcode -b /boot/pmbr -p /boot/gptboot -i1 ada0
> # gpart add -t freebsd-ufs -l ssdrootfs -b 1m -s 2g ada0
> # gpart add -t freebsd-ufs -l ssdvarfs -a 1m -s 2g ada0
> # gpart add -t freebsd-ufs -l ssdusrfs -a 1m ada0
> # newfs -U -t /dev/gpt/ssdrootfs
> # newfs -U -t /dev/gpt/ssdvarfs
> # newfs -U -t /dev/gpt/ssdusrfs
>
> # gpart show -l ada0
> => 40 1000215136 ada0 GPT (477G)
> 40 1024 1 ssdboot (512K)
> 1064 984 - free - (492K)
> 2048 4194304 2 ssdrootfs (2.0G)
> 4196352 4194304 3 ssdvarfs (2.0G)
> 8390656 16777216 4 ssdswap (8.0G)
> 25167872 975046656 5 ssdusrfs (465G)
> 1000214528 648 - free - (324K)
>
> # mount -t ufs
> /dev/gpt/ssdrootfs on / (ufs, local, soft-updates)
> /dev/gpt/ssdvarfs on /var (ufs, local, soft-updates)
> /dev/gpt/ssdusrfs on /usr (ufs, local, soft-updates)
>
> When I run in the /usr fs the command
>
> # cp -p guru-20210102.tar.gz xxx
>
> it copies around 168M per minute.
>
> matthias
>
Is that copying from /usr to /usr, or from /usr to /var or /?
More information about the freebsd-current
mailing list