[Bug 272434] du and disk operations (like mv,cp,shutil.move/rmtree,os.rename) combination issues

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272434

            Bug ID: 272434
           Summary: du and disk operations (like
                    mv,cp,shutil.move/rmtree,os.rename) combination issues
           Product: Base System
           Version: 13.2-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: meetshamsher@gmail.com

Hi,
We were performing file system operations using du,cp,mv,shutil.move/rmtree and
os.rename. Observed one of the directory is going for toss in 13.1 and 13.2
(also checked in iso of 13.0,12.4 same issues observed). Yet to verify in
current.

rtestuser@freeBSD13_2:~ $ uname -a
FreeBSD freeBSD13_2.cs39 13.2-RELEASE FreeBSD 13.2-RELEASE
releng/13.2-n254617-525ecfdad597 GENERIC amd64
rtestuser@freeBSD13_2:~ $ cd /data/infra/
rtestuser@freeBSD13_2:/data/infra $ ps -aux | grep python
root        726   0.0  9.1 23100 8768 v0  I    14:34   14:51.95 python
move-move-delete.py (python3.9)
root       3332   0.0  7.3 18800 7028 v0  I    14:35    0:00.03 python du.py
(python3.9)
root      23350   0.0  7.3 18800 7028 v0  I    15:07    0:00.02 python du.py
(python3.9)
rtestuser 48818   0.0  2.1 12812 1984  0  S+   15:53    0:00.00 grep python
rtestuser@freeBSD13_2:/data/infra $ ls
createfiles.py          dir1                    dir4_ren                du.py  
                move-move-delete.py
rtestuser@freeBSD13_2:/data/infra $ cat move-move-delete.py 
import os
import shutil
import time
path2 = "/data/infra/dir2_ren"
path3 = "/data/infra/dir3_ren"
path4 = "/data/infra/dir4_ren"
while 1:
    os.rename("/data/infra/dir1/","/data/infra/dir2_ren/")
    os.system("ls -ir /data/infra/")
    print("rename done")
    os.mkdir(path3)
    files = [f for f in os.listdir("/data/infra/dir2_ren/")]
    for file in files:
        old = "/data/infra/dir2_ren/" + file
        ipath = "/data/infra/dir2_ren/" + file + "/"
        files1 = [af for af in os.listdir(ipath)]
        for ff in files1:
            fpath = "/data/infra/dir2_ren/" + file + "/" + ff   
            #keeping files opened to check it further
            p = open(fpath,"w")
            p1 = open(fpath,"w")
            print("I AM WRITTING")
            p.write("I am writting now!")
        new = "/data/infra/dir3_ren/" + file
        shutil.move(old,new)  
    print("shutil.move done")
    shutil.rmtree("/data/infra/dir2_ren")
    print("deleted dir2_ren")
    os.system("ls -ir /data/infra/")
    os.mkdir(path4)
    os.system("mv /data/infra/dir3_ren/* /data/infra/dir4_ren/")     
    print("os mv command success")
    shutil.rmtree("/data/infra/dir3_ren")
    print("deleted dir3_ren")
    print("after dir3_ren")
    os.system("ls -ir /data/infra/")
    os.system("mkdir /data/infra/dir1")
    os.system("cp -rf /data/infra/dir4_ren/* /data/infra/dir1/")     
    print("dir1 created back and copied data from dir4 to dir1")
    shutil.rmtree("/data/infra/dir4_ren")
rtestuser@freeBSD13_2:/data/infra $ cat du.py 
import os
import time;
while 1:
    os.system("du /")
    time.sleep(10)
rtestuser@freeBSD13_2:/data/infra $ ls -lrt dir1/
total 0
rtestuser@freeBSD13_2:/data/infra $ ls -lrt dir4_ren/


^C


Note: In next run got stuck in shutil.move.

rtestuser@freeBSD13_2:~ $ uname -r
13.2-RELEASE
rtestuser@freeBSD13_2:~ $ uname -a
FreeBSD freeBSD13_2.cs39 13.2-RELEASE FreeBSD 13.2-RELEASE
releng/13.2-n254617-525ecfdad597 GENERIC amd64
rtestuser@freeBSD13_2:~ $ top


last pid: 76719;  load averages:  2.02,  1.48,  1.04                           
                                                              up 0+00:27:10 
18:00:02
28 processes:  3 running, 25 sleeping
CPU: 19.5% user,  0.0% nice, 44.8% system,  2.4% interrupt, 33.3% idle
Mem: 25M Active, 7184K Inact, 2388K Laundry, 54M Wired, 12M Buf, 3496K Free
Swap: 723M Total, 13M Used, 710M Free, 1% Inuse

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND
60795 root          1 103    0    22M  8908K RUN      3:23 100.00% python3.9
 4544 rtestuser     1  20    0    21M  4576K select   0:00   0.00% sshd
76693 root          1  37    0    15M  2000K ufs      0:00   0.00% du
76694 root          1  36    0    15M  2000K ufs      0:00   0.00% du
76662 rtestuser     1  20    0    15M  4700K ufs      0:00   0.00% du
  758 root          1  20    0    16M  1540K ttyin    0:00   0.00% csh
 8447 root          1  20    0    18M  6888K wait     0:00   0.00% python3.9
  764 root          1  20    0    18M  3764K wait     0:00   0.00% python3.9
  702 root          1  20    0    18M  2172K select   0:00   0.00% sendmail
76699 root          1  21    0    21M  7924K select   0:00   0.00% sshd
 4396 root          1  21    0    21M  4452K select   0:00   0.00% sshd
 4550 rtestuser     1  21    0    13M  2464K wait     0:00   0.00% sh
  596 root          1  20    0    13M  1412K select   0:00   0.00% syslogd
76703 rtestuser     1  20    0    13M  2852K wait     0:00   0.00% sh
....

rtestuser@freeBSD13_2:/data/infra $ ls 
createfiles.py          dir2_ren                dir3_ren                du.py  
                move-move-delete.py
rtestuser@freeBSD13_2:/data/infra $ ls dir3_ren/
dir0    dir1    dir10   dir11   dir12   dir13   dir14   dir15
rtestuser@freeBSD13_2:/data/infra $ ls dir2_ren


^C
^C

System config: 128MB RAM, CPU core -2, Disk-250GB.
Thanks & regards,
Shamsher

-- 
You are receiving this mail because:
You are the assignee for the bug.