[Bug 226110] Observing ASSERT at cfiscsi_session_delete+0x49 while running traffic on 100 iSCSI LUNs

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226110

            Bug ID: 226110
           Summary: Observing ASSERT at cfiscsi_session_delete+0x49 while
                    running traffic on 100 iSCSI LUNs
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: freebsd-bugs at FreeBSD.org
          Reporter: manishk at chelsio.com
                CC: trasz at FreeBSD.org

Created attachment 190884
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=190884&action=edit
core text file

Observing an assert while running traffic on 100 iSCSI LUNs (used zvols as
LUNs) on the target machine. The panic stack trace:  

panic: destroying session with outstanding CTL pdus
cpuid = 4
time = 1519157619
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe005f30f880
kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe005f30f950
vpanic() at vpanic+0x268/frame 0xfffffe005f30fa20
kassert_panic() at kassert_panic+0xc7/frame 0xfffffe005f30fab0
cfiscsi_session_delete() at cfiscsi_session_delete+0x49/frame
0xfffffe005f30fae0
cfiscsi_maintenance_thread() at cfiscsi_maintenance_thread+0x110/frame
0xfffffe005f30fb30
fork_exit() at fork_exit+0x145/frame 0xfffffe005f30fbb0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe005f30fbb0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic

-------------------------------------
Steps to recreate the issue:
-------------------------------------
The target machine exposes 100 zvols as iSCSI LUNs. The Initiator discovers
these LUNS, creates file system on them, and runs traffic (iozone). The target
machine asserts after few hours.

OS: FreeBSD 12.0-CURRENT (svn r329019)
I'm using GENERIC config file, with "nooptions       VIMAGE"

==>Start the Target (FreeBSD)
    1. Create 100 zvols
      # zpool create iscsi <slice/partition>
      # for ((i=0;i<100;i++)); do zfs create -V1G -o volmode=dev iscsi/d$i;
done

    2. configure the network interface to be used (Target_IP_Address)

    3. /etc/ctl.conf file:
         portal-group pg0 {
            discovery-auth-group no-authentication
            listen <Target_IP_Address>
         }
         target iqn.2016-11.com.xyz.abc:0 {
            auth-group no-authentication
            portal-group pg0
                     lun 0 {
                              path /dev/zvol/iscsi/d0
                              size 1G
                              option vendor "foo"
                              option product "bar"
                              option revision "d0"
                     }
         }
         .
         .

        100 targets with 1 lun each

    4. start the target daemon 
        # service ctld onestart

==>Initiator (Linux)
    1. login/discover the target 
        # iscsiadm -m discovery -t st -p <Target_IP_Address> -l
    2. List all Luns 
        # lsscsi        
    3. Format and create file system ONLY on all discovered luns (TAKE CARE TO
NOT TOUCH OTHER LUNS). 
        # mkfs.ext3 /dev/sdxx      // for all discovered LUNs
    4. mount all the devices formatted in the above step to different mount
points
        # mount /dev/sdxx /mnt/iscsiyy    // for all the luns formatted in the
above step
    5. Run traffic on all luns using iozone
        # cd /mnt/iscsiyy
        # iozone -a -I -+d -g 512m &


==> Target machine asserts after 6-7 hours.

-- 
You are receiving this mail because:
You are the assignee for the bug.