panic: devfs_fsync: vop_stdfsync failed.

Adam Parco aparco at sandvine.com
Mon Oct 27 15:16:12 UTC 2014 

Hello,

I am investigating making FreeBSD 8.2 more resilient to removing a USB during a write.  In my short testing I have gotten 3 different failures.  I would like to discuss potential solutions for the first failure.  I looked at head source and there doesn't appear to be any fixes for this particular issue.

>From what I have gathered, it looks like:

-          the filesystem synchronizer daemon wakes up

-          tries to sync vnodes

-          devfs_fsync realizes the device went away with dirty set, warns of data loss:

o   "Device da0s1 went missing before all of the data could be written to it; expect data loss."

-          vop_stdfync loops a couple times trying to handle the dirty items

-          eventual gives up

o   "fsync: giving up on dirty"

-          Dirty count will still be >0 and errno should be EAGAIN (not 0)

-          Panic

o   "devfs_fsync: vop_stdfsync failed."

When we give up on dirty, should we be clearing the dirty count?  Otherwise we will always panic after this.  Thoughts?  Other suggestions?

Thanks,
Adam.

bt:
#0  doadump () at pcpu.h:224
#1  0xffffffff8045fbe9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:508
#2  0xffffffff8046011d in panic (fmt=0x1 <Address 0x1 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:775
#3  0xffffffff803ec3db in devfs_fsync (ap=0xffffff862b3edb20) at /usr/src/sys/fs/devfs/devfs_vnops.c:569
#4  0xffffffff8069c8ca in VOP_FSYNC_APV (vop=0xffffffff80892fa0, a=0xffffff862b3edb20) at vnode_if.c:1267
#5  0xffffffff804e3e27 in sync_vnode (slp=0xffffff0006136af0, bo=0xffffff862b3edbc0, td=0xffffff0006005480) at vnode_if.h:549
#6  0xffffffff804e406d in sched_sync () at /usr/src/sys/kern/vfs_subr.c:1841
#7  0xffffffff80438ed2 in fork_exit (callout=0xffffffff804e3ec0 <sched_sync>, arg=0x0, frame=0xffffff862b3edc50) at /usr/src/sys/kern/kern_fork.c:847
#8  0xffffffff80633dbe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:599

console:
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Kingston DataTraveler G3 PMAP> Removable Direct Access SCSI-0 device
da0: 40.000MB/s transfers
da0: 7441MB (15240576 512 byte sectors: 255H 63S/T 948C)
[-- MARK -- Fri Oct 24 13:39:00 2014]
ugen1.3: <Kingston> at usbus1 (disconnected)
umass0: at uhub3, port 1, addr 3 (disconnected)
(da0:umass-sim0:0:0:0): AutoSense failed
g_vfs_done():da0s1[WRITE(offset=50139136, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50204672, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50270208, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50335744, length=65536)](da0:umass-sim0:0:0:0): lost device error = 5
g_vfs_done():da0s1[WRITE(offset=50401280, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50466816, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50532352, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50597888, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50663424, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50728960, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50794496, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50860032, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50925568, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50991104, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=50073600, length=65536)]error = 5
g_vfs_done():da0s1[WRITE(offset=51056640, length=65536)]error = 6
g_vfs_done():da0s1[WRITE(offset=51122176, length=65536)]error = 6
(da0:umass-sim0:0:0:0): Synchronize cache failed, status == 0xa, scsi status == 0x0
(da0:umass-sim0:0:0:0): removing device entry
g_vfs_done():[unknown][WRITE(offset=51187712, length=65536)]error = 6
g_vfs_done():[unknown][WRITE(offset=51253248, length=65536)]error = 6
g_vfs_done():[unknown][WRITE(offset=51318784, length=65536)]error = 6
Device da0s1 went missing before all of the data could be written to it; expect data loss.
fsync: giving up on dirty
0xffffff08c7eec1d8: tag devfs, type VCHR
    usecount 1, writecount 0, refcount 4 mountedhere 0xffffff0109ddf400
    flags (VI_DOOMED)
    v_object 0xffffff08c7ec7438 ref 0 pages 2169
   lock type devfs: EXCL by thread 0xffffff0adf0b6900 (pid 64960)
                dev da0s1
panic: devfs_fsync: vop_stdfsync failed.
pci: 1752 correctable, 0 uncorrectable, 0 fatal
cpu: 2 correctable, 0 uncorrectable, 0 fatal
cpuid = 4
curthread = getty/getty (64960/100629)
cpu_ticks = 19048675905756
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff801e51ca = db_trace_self_wrapper+0x2a
panic() at 0xffffffff80460148 = panic+0x228
devfs_fsync() at 0xffffffff803ec3db = devfs_fsync+0x8b
VOP_FSYNC_APV() at 0xffffffff8069c8ca = VOP_FSYNC_APV+0x4a
bufsync() at 0xffffffff804cbcd8 = bufsync+0x38
bufobj_invalbuf() at 0xffffffff804e1997 = bufobj_invalbuf+0x87
vgonel() at 0xffffffff804e1c56 = vgonel+0xb6
vgone() at 0xffffffff804e1e89 = vgone+0x39
devfs_delete() at 0xffffffff803eab89 = devfs_delete+0x189
devfs_populate_loop() at 0xffffffff803eb37d = devfs_populate_loop+0x3ad
devfs_populate() at 0xffffffff803eb461 = devfs_populate+0x21
devfs_lookup() at 0xffffffff803eeb94 = devfs_lookup+0x2d4
VOP_LOOKUP_APV() at 0xffffffff8069e9bc = VOP_LOOKUP_APV+0x4c
lookup() at 0xffffffff804d7eea = lookup+0x37a
namei() at 0xffffffff804d8cff = namei+0x3bf
vn_open_cred() at 0xffffffff804ed583 = vn_open_cred+0x1e3
kern_openat() at 0xffffffff804eaab9 = kern_openat+0x149
syscallenter() at 0xffffffff8049bad4 = syscallenter+0x104
syscall() at 0xffffffff8064a15c = syscall+0x4c
Xfast_syscall() at 0xffffffff80633b52 = Xfast_syscall+0xe2
--- syscall (5, FreeBSD ELF64, open), rip = 0x300845cfc, rsp = 0x7fffffffecd8, rbp = 0x5086a0 ---
Uptime: 2h26m42s
Physical memory: 49040 MB
Dumping 1934 MB: 1919 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071[-- MARK -- Fri Oct 24 13:40:00 2014]
1055 1039 1023 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15
Dump complete

More information about the freebsd-fs mailing list