fusefs broken on 8-stable?
David P Discher
dpd at bitgravity.com
Thu Dec 29 21:36:06 UTC 2011
Johannes - Were you able to get a resolution for this ?
I've been able to replicate this panic in fiov_adjust(), but being called from vnlru, looks like its doing VOP_RECLAIM_APV().
I'm willing to dive deeper and debug this, however would appreciate some guidance form the community. I'm having issues wrapping my head around macro FUSE_DIMALLOC(), which I think is using some sort of anonymous function call to get to fiov_adjust(). I've been battling this for a week or two now, so I have symbols in fuse module, so I have a bit more info.
========================================
(kgdb) up
#11 0xffffffff81147412 in fdisp_make_pid (fdip=0xffffff9bce6399f0, mp=Variable "mp" is not available.
)
at fuse_msg.c:788
788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, fdip->finh,
(kgdb) l
783 if (fdip->tick)
784 fticket_refresh(fdip->tick);
785 else
786 fdip->tick = fuse_ticket_fetch(data);
787
788 FUSE_DIMALLOC(&fdip->tick->tk_ms_fiov, fdip->finh,
789 fdip->indata, fdip->iosize);
790 fuse_setup_ihead(fdip->finh, fdip->tick, nid, op, fdip->iosize, pid, cred);
791 }
792
(kgdb) p fdip
$4 = (struct fuse_dispatcher *) 0xffffff9bce6399f0
(kgdb) p *(struct fuse_dispatcher *)fdip
$5 = {tick = 0x0, finh = 0x0, indata = 0xffffff9bce639a30, iosize = 8,
nodeid = 0, answ_stat = 1356898584, answ = 0xffffff0350e0a138}
(kgdb) p *(struct fuse_dispatcher *)fdip->tick
Cannot access memory at address 0x0
==========================================
(kgdb) up
#9 0xffffffff807bd7b8 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:228
228 call trap
Current language: auto; currently asm
(kgdb) up
#10 0xffffffff81146b17 in fiov_adjust (fiov=0x30, size=48) at fuse_msg.c:96
96 {
Current language: auto; currently c
(kgdb) l
91 free(fiov->base, M_FUSEMSG);
92 }
93
94 void
95 fiov_adjust(struct fuse_iov *fiov, size_t size)
96 {
97 if (fiov->allocated_size < size ||
98 (fuse_iov_permanent_bufsize >= 0 &&
99 fiov->allocated_size - size > fuse_iov_permanent_bufsize &&
100 --fiov->credit < 0)) {
(kgdb) p fiov
$1 = (struct fuse_iov *) 0x30
(kgdb) p (struct fuse_iov *)fiov
$2 = (struct fuse_iov *) 0x30
(kgdb) p *(struct fuse_iov *)fiov
Cannot access memory at address 0x30
(kgdb) p size
$3 = 48
==========================================
(kgdb) bt
#0 doadump () at /usr/src/sys/kern/kern_shutdown.c:263
#1 0xffffffff801f6c9c in db_fncall (dummy1=Variable "dummy1" is not available.
)
at /usr/src/sys/ddb/db_command.c:548
#2 0xffffffff801f6fd1 in db_command (last_cmdp=0xffffffff80b67a40, cmd_table=Variable "cmd_table" is not available.
)
at /usr/src/sys/ddb/db_command.c:445
#3 0xffffffff801f7220 in db_command_loop ()
at /usr/src/sys/ddb/db_command.c:498
#4 0xffffffff801f9269 in db_trap (type=Variable "type" is not available.
) at /usr/src/sys/ddb/db_main.c:229
#5 0xffffffff80575ff1 in kdb_trap (type=12, code=0, tf=0xffffff9bce6398a0)
at /usr/src/sys/kern/subr_kdb.c:548
#6 0xffffffff807d4bad in trap_fatal (frame=0xffffff9bce6398a0, eva=Variable "eva" is not available.
)
at /usr/src/sys/amd64/amd64/trap.c:820
#7 0xffffffff807d4eea in trap_pfault (frame=0xffffff9bce6398a0, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:741
#8 0xffffffff807d52ff in trap (frame=0xffffff9bce6398a0)
at /usr/src/sys/amd64/amd64/trap.c:478
#9 0xffffffff807bd7b8 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:228
#10 0xffffffff81146b17 in fiov_adjust (fiov=0x30, size=48) at fuse_msg.c:96
#11 0xffffffff81147412 in fdisp_make_pid (fdip=0xffffff9bce6399f0, mp=Variable "mp" is not available.
)
at fuse_msg.c:788
#12 0xffffffff8114a6d4 in fuse_send_forget (mp=Variable "mp" is not available.
) at fuse_vnops.c:378
#13 0xffffffff8114a7d6 in fuse_recyc_backend (vp=0xffffff0350e0a000,
td=0xffffff0029533000) at fuse_vnops.c:539
#14 0xffffffff808272fa in VOP_RECLAIM_APV (vop=0xffffffff811504e0,
a=0xffffff9bce639a90) at vnode_if.c:1926
#15 0xffffffff805ccf95 in vgonel (vp=0xffffff0350e0a000) at vnode_if.h:830
#16 0xffffffff805cf9c4 in vnlru_free (count=28)
at /usr/src/sys/kern/vfs_subr.c:975
#17 0xffffffff805d030f in vnlru_proc () at /usr/src/sys/kern/vfs_subr.c:834
#18 0xffffffff805211bd in fork_exit (callout=0xffffffff805cfec0 <vnlru_proc>,
arg=0x0, frame=0xffffff9bce639c50) at /usr/src/sys/kern/kern_fork.c:876
#19 0xffffffff807bdcfe in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:602
==========================================
Fatal trap 12: page fault while in kernel mode^M
cpuid = 7; apic id = 15^M
fault virtual address = 0x40^M
fault code = supervisor read data, page not present^M
instruction pointer = 0x20:0xffffffff81146b17^M
stack pointer = 0x28:0xffffff9bce639950^M
frame pointer = 0x28:0xffffff9bce639980^M
code segment = base 0x0, limit 0xfffff, type 0x1b^M
= DPL 0, pres 1, long 1, def32 0, gran 1^M
processor eflags = interrupt enabled, resume, IOPL = 0^M
current process = 18 (vnlru)^M
[thread pid 18 tid 100138 ]
Stopped at fiov_adjust+0x17: movq 0x10(%rdi),%rax
db> [-- MARK -- Thu Dec 29 01:15:00 2011]
bt
Tracing pid 18 tid 100138 td 0xffffff0029533000
fiov_adjust() at fiov_adjust+0x17
fdisp_make_pid() at fdisp_make_pid+0xd2
fuse_send_forget() at fuse_send_forget+0x44
fuse_recyc_backend() at fuse_recyc_backend+0xc6
VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0x4a
vgonel() at vgonel+0xf5
vnlru_free() at vnlru_free+0x294
vnlru_proc() at vnlru_proc+0x44f
fork_exit() at fork_exit+0x11d
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff9bce639d00, rbp = 0 ---
db> call doadump
---
David P. Discher
dpd at bitgravity.com * AIM: bgDavidDPD
BITGRAVITY * http://www.bitgravity.com
On Jun 3, 2011, at 6:49 PM, Johannes Totz wrote:
> On 13/05/2011 13:56, Johannes Totz wrote:
>> Heya!
>>
>> Using encfs (built on top of fuse) gives me panics in combination with
>> rsync. Dump didn't succeed. The info below is transcribbled from a
>> photograph. This is repeatable.
>> Without dump this is probably not very helpful....
>>
>>
>> # uname -a
>> FreeBSD XXX 8.2-STABLE FreeBSD 8.2-STABLE #1: Thu Mar 10 23:30:08 GMT
>> 2011 root at XXX:/usr/obj/usr/src/sys/GENERIC amd64
>
> Managed to get a dump:
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 1; apic id = 01
> fault virtual address = 0x40
> fault code = supervisor read data, page not present
> instruction pointer = 0x20:0xffffffff81f2ba3c
> stack pointer = 0x28:0xffffff813aa2f570
> frame pointer = 0x28:0xffffff813aa2f5a0
> code segment = base 0x0, limit 0xfffff, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process = 1560 (rsync)
> trap number = 12
> panic: page fault
> cpuid = 1
> KDB: stack backtrace:
> #0 0xffffffff805cc226 at kdb_backtrace+0x5c
> #1 0xffffffff8059d000 at panic+0x1b4
> #2 0xffffffff80875a75 at trap_fatal+0x394
> #3 0xffffffff80875ce5 at trap_pfault+0x252
> #4 0xffffffff808761b5 at trap+0x3f4
> #5 0xffffffff8085f384 at calltrap+0x8
> #6 0xffffffff81f2c170 at fdisp_make+0xe4
> #7 0xffffffff81f3033d at fuse_lookup+0x1dc
> #8 0xffffffff808caac8 at VOP_LOOKUP_APV+0x4c
> #9 0xffffffff8061b0db at lookup+0x61e
> #10 0xffffffff8061bf1f at namei+0x592
> #11 0xffffffff80629df4 at kern_statat_vnhook+0x87
> #12 0xffffffff80629f98 at kern_statat+0x15
> #13 0xffffffff80629fb6 at kern_lstat+0x1c
> #14 0xffffffff8062a03d at lstat+0x25
> #15 0xffffffff805d7cf4 at syscallenter+0x2d9
> #16 0xffffffff80875d45 at syscall+0x38
> #17 0xffffffff8085f662 at Xfast_syscall+0xe2
>
>
> #0 doadump () at pcpu.h:224
> #1 0xffffffff8059cd12 in boot (howto=260) at
> /usr/src/sys/kern/kern_shutdown.c:419
> #2 0xffffffff8059d03b in panic (fmt=Variable "fmt" is not available.)
> at /usr/src/sys/kern/kern_shutdown.c:592
> #3 0xffffffff80875a75 in trap_fatal (frame=0xffffff813aa2f4c0, eva=64)
> at /usr/src/sys/amd64/amd64/trap.c:809
> #4 0xffffffff80875ce5 in trap_pfault (frame=0xffffff813aa2f4c0,
> usermode=0) at /usr/src/sys/amd64/amd64/trap.c:725
> #5 0xffffffff808761b5 in trap (frame=0xffffff813aa2f4c0) at
> /usr/src/sys/amd64/amd64/trap.c:475
> #6 0xffffffff8085f384 in calltrap () at
> /usr/src/sys/amd64/amd64/exception.S:228
> #7 0xffffffff81f2ba3c in fiov_adjust () from /usr/local/modules/fuse.ko
> #8 0xffffffff81f2c170 in fdisp_make () from /usr/local/modules/fuse.ko
> #9 0xffffffff81f3033d in fuse_lookup () from /usr/local/modules/fuse.ko
> #10 0xffffffff808caac8 in VOP_LOOKUP_APV (vop=0x0, a=0x30) at vnode_if.c:123
> #11 0xffffffff8061b0db in lookup (ndp=0x618) at vnode_if.h:54
> #12 0xffffffff8061bf1f in namei (ndp=0xffffff813aa2f8e0) at
> /usr/src/sys/kern/vfs_lookup.c:269
> #13 0xffffffff80629df4 in kern_statat_vnhook (td=0xffffff00039f2460,
> flag=Variable "flag" is not available.) at
> /usr/src/sys/kern/vfs_syscalls.c:2346
> #14 0xffffffff80629f98 in kern_statat (td=Variable "td" is not
> available.) at /usr/src/sys/kern/vfs_syscalls.c:2327
> #15 0xffffffff80629fb6 in kern_lstat (td=Variable "td" is not
> available.) at /usr/src/sys/kern/vfs_syscalls.c:2400
> #16 0xffffffff8062a03d in lstat (td=Variable "td" is not available.) at
> /usr/src/sys/kern/vfs_syscalls.c:2390
> #17 0xffffffff805d7cf4 in syscallenter (td=0xffffff00039f2460,
> sa=0xffffff813aa2fbc0) at /usr/src/sys/kern/subr_trap.c:315
> #18 0xffffffff80875d45 in syscall (frame=0xffffff813aa2fc50) at
> /usr/src/sys/amd64/amd64/trap.c:914
> #19 0xffffffff8085f662 in Xfast_syscall () at
> /usr/src/sys/amd64/amd64/exception.S:381
> #20 0x000000080097a5dc in ?? ()
>
> At #10 a=0x30 looks dodgy and fault address is 0x40, but I have no idea
> what that means...
>
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
More information about the freebsd-fs
mailing list