mfi0 timeout error zfs boot mount problem

Tue Oct 23 09:04:41 UTC 2012

The usb shutdown issue could well be down to the new cleanup code in
9.x and a bad driver.

You could try sysctl hw.usb.no_shutdown_wait=1 to see if that helps
did here, although be aware this may have other implications.

    Regards
    Steve
----- Original Message ----- 
From: "Jürgen Weber" <jurgen.weber at theiconic.com.au>
To: <freebsd-fs at freebsd.org>
Sent: Tuesday, October 23, 2012 5:01 AM
Subject: Re: mfi0 timeout error zfs boot mount problem

Hi

Well, while this thread has been very quiet I have resolved my issues.
With patience changing:

kern.maxfiles=5000000
kern.maxvnodes=5000000
vfs.zfs.zil_disable="1"
vfs.zfs.prefetch_disable="1"
vfs.zfs.txg.timeout="5"
The above solves the system unable to import or mount the pool.

I have also gone into the Card settings BIOS and changed under advanced
settings "Forward Read" to "none". This solves the mfi0 timeout.

Once I had the system up, I then added a l2arc cache via a usb2 SSD HDD.

I then shut hte system down and it took 3 hours to shut down.. but it
eventually did.

When I turned the system back on again, it booted as normal.

The lesson learnt?! Do not turn on deduping on a large file system
unless you have a lot of RAM or L2ARC! I would say 32GB of RAM/L2ARC for
every 10TB as a good rule of thumb, if not... double.

Thanks

Jurgen
On 22/10/12 12:30, Jürgen Weber wrote:
> Some more updates!
>
> on the bootloader I have also tried:
> kern.maxfiles=5000000
> kern.maxvnodes=5000000
>
> I have also gone into the Card settings BIOS and changed under advanced settings "Forward Read" to "none".
>
> Now the systems gets to
>
> "Trying to mount root from zfs:tank/root []..... " and then after maybe 1 to 5 minutes the next couple of lines load like its 
> working!
>
> eg:
> "Setting hostuuid: xxxxx"
> "Setting hostid: xxxxxx"
> "Entropy harvesting:interrupts ethernet point_to_point kickstart"
> "Starting file system checks:"
> "Mounting local file systems:."
>
> and stops. I have had the machine on my desk all morning observing it and I can see the disk access is going crazy,, it is doing 
> something.
>
> I have found this article:
>
> http://constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe
>
> I have a 15TB file system which has dedup on from the start (10TB. I feel its trying to load the DDT and its going to swap/there 
> is not enough RAM (only have 16GB's). Hopefully my 64GB RAM upgrade is enough.
>
> Thanks
>
> Jurgen
>
>
>
> On 22/10/12 09:06, Jürgen Weber wrote:
>> This is still a problem for me, is anyone there? :)
>>
>> I have tried the following at the bootime loader.
>>
>> vfs.zfs.zil_disable="1"
>> vfs.zfs.prefetch_disable="1"
>> vfs.zfs.txg.timeout="5"
>>
>> Any other suggestions on how to get this zpool to import and mount again?
>>
>> Thanks
>>
>> On 21/10/12 07:50, Jurgen Weber wrote:
>>> Hi
>>>
>>> Lastly, is there a way at boot time, some sysctl's or something I can set to bring zfs to a minimalistic state? Turn off 
>>> features, etc to get this to mount?
>>>
>>> Any ideas appreciated.
>>>
>>> Thanks
>>>
>>> Jurgen
>>> On 20/10/2012 9:02 AM, Jurgen Weber wrote:
>>>> Guys
>>>>
>>>> Some more details on this, some insight would be greatly appreciated.
>>>>
>>>> As my day wore on trying to get this zpool to import or mount I have learnt a few things. I think over time this issue has 
>>>> came about as more and more data was added to the file systems.
>>>>
>>>> Some further details:
>>>>
>>>> Its a 8 disk raidz pool that the system boots from as well. The disk are all 2TB.
>>>> The server has 16GB Of RAM, I notcied the day before this happen the server was struggling with its RAM griding to a halt and 
>>>> dumping its RAM.
>>>> The issue is not hardware because I found another server (same one) swapped the harddrives out took another 8GB of RAM and I 
>>>> have the same problem.
>>>> The main data file systems have dedup and gzip compression on.
>>>>
>>>> I have booted from open/Oracle Solars 11 adn attempted to import and the Solaris live CD will not import either. In the 
>>>> Solaris system the disk detach from the system.
>>>>
>>>> I get the feeling that ZFS is hitting some root limit when attempting to mount and its not finishing the job.
>>>>
>>>> Thanks
>>>>
>>>> Jurgen
>>>>
>>>> On 19/10/2012 10:29 AM, Jürgen Weber wrote:
>>>>> Team
>>>>>
>>>>> I have googled around for a solution and I see a lot of posts about firmware versions and patches for FreeBSD 8.*.
>>>>>
>>>>> I have a FreeBSD 9.1rc1 system, which was beta1 orginally and has been running for months.
>>>>>
>>>>> Now it will not boot, I get the following:
>>>>>
>>>>> "Trying to mount root from zfs:tank/root [].....
>>>>> mfi0: COMMAND 0Xffffff8000cb83530 TIMEOUT AFTER xxx SECONDS
>>>>> (this just repeats).
>>>>>
>>>>> I have not seen this error before during normal runtime, _only_ during boot.
>>>>>
>>>>> Originally when I had the problem I could boot off a USB stick (9.1beta1 or rc1), run a 'zpool import -f tank' and it would 
>>>>> work on the livecd. Rebooting and the main system would work.
>>>>>
>>>>> This time this work around does not work for me. When I am on the USB stick I can run a 'zpool import' and all of the disk 
>>>>> are recognised, the pool is recognised and the file system is healthy.
>>>>>
>>>>> The Card is a H700 PERC, with 12.10.3 firmware in a Dell R515.
>>>>> Running FreeBSD 9.1-RC1, latest zfs and zpool versions.
>>>>>
>>>>> I have tried disabling the cache (mfiutil cache xxx disable). I have also gone into the Card settings and changed under 
>>>>> advanced settings "adaptive forward read" to "read only".
>>>>>
>>>>> Any help, appreciated.
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>
>>
>
> -- 
> Jürgen Weber
>
> Systems Engineer
> IT Infrastructure Team Leader
>
> THE ICONIC | Ejurgen.weber at theiconic.com.au  |www.theiconic.com.au

-- 
Jürgen Weber

Systems Engineer
IT Infrastructure Team Leader

THE ICONIC | E jurgen.weber at theiconic.com.au | www.theiconic.com.au

_______________________________________________
freebsd-fs at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"

================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster at multiplay.co.uk.