zpool import taking weeks ...

[Volodymyr Kostyrko]

15.01.2014 10:29, Ulysse 31 wrote:
> Thanks for the answer.
>
> I was beginning to ... well ... have a little start of panic ^^'
> The import still running but no real activity on disks : sometimes,
> maybe something like each hour, there is some really brief led
> activity on the hdd, but nothing more.
> one thing that would be great is to have an overall percentage or just
> counting of what is done or what is left ... for now i don't even know
> if it's gonna end on the next 5 mins or on the next 5 weeks :s
> So now since i have no real activity on the actual zpool import (the
> last import started a week ago, it do not have go out of swap since i
> added lot of swap (260G HDD swap)), should i just let it run, or make
> a reboot and start over ?
> Also once imported, and since i don't care about the data actually on
> the incriminated dataset (the one that crashed with the
> recv/rollback), I was planning to simply delete that dataset and make
> a new full sync, what do you think about it ?
> Thanks again for your help.

You odds are grim. I had the same problem with crashed ZFS when dropping 
some snapshots (deduplicated ones). The situation looks like:

1. Boot. Import start.
2. Memory depletes.
3. When memory is over the process is stuck or machine crashes. In your 
case no crash means you are running a lil bit newer code.

All of this comes in something like 15 minutes. My machine was at 4G of 
memory back then. I added 4G more and process successfully completed in 
near 20 minutes eating ~6G of mem. The culprit was deleting deduplicated 
snapshot worth 14G of disk space.

I personally think that previous versions of ZFS tend to interpret all 
snapshot operations as atomic ones meaning all changes to fs should 
happen simultaneously thus pushing to memory:

- all old dedup data that still can be accessed,
- all new dedup data that should be active after the change,
- all metadata of filesystem in question.

I'm not a pro in this and haven't retested this situation after the 
recent changes with snapshots (like background deleting). But this was a 
final grave reason for me to stop thinking about dedup. Or at least in 
ZFS as I still like the way HAMMER does it.

-- 
Sphinx of black quartz, judge my vow.