poudriere bug [WAS: Re: poudriere zfs setup help ]

[Anton Shterenlikht]

>From bsam at passap.ru Fri Oct 25 11:47:07 2013
>>>>>> I'm getting this error:
>>>>>> => SHA256 Checksum OK for pkg-1.1.4.tar.xz.
>>>>>> mkdir: /usr/ports/ports-mgmt/pkg/work: Read-only file system
>>>>>> *** Error code 1
>>>>>>
>>>>>> So something is wrong with my setup?
>>>>>> Does this look all right:
>>>>>>
>>>>>> # poudriere jails -l
>>>>>> JAILNAME             VERSION              ARCH    METHOD  PATH
>>>>>> ia64                 10.0-ALPHA5 r256218  ia64    svn+https /pdr/jails/ia64
>>>>>> # poudriere ports -l
>>>>>> PORTSTREE            METHOD     PATH
>>>>>> default              svn+https  /pdr/ports/default
>>>>>> # df
>>>>>> Filesystem                   1K-blocks     Used    Avail Capacity  Mounted on
>>>>>> /dev/da0p2                    60931274 49946430  6110344    89%    /
>>>>>> devfs                                1        1        0   100%    /dev
>>>>>> /dev/da0p1                      409504     1184   408320     0%    /efi
>>>>>> tank/poudriere/data           31736572     1290 31735281     0%    /pdr/data
>>>>>> tank/poudriere/jails/ia64     33931466  2196185 31735281     6%    /pdr/jails/ia64
>>>>>> tank/poudriere/ports/default  32633749   898468 31735281     3%    /pdr/ports/default
>>>>>> tank                          31735313       32 31735281     0%    /tank
>>>>>> tank/poudriere                31735314       33 31735281     0%    /tank/poudriere
>>>>>> tank/poudriere/jails          31735312       31 31735281     0%    /tank/poudriere/jails
>>>>>> tank/poudriere/ports          31735312       31 31735281     0%    /tank/poudriere/ports
>>>>>> # 
>>>>>>
>>>>>> # grep "^[A-Z]" /usr/local/etc/poudriere.conf
>>>>>> ZPOOL=tank
>>>>>> FREEBSD_HOST=https://svn0.eu.freebsd.org
>>>>>> RESOLV_CONF=/etc/resolv.conf
>>>>>> BASEFS=/pdr
>>>>>> USE_PORTLINT=yes
>>>>>> USE_TMPFS=yes
>>>>>> DISTFILES_CACHE=/usr/ports/distfiles
>>>>>> SVN_HOST=svn0.eu.FreeBSD.org
>>>>>> CHECK_CHANGED_OPTIONS=verbose
>>>>>> CHECK_CHANGED_DEPS=yes
>>>>>> PKG_REPO_SIGNING_KEY=/root/pkg.key
>>>>>> CCACHE_DIR=/var/cache/ccache
>>>>>> SAVE_WRKDIR=yes
>>>>>> # 
>>>>>>
>>>>>> What am I missing?
>>>>>>
>>>>>> On another box I set up poudriere with no zfs
>>>>>> and it works all right. So I think I messed up
>>>>>> the zfs setup.
>>>>>
>>>>> You may try to use "poudriere testport -i ...". Then poudriere will
>>>>> not umount jail (it will be umounted when you exit the jail command
>>>>> propmt) and you may explore the system (i.e. mount, jls, etc.) to
>>>>> find out what's going on.
>>>>
>>>> ok, so after a failure, still in a jail:
>>>>
>>>> root at ia64-default:~ # df
>>>> Filesystem                            1K-blocks    Used    Avail Capacity  Mounted on
>>>> tank/poudriere/jails/ia64-default-ref  33930819 2198542 31732277     6%    /
>>>> root at ia64-default:~ # ls -al /usr/ports/ports-mgmt/pkg
>>>> total 15
>>>> drwxr-xr-x   3 root  wheel     8 Oct  9 13:23 .
>>>> drwxr-xr-x  85 root  wheel    86 Oct  9 13:23 ..
>>>> -rw-r--r--   1 root  wheel  2544 Oct  9 13:23 Makefile
>>>> -rw-r--r--   1 root  wheel   127 Oct  9 13:23 distinfo
>>>> drwxr-xr-x   2 root  wheel    13 Oct  9 13:23 files
>>>> -rw-r--r--   1 root  wheel    87 Oct  9 13:23 pkg-descr
>>>> -rw-r--r--   1 root  wheel    73 Oct  9 13:23 pkg-message
>>>> -rw-r--r--   1 root  wheel  1420 Oct  9 13:23 pkg-plist
>>>> root at ia64-default:~ # whoami
>>>> root
>>>> root at ia64-default:~ # mkdir /usr/ports/ports-mgmt/pkg/work
>>>> mkdir: /usr/ports/ports-mgmt/pkg/work: Read-only file system
>>>> root at ia64-default:~ # 
>>>>
>>>> On ufs this would mean something is seriously wrong.
>>>> But on zfs, I'm not so sure.
>>>> Perhaps I'm not understanding it right.
>>>
>>> Actually I meant to investigate host, not jail. Jail ment to just
>>> exist. Sorry to be not clear. (I usually use sysutils/tmux to have
>>> one seccion and several terminals at a host.) So look at commands
>>> (at the host, not in the jail): mount, jls. Does somtethig looks
>>> suspicious?
>> 
>> Still no luck. Now I cannot even get 'poudriere testport -i'
>> to leave the jail intact. I get:
>> 
>> # poudriere testport -i -o ports-mgmt/pkg -j ia64
>> ====>> Creating the reference jail... done
>> *skip*
>> => SHA256 Checksum OK for pkg-1.1.4.tar.xz.
>> mkdir: /usr/ports/ports-mgmt/pkg/work: Read-only file system
>> *** Error code 1
>> 
>> Stop.
>> make[1]: stopped in /usr/ports/ports-mgmt/pkg
>> *** Error code 1
>> 
>> Stop.
>> make: stopped in /usr/ports/ports-mgmt/pkg
>> ====>> Cleaning up
>> ====>> Umounting file systems
>> # 
>> 
>> I also tried "-I" option - still the same - the jail
>> is unmounted after the failure.
>> 
>> This is poudriere-3.0.11.
>> 
>> What else can I try?
>
>Investigate your system as I had described earlier.

The root cause of this seems to be that
the poudriere ports tree is nullfs mounted read-only.
I don't know why, but this seems to be a bug.
I updated this ticket, which seems to report a similar issue:

https://fossil.etoilebsd.net/poudriere/tktview?name=15bf9880c4

Thanks

Anton