FreeBSD 10.0-STABLE on EdgeRouter Lite notes/issues
Warner Losh
imp at bsdimp.com
Thu Aug 28 20:06:35 UTC 2014
On Aug 28, 2014, at 12:59 PM, Stacey Son <sson at FreeBSD.org> wrote:
> Hi Sean:
>
> Some answers to your questions are inline below...
>
> On Aug 24, 2014, at 4:23 PM, Sean Hamilton <seanhamilton at gmail.com> wrote:
>
>> Running 10.0-STABLE r270425 on an EdgeRouter Lite.
>>
>> "makefs -B big -o version=2" seems to produce a broken image which
>> cannot be mounted read/write. See misc/162503, bin/188762. FS
>> performance on USB flash is awful without softupdates, so UFS2 is
>> effectively mandatory. I built a rescue-only mfsroot and newfs'd and
>> untar'd the real system from there. Flash performance still isn't
>> great, but it is tolerable. Mounting noatime helps. Were it not for
>> the makefs bug, this process would be much easier.
>>
>> Attempting to makefs directly onto the USB flash is painfully slow --
>> writing a file, then dd'ing that to the device is much faster. I
>> assume makefs does a lot of small writes. Can the OS cache and combine
>> these? It would also be nice if makefs could detect the size of the
>> target device and set the FS size and number of inodes appropriately.
>> Of course at that point it's basically newfs.
>
>
> The way I do it is create a FS only the size that is needed and 'dd' to USB then 'growfs' later.
>
> Nathan Dorfman's script makes it easy to make images...
>
> http://rtfm.net/FreeBSD/ERL/mkerlimage
>
> You may have also have seen his webpage on the ERL:
>
> http://rtfm.net/FreeBSD/ERL
>
>> Also for some reason, ERL u-boot's tftpboot seems to refuse to talk to
>> FreeBSD's tftpd, so I had to copy my rescue-only image to the flash. I
>> didn't investigate this closely, could be operator error.
>>
>> Any suggestions for VFS tuning on ERL? Ideally the OS could
>> aggressively cache filesystem writes, then lazily write them out
>> asynchronously. Would softupdate journaling be of any benefit?
>>
>> gpart insists on aligning MBR partitions to cylinder boundaries, and
>> seemingly ignores -a. Is there some way to prevent this? If not, can
>> it be fooled by overriding the disk geometry? I'm not sure how much
>> this matters on USB flash devices, but it matters a lot on SSDs. I
>> wound up partitioning with GNU parted, since it does what it's told.
>>
>> I would prefer to use GPT, but it seems like ERL u-boot doesn't
>> support it. Another option might be to use GPT, create a freebsd-boot
>> partition, dd the kernel directly there, then just load the sectors
>> directly with u-boot, bypassing the filesystem entirely.
>
> Yeah, the ERL u-boot seems kind of simple. You may have noticed that 'mkerlimage' puts /boot in a FAT partition. Once the kernel is loaded you can do about any partitioning scheme you want. IDK. It would be nice if the ERL had a second USB or some onboard flash that could be used or something.
>
>> Ports are totally unbuildable on ERL. This seems to be a toolchain
>> problem. ports-mgmt/pkg (which everything depends on) fails to build
>> with:
>>
>> libtool: install: ranlib
>> /usr/ports/ports-mgmt/pkg/work/stage/usr/local/lib/libpkg_static.a
>> ranlib: fatal: Invalid filename
>>
>> I imagine this relates to the recent freebsd-mips posts about strip
>> corrupting static binaries.
>
> Yes, this is most likely the 'strip' bug. See https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191587 for the details. The workaround is to not strip static libraries.
>
> FYI, you can find some ports I have built for FreeBSD/mips64 (using qemu user-mode) at:
>
> http://www.cl.cam.ac.uk/research/security/ctsrd/mips64-packages/
>
> Sean Bruno may even have more some place.
>
>> tcpdump is broken, as the packet lengths it reports are incorrect, and
>> it eventually segfaults. I thought this might be an issue with the
>> octe driver, but it seems even the loopback device is afflicted, so
>> perhaps it's an endianness issue?
>
> You should file a bugzilla bug report especially if you have a packet trace that can be used to easily reproduce the problem.
>
>> Regarding https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=177876 --
>> is this patch still a good idea? Or has it been merged?
>
> I have been using it for some time. If you plan on mounting and using NFS volumes or anything else the might take a lot of kernel thread stack space then it is a good idea.
>
> A better version of this patch (that moves the mips dependent code out of the vm layer) has been merged into a MIPS64 FreeBSD project and that code will be upstreamed sometime in the near future. You can find the changes here:
>
> https://github.com/CTSRD-CHERI/cheribsd/commit/b39bec2cefe36293ffb2322969c9f7c9cae8c1a2
>
> https://github.com/CTSRD-CHERI/cheribsd/commit/04ea99d04f6d5a595be759d34e84e7fd9cea8166
I don’t suppose this could be done w/o adding #ifdef __mips__ to vm_glue? No need to block upstreaming on this, but it seems this could be a MD routine instead...
> https://github.com/CTSRD-CHERI/cheribsd/commit/ae4f8d84bc30ffe2b2698534a2e6ab64480e8432
On the whole, I really like this, but was hoping it would be cleaner. There’s a crying need for it…
Warner
> For now, I would just used the kernel patch at:
>
> https://people.freebsd.org/~sson/mips/kstack/kstack_large_page.diff
>
>> Is there an estimate as to how many people are running FreeBSD on ERL?
>> Many thanks in advance for suggestions on any of the above!
>
> Good question. I know of three or so.
>
> -stacey.
>
> _______________________________________________
> freebsd-mips at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-mips
> To unsubscribe, send any mail to "freebsd-mips-unsubscribe at freebsd.org"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-mips/attachments/20140828/969cdf95/attachment.sig>
More information about the freebsd-mips
mailing list