FreeBSD on the AP121 (AR9330)

[Warner Losh]

On Mar 27, 2013, at 10:23 PM, Adrian Chadd wrote:

> On 27 March 2013 20:22, Warner Losh <imp at bsdimp.com> wrote:
> 
>> I was able to save about 40k by uninlining mutexes, etc. But that took the AP96 kernel from 6.5MB to 6.4MB.
> 
> The AP96 is an all-in-one kernel for a much more useful system. The
> AP91 is the kicker. Same architecture, but I have to shrink the kernel
> down to fit inside an 896k lzma'ed partition. That, and 16MB of RAM
> makes it very, very tight.
> 
>> 4680311  266388 1576752 6523451  638a3b /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/kernel
>> 4641469  266372 1576624 6484465  62f1f1 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/kernel
> 
> Ok, I'll try that. I wonder how badly it's going to affect performance
> though. :-(

On the Atmel AT91RM9200 running at 180MHz it didn't affect things much...

>> Here's the top 10 in terms of text size:
>> 
>>  57344     160   49184  106688   1a0c0 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/kern_umtx.o
> 
> This is likely due to the default size of the mtx array. I dropped
> that from 512 to 64, no appreciable drop. :-(

Well, the 57k of text isn't going to be affected by that much at all...

>>  57004     848      64   57916    e23c /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/pci.o
>>  48956   10672      80   59708    e93c /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/scsi_all.o
>>  48664    1680     256   50600    c5a8 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/vfs_subr.o
>>  45156     624       0   45780    b2d4 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/if_ath.o
>>  44932    2000     320   47252    b894 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/vfs_bio.o
>>  41796     992     192   42980    a7e4 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/ffs_alloc.o
>>  41376       0       0   41376    a1a0 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/if_ath_tx.o
>>  38272    5120      80   43472    a9d0 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/kern_jail.o
>>  34340     752     192   35284    89d4 /dune/imp/obj/mips.mips/dune/imp/FreeBSD/sys/AP96/cam_xpt.o
>> 
>> two of which you might be able to do something about. One suspects that PCIe support could be compiled out of pci.o, and there's two from CAM, and another three from file systems... Maybe there's a smaller subset of CAM that can be compiled in for the USB drive support?
> 
> The AR724x uses PCIe; so I can't kill that.

Bummer. 56k to implement the pci bus seems large and like there should be room to trim.

> CAM is a big one, unfortunately. It'd be nice if we had a smaller
> layer for this but it seems a losing battle without the USB/CAM people
> jumping in and considering it as part of their architecture.

Well, I'm thinking that there's not much of CAM used for the mass storage attached via USB that could be conditionally compiled. But I haven't gone and tried to see where the size bloat comes from and if it is at all trimmable.

>> Last time I fought this battle, it was a battle of attrition: 20k here, 20k there, 5k over there. Sadly, you'll need about 100 of these.  Also, inlining can cause significant bloat, and we inline a lot...
> 
> The big big thing is how big some of the subsystems are. ~ 200k just
> for FFS. ~100k just for uipc routines. mtx is 100k all up and that's
> kind of scary. etc.

Yes. At one point a lot of that was coming form overly aggressive inlining. I haven't tried to trim what gets inlined lately though.

> It'd be nice if we could trim that much code out of different
> subsystems. I'm going to make a concerted effort to shrink down bits
> of the wireless stack and ath driver as they're a bit too kitchen sink
> for me. But I first need to fit a normal kernel in the damned thing.

You can always shrink on larger systems that you artificially constrain a bit at a time...

> Sniffle. :-( I really could do with some more help here.

Yea, wish I had more time to actually focus on this...

Warner