ARM Graviton AWS Processor (AMI Image)

[Marcin Wojtas]

+FreeBSD ENA maintainers

W dniu pon., 11.03.2019 o 00:40 Greg V <greg at unrelenting.technology>
napisał(a):

>
> On Tue, Jan 22, 2019 at 9:39 PM, Greg V <greg at unrelenting.technology>
> wrote:
> > On Mon, Jan 21, 2019 at 1:11 PM, Martin Karrer <martin at bmalum.com>
> > wrote:
> >> My question is if there are any plans yet to support the Graviton
> >> ARM  instances of AWS?
> >>
> >> We have a heavy load on FreeBSD and would also use the ARM
> >> instances.  Are there any other interested parties?
> >
> > I have tried this. It should work very well in theory, e.g. the
> > network card driver (if_ena) compiles with no changes for aarch64,
> > and in fact NetBSD has ported this driver and is up and running on
> > these instances: https://dmesgd.nycbug.org/index.cgi?do=view&id=4623
> >
> > But my result with FreeBSD was: nothing on the console after
> > loader.efi hands control to the kernel.
> > […]
>
> Hello everyone, big update:
>
> FreeBSD/aarch64 on Amazon EC2 a1 (AWS Graviton) instances WORKS!
>
> https://dmesgd.nycbug.org/index.cgi?do=view&id=4813
>
> And you can try it (well, my -CURRENT build, NO WARRANTY etc) right now:
>
> ami-0c2829a0b82a62ca6 in eu-west-1 (Ireland)
>
> -----
>
> So, what I had to do / what should be done / how others can help get
> this into a finished state:
>
> 1. Serial console:
>    - I fixed it: https://reviews.freebsd.org/D19507
>    - (I learned some things about UARTs and their support in FreeBSD,
> should write a blog post about that)
>
> 2. aarch64 build configuration:
>    - if_ena network driver module should be enabled:
> https://reviews.freebsd.org/D18372
>    - NVMe driver should be enabled in the GENERIC kernel config (device
> nvme, device nvd)
>    - BTW, why not also go with hw.nvme.use_nvd="0" by default on
> aarch64, IIRC that was done on powerpc64
>
> 3. VM image build system:
>    - GPT+EFI should be used (amd64 was GPT with no EFI, and aarch64 was
> MBR with EFI (???)): https://reviews.freebsd.org/D18371
>    - bsdec2-image-upload --arm64 flag should be supported: included
> above ^^
>    - ec2.conf: amazon-ssm-agent shouldn't be installed when building
> for aarch64 TARGET, since that's written in Go, and Go isn't ported to
> FreeBSD/aarch64 yet:
>
> https://github.com/myfreeweb/freebsd/commit/5b530ebf7385d8320b9076cf84f50aad01689bc
> (untested patch, I actually used an interactive shell in between the
> image build commands)
>    - qemu-aarch64-static should be used for preinstalling pkgs when
> chrooting into the image: rough version included above ^^
>
> 4. ENA (Elastic Network Adapter) driver:
>    - it works
>    - except there's something funky with interrupt activation, and it
> hits panic("Attempt to double activation of resource id: %u\n", res_id)
> (for the management IRQ) on boot, so I applied the obvious silly
> workaround of "don't panic":
>
> https://github.com/myfreeweb/freebsd/commit/a7e7c6e48cdbdb0fdc6c4e0ba63392262938e62c
>    - but still, it doesn't properly reactivate interrupts (and the box
> becomes unreachable over the net) after going down and up again —
> guess what does that on boot? dhclient applying the big jumbo MTU —
> so I set dhclient.conf to reject MTU changes:
>
> https://github.com/myfreeweb/freebsd/commit/03ec4d417b0b4252285baaf4e294cc6d8c870f7f
>
>
> Would be great if someone familiar with interrupts and stuff could help
> debug the ena driver and make it work without these hacks :)
>
>
> _______________________________________________
> freebsd-arm at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"
>