ARM Graviton AWS Processor (AMI Image)

Thu Mar 21 10:53:57 UTC 2019

Hi Colin,

Fixes pushed:
https://svnweb.freebsd.org/base?view=revision&revision=345371
https://svnweb.freebsd.org/base?view=revision&revision=345373

Best regards,
Marcin

czw., 21 mar 2019 o 06:05 Colin Percival <cperciva at tarsnap.com> napisał(a):
>
> Hi Michał,
>
> Thanks, those two patches fix both problems.  Can you get them into
> the FreeBSD tree or should I commit them?
>
> Colin Percival
>
> On 3/20/19 3:24 AM, Michał Krawczyk wrote:
> > Hi,
> >
> > we have solution for two of the issues with ENA,
> >
> > The double resource activation can be fixed by this patch:
> > https://reviews.freebsd.org/D19655
> >
> > And the issue with reloading module/reconfiguring it by using ifconfig
> > can be fixed by using this patch:
> > https://reviews.freebsd.org/D19641
> >
> > Please check if that fixes it for you.
> >
> > Thanks,
> > Michał
> >
> >
> > śr., 20 mar 2019 o 04:35 Colin Percival <cperciva at tarsnap.com> napisał(a):
> >>
> >> Hi ENA people,
> >>
> >> Any luck figuring out what was going on with the ENA driver here
> >> with interrupt activation?
> >>
> >> Colin Percival
> >>
> >> On 3/11/19 1:35 AM, Marcin Wojtas wrote:
> >>> +FreeBSD ENA maintainers
> >>>
> >>> W dniu pon., 11.03.2019 o 00:40 Greg V <greg at unrelenting.technology>
> >>> napisał(a):
> >>>
> >>>>
> >>>> On Tue, Jan 22, 2019 at 9:39 PM, Greg V <greg at unrelenting.technology>
> >>>> wrote:
> >>>>> On Mon, Jan 21, 2019 at 1:11 PM, Martin Karrer <martin at bmalum.com>
> >>>>> wrote:
> >>>>>> My question is if there are any plans yet to support the Graviton
> >>>>>> ARM  instances of AWS?
> >>>>>>
> >>>>>> We have a heavy load on FreeBSD and would also use the ARM
> >>>>>> instances.  Are there any other interested parties?
> >>>>>
> >>>>> I have tried this. It should work very well in theory, e.g. the
> >>>>> network card driver (if_ena) compiles with no changes for aarch64,
> >>>>> and in fact NetBSD has ported this driver and is up and running on
> >>>>> these instances: https://dmesgd.nycbug.org/index.cgi?do=view&id=4623
> >>>>>
> >>>>> But my result with FreeBSD was: nothing on the console after
> >>>>> loader.efi hands control to the kernel.
> >>>>> […]
> >>>>
> >>>> Hello everyone, big update:
> >>>>
> >>>> FreeBSD/aarch64 on Amazon EC2 a1 (AWS Graviton) instances WORKS!
> >>>>
> >>>> https://dmesgd.nycbug.org/index.cgi?do=view&id=4813
> >>>>
> >>>> And you can try it (well, my -CURRENT build, NO WARRANTY etc) right now:
> >>>>
> >>>> ami-0c2829a0b82a62ca6 in eu-west-1 (Ireland)
> >>>>
> >>>> -----
> >>>>
> >>>> So, what I had to do / what should be done / how others can help get
> >>>> this into a finished state:
> >>>>
> >>>> 1. Serial console:
> >>>>    - I fixed it: https://reviews.freebsd.org/D19507
> >>>>    - (I learned some things about UARTs and their support in FreeBSD,
> >>>> should write a blog post about that)
> >>>>
> >>>> 2. aarch64 build configuration:
> >>>>    - if_ena network driver module should be enabled:
> >>>> https://reviews.freebsd.org/D18372
> >>>>    - NVMe driver should be enabled in the GENERIC kernel config (device
> >>>> nvme, device nvd)
> >>>>    - BTW, why not also go with hw.nvme.use_nvd="0" by default on
> >>>> aarch64, IIRC that was done on powerpc64
> >>>>
> >>>> 3. VM image build system:
> >>>>    - GPT+EFI should be used (amd64 was GPT with no EFI, and aarch64 was
> >>>> MBR with EFI (???)): https://reviews.freebsd.org/D18371
> >>>>    - bsdec2-image-upload --arm64 flag should be supported: included
> >>>> above ^^
> >>>>    - ec2.conf: amazon-ssm-agent shouldn't be installed when building
> >>>> for aarch64 TARGET, since that's written in Go, and Go isn't ported to
> >>>> FreeBSD/aarch64 yet:
> >>>>
> >>>> https://github.com/myfreeweb/freebsd/commit/5b530ebf7385d8320b9076cf84f50aad01689bc
> >>>> (untested patch, I actually used an interactive shell in between the
> >>>> image build commands)
> >>>>    - qemu-aarch64-static should be used for preinstalling pkgs when
> >>>> chrooting into the image: rough version included above ^^
> >>>>
> >>>> 4. ENA (Elastic Network Adapter) driver:
> >>>>    - it works
> >>>>    - except there's something funky with interrupt activation, and it
> >>>> hits panic("Attempt to double activation of resource id: %u\n", res_id)
> >>>> (for the management IRQ) on boot, so I applied the obvious silly
> >>>> workaround of "don't panic":
> >>>>
> >>>> https://github.com/myfreeweb/freebsd/commit/a7e7c6e48cdbdb0fdc6c4e0ba63392262938e62c
> >>>>    - but still, it doesn't properly reactivate interrupts (and the box
> >>>> becomes unreachable over the net) after going down and up again —
> >>>> guess what does that on boot? dhclient applying the big jumbo MTU —
> >>>> so I set dhclient.conf to reject MTU changes:
> >>>>
> >>>> https://github.com/myfreeweb/freebsd/commit/03ec4d417b0b4252285baaf4e294cc6d8c870f7f
> >>>>
> >>>>
> >>>> Would be great if someone familiar with interrupts and stuff could help
> >>>> debug the ena driver and make it work without these hacks :)
> >> --
> >> Colin Percival
> >> Security Officer Emeritus, FreeBSD | The power to serve
> >> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
> >
> >
>
> --
> Colin Percival
> Security Officer Emeritus, FreeBSD | The power to serve
> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
> _______________________________________________
> freebsd-arm at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-arm
> To unsubscribe, send any mail to "freebsd-arm-unsubscribe at freebsd.org"