Re: netinet & netpfil tests failing

On 18 Jan 2022, at 3:07, Gleb Smirnoff wrote:
> * Another factor - scapy.  The python scapy library would emit warning to stderr
>   if it sees interface without any IP address.  This happens right at 'import scapy'.
>   The test suite considers a test failed if it has something on stderr, even if
>   it returned success.
>
> So, result is that some test (absolutely unrelated to pcbs) leaves a jail with
> interfaces, then jail is released, interfaced pop up at vnet0, and then some
> other test (absolutely unrelated to pcbs) using scapy writes a warning to stderr
> and triggers failure.
>
Several of the pf scapy scripts deal with that issue by setting the scapy log level:
https://cgit.freebsd.org/src/tree/tests/sys/netpfil/pf/CVE-2019-5597.py#n30

So that part at least we could probably mitigate easily.

(I’m not overly fond of that decision in scapy, but didn’t want to resort to patching scapy to cope with our fairly specific requirements.)

Kristof