Problems with realtek NIC

Stefan Esser se at freebsd.org
Sun May 2 08:27:00 UTC 2021 

Am 02.05.21 um 01:37 schrieb Greg Rivers:
> On Saturday, 1 May 2021 16:45:03 CDT Stefan Esser wrote:
>> Am 01.05.21 um 21:48 schrieb Greg Rivers via freebsd-current:
>>> On Saturday, 1 May 2021 14:09:46 CDT Nilton Jose Rizzo wrote:
>>>> I using a FreeBSD 14-Current and get random error with my NIC. The watchdog timer send a timeout message and I loose connection temporaly. In 
logs show only this message:
>>>>
>>> Switch to the official Realtek driver in ports: net/realtek-re-kmod
>>
>> The "official" RealTek driver is based on a very old version of "our"
>> driver that was written by Bill Paul.
>>
>> It lacks many features that have been introduced in FreeBSD in the
>> last decade (or even earlier) like NETMAP-Support.
>>
>> The RealTek-driver has special cases for some 50 variants of RealTek
>> Ethernet chips and contains individual firmware patches for nearly all
>> of them.
>>
>> I had started to merge chip specific changes from the official driver
>> to the FreeBSD driver in the hope to get it to support the RTL8125A/B
>> chips. But I have stopped that project for lack of RTL8125 documentation,
>> especially regarding the PHY, which has its own driver module in our
>> version but not the RealTek code. (And somebody claimed to know that
>> another FreeBSD developer was working on RTL8125 support but did not
>> tell who that might be and whether he had documentation.)
>>
>> Anyway, there are changes regarding the initialization and error recovery
>> of different RealTek chips in the official driver that could be merged
>> into our version. But I do not know whether these changes require the
>> firmware changes provided by the RealTek driver to correctly work.
>>
> Thanks for the information Stefan, and for your work on FreeBSD. My use
> of the term "official" was apparently inaccurate. I was not aware of the
> deficiencies in the RealTek driver. I would prefer to use the FreeBSD
> driver, but I don't for purely pragmatic reasons: the FreeBSD driver
> continually locks up and resets under load (as described by the OP),
> while the RealTek driver does not.

Hi Greg,

the RealTek driver is "official" in the sense that it is provided by the
vendor and written with knowledge about all the (many!) deficiencies of
the RealTek Ethernet chips. And yes, the FreeBSD drived definitely needs
work to fully support all variants of the RealTek chip. I guess that due
to uncovered chip specifics or hardware issues, the FreeBSD driver will
often lack the special code (or firmware patches) and will have to recover
chip operations be going through a hard reset.

If you look at the "official" driver sources, you'll find #ifdefs for
FreeBSD versions  before 4.9, but that is not the reason the main driver
source file is more than 30000 lines long.

The driver distinguishes between more than 70 different chip versions
(identified by MACFG_3 to MACFG_84 with some IDs missing). And each one
has specific requirements regarding firmware patches, initialization and
reset behavior, error handling, ...

I have analyzed these differences (see the attached file) but for lack
of RTL8125 documentation not preceded with this project at this time.
(The column lx_fw identifies firmware patches used by the Linux driver,
while rt_fw identifies those embedded into the RealTek driver for
FreeBSD - and those differ somewhat, and I have no idea why ...).

I have local modifications of the re driver in my sources, but have
one other project that I really want to get ready in the next few
months (after working on it for nearly 2 years) and I do not want to
become responsible for issues of the RealTek driver in base (after
committing fixes that also might cause regressions, if they need to
be accompanied by firmware patches ...)

> FWIW, here are the particulars on the RealTek chip-set that I've got:
> 
> re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xe000-0xe0ff mem 0xb0804000-0xb0804fff,0xb0800000-0xb0803fff irq 16 at device 0.0 on pci1
> re0: Using 1 MSI-X message
> re0: Chip rev. 0x2c800000
> re0: MAC rev. 0x00100000

The Chip rev. indicates that you have got a RTL8168E_VL, which does not
need a firmware patch according to RealTek driver, but gets one in Linux.

It is identified by MACFG_38 in the RealTek driver, BTW, and there are
only a few chip specific code fragments relevant to that chip. It seems,
it needs special handling when the MAC address is programmed, but I did
not spot any other special code for that particular chip in the "official"
driver.

> re0 at pci0:1:0:0: class=0x020000 rev=0x06 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x1458 subdevice=0xe000
>     vendor     = 'Realtek Semiconductor Co., Ltd.'
>     device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
>     class      = network
>     subclass   = ethernet
> 
> Would the FreeBSD Foundation be able to help with getting documentation 
from RealTek?

I have no idea - there is a RTL8125 driver in OpenBSD (which is separate
from the RTL8111/8168 drivers, there) and that driver is actually relatively
short, but it consists just of sequences of register reads and writes without
any comments that could help understand the purpose of these operations.
And there are not so many differences between the RTL8125 and prior chips
(except that the RTL8125 expects 32 bit wide register writes, while the
RTL8111 was limited to 16 bit reads/writes - but this is trivially solved
by having 2 sets of low-level routines without any impact on the higher
level code).

There is also driver code in Linux that can be used to reverse engineer
device specific setup and operation procedures. But that driver is ugly
(IMHO at least) and I'm afraid that there might be legal issues if I use
initialization sequences, firmware patches or other information that can
be found in that driver. Therefore I have only taken a cursory look and
identified the internal chip IDs in the Linux driver and the firmware
patches used for each supported chip.

If there was detailed documentation (detailed register set specification
and operation details like initialization sequence, delays/timeouts that
are required, etc.) this would greatly help getting the RTL8125 into our
version of the "re" driver (with NETMAP and other features that we have
that are not present in the driver from RealTek).

I might try to get support from the author of the OpenBSD driver, but he
might have received documentation under an NDA that does not allow to share
it with us.

Anyway, since the RealTek driver packages works quite well, I can use my
RTL8125 chip (on an AMD B550 mainboard) and it was not that urgent (for me)
to get that chip supported in base.

But if I get documentation, I might be able to integrate the RTL8125 (and
at the same time apply all the special handling and firmware patching the
RealTek driver does - it is still BSD licensed, but in the 4-clause form)
during summer.

Best regards, STefan
-------------- next part --------------
RealTek Version:
#define	RL_FLAG_MSI		        0x00000001
#define RL_FLAG_PHYWAKE_PM              0x00000004
#define RL_FLAG_DESCV2                  0x00000040
#define	RL_FLAG_MSIX	                0x00000800
#define RL_FLAG_MAGIC_PACKET_V2         0x20000000
#define RL_FLAG_PCIE                    0x40000000
#define RL_FLAG_MAGIC_PACKET_V3         0x80000000

FreeBSD Version:
#define	RL_FLAG_MSI			0x00000001	// ?
#define	RL_FLAG_AUTOPAD			0x00000002
#define	RL_FLAG_PHYWAKE_PM		0x00000004	// ?
#define	RL_FLAG_PHYWAKE			0x00000008
#define	RL_FLAG_JUMBOV2			0x00000010
#define	RL_FLAG_PAR			0x00000020	// Ethernet-Address Access Mode
#define	RL_FLAG_DESCV2			0x00000040
#define	RL_FLAG_MACSTAT			0x00000080
#define	RL_FLAG_FASTETHER		0x00000100
#define	RL_FLAG_CMDSTOP			0x00000200
#define	RL_FLAG_MACRESET		0x00000400
#define	RL_FLAG_MSIX			0x00000800	// ?
#define	RL_FLAG_WOLRXENB		0x00001000
#define	RL_FLAG_MACSLEEP		0x00002000
#define	RL_FLAG_WAIT_TXPOLL		0x00004000
#define	RL_FLAG_CMDSTOP_WAIT_TXQ	0x00008000
#define	RL_FLAG_WOL_MANLINK		0x00010000
#define	RL_FLAG_EARLYOFF		0x00020000
#define	RL_FLAG_8168G_PLUS		0x00040000
#define	RL_FLAG_MAGIC_PACKET_V2		0x20000000	// "new" register layout (RTL8139C+ and newer)
#define	RL_FLAG_PCIE			0x40000000	// ?
#define	RL_FLAG_LINK			0x80000000	// ?
#define	RL_FLAG_MAGIC_PACKET_V3		0x80000000	// 8125 only ??? XXX duplicate def

chip		rev	macfg	linux	jumbo	cmd	cfgmode	power	magic	lx_fw	rt_fw	rl_flags
----------------------------------------------------------------------------------------------------------------------------------------
?8169												MACRESET
8169S		0x008	3	2	7	0xff00	cfg1	-	-			MACRESET
8110S		0x040	3	3	7	0xff00	cfg1	-	-			MACRESET
8169_8110SB	0x100	4	4	7	0xff00	cfg1	-	-			MACRESET|PHYWAKE
8169_8110SBL?											MACRESET|PHYWAKE
8169_8110SC	0x180	5	5	7	0xff00	cfg1	-	-			MACRESET|PYHWAKE
8169_8110SCE	0x980	6	6	7	0xff00	cfg1	-	-			MACRESET|PHYWAKE

8100E		0x308	?	14	0	0xe700	cfg1	-	-			FASTETHER|AUTOPAD
8101E		0x340	11	13	0	0xe700	cfg1	-	-			FASTETHER|AUTOPAD
8101E		0x342	12	16	0	0xe700	cfg1	-	-			FASTETHER|AUTOPAD
8101E		0x343	13	10	0	0xe700	cfg1	-	-			FASTETHER|AUTOPAD

?8102E		0x348	?	7	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E		0x349	14	8	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E		0x34A	15	9	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102E		0x34B	16	9	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8103E		0x34C	17	9	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP
8103E		0x34D	18	9	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP
8103E		0x34E	19	9	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|MACSLEEP

?8102EL		0x248	?	7	0	0xe700	cfg2	-	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL		0x249	14	8	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL		0x24A	15	9	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL		0x24B	16	9	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1	0x24C	17	9	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1	0x24D	18	9	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8102EL_SPIN1	0x24E	19	9	0	0xe700	cfg2	pm	-			PHYWAKE|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD

8168B_SPIN1	0x300	21	11	4	0xe700	cfg2	-	-			PYHWAKE|MACSTAT|WOLRXENB
8168B_SPIN2	0x380	22	12	4	0xe700	cfg2	-	-			PYHWAKE|MACSTAT|WOLRXENB
8168B_SPIN3	0x384	23	17	4	0xe700	cfg2	-	-			PYHWAKE|MACSTAT

8168C		0x3C0	24	19	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
8168C		0x3C2	25	20	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
?8168C		0x3C3	?	21	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP?
8168C_SPIN2	0x3C4	26	22	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK|MACSLEEP
?8168CP		0x3C8	?	18	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168CP		0x3C9	27	23	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168CP		0x3CB	28	24	6	0xc700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK

8168D		0x281	31	25	9	0x8700	cfg2	pm	-	8168d_1		PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168D		0x282	32	26	9	0x8700	cfg2	pm	-	8168d_2		PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168D		0x283	33	26	9	0x8700	cfg2	pm	-	8168d_2		PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168DP		0x288	63	27	9	0x8700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP		0x289	64	?	9	0x8700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP		0x28A	65	28	9	0x8700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168DP		0x28B	66	31	9	0x8700	cfg2	pm	-			PYHWAKE|PAR|DESCV2|MACSTAT|AUTOPAD|JUMBOV2|TXPOLL|WOL_MANLINK
8168E		0x2C1	36	32	9	0x8700	cfg2	pm	-	8168e_1		PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK
8168E		0x2C2	37	33	9	0x8700	cfg2	pm	-	8168e_2		PYHWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|WOL_MANLINK

8168E_VL	0x2C8	38	34	9	0x8700	cfg2	pm	mp2	8168e_3		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168E_VL	0x2C9	39	34	9	0x8700	cfg2	pm	mp2	8168e_3		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF

8401E		0x240	41	13	0	0xe700	cfg2	pm	-			PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E		0x409	42	29	0	0xe700	cfg2	pm	-	8105E_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E		0x40A	43	30	0	0xe700	cfg2	pm	-	8105e_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E		0x40B	43	30	0	0xe700	cfg2	pm	-	8105e_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8105E_SPIN1	0x40C	43	30	0	0xe700	cfg2	pm	-	8105e_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD

8168F		0x480	50	35	9	0xbf00	cfg2	pm	mp2	8168f_1		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168F		0x481	51	36	9	0xbf00	cfg2	pm	mp2	8168f_2		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8411		0x488	52	38	9	0xbf00	cfg2	pm	mp2	8411_1		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK
8402		0x440	53	37	0	0xe700	cfg2	pm	mp2	8402_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ

8106E		0x448	54	39	0	0xe700	cfg2	pm	-	8106e_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8106E		0x449	55	39	0	0xe700	cfg2	pm	-	8106e_1		PHYWAKE|PHYWAKE_PM|PAR|DESCV2|MACSTAT|FASTETHER|CMDSTOP|AUTOPAD
8106EUS		?	?	43						8106e_2		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER
8107E		?	?	47						8107e_1		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER
8107E		?	?	48						8107e_2		PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|FASTETHER

8168G		0x4C0	56	40	9	0xcf00	cfg2	pm	mp2	8168g_2	8168g_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168G		0x4C1	57	41	9	0xcf00	cfg2	pm	mp2			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8411B		0x5C8	60	44	9	0xcf00	cfg2	pm	mp2	8411_2	8411b_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP		0x500	61	49	9	0xcf00	cfg2	pm	mp2			PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP		0x501	62	50	9	0xcf00	cfg2	pm	mp2		8168ep_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168EP		0x502	67	51	9	0xcf00	cfg2	pm	mp2		8168ep_2	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|8168G_PLUS
8168GU		0x508	58	?	9	0xcf00	cfg2	pm	mp2		8168gu_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168GU		0x509	59	42	9	0xcf00	cfg2	pm	mp2	8168g_3	8168gu_2	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168H		0x540	68	45	9	0xcf00	cfg2	pm	mp2	8168h_1	6168h_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168H		0x541	69	46	9	0xcf00	cfg2	pm	mp2	8168h_2	6168h_1	PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|CMDSTOP_WAIT_TXQ|8168G_PLUS|JUMBOV2|WOL_MANLINK
8168FP		0x549	70	?	9	0xcf00	cfg2	pm	mp2		6168fp_1/2	?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168FP(8117?)	0x54A	71	52	9	0xcf00	cfg2	pm	mp2	8168fp_3 8168fp_3	?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF
8168FP		0x54B	72		9	0xcf00	cfg2	pm	mp2		8168fp_4	?PYHWAKE|PAR|DESCV2|MACSTAT|CMDSTOP|AUTOPAD|JUMBOV2|CMDSTOP_WAIT_TXQ|WOL_MANLINK|EARLYOFF

8125A		0x608	80	60	9	0xcf00	cfg3	pm	mp3		8125a_1	?DESCV3
8125A		0x609	81	61	9	0xcf00	cfg3	pm	mp3	8125a_3	8125a_2	?DESCV3
8125B		0x640	82		9	0xcf00	cfg3	pm	mp3		8125b_1	?DESCV3
8125B		0x641	83		9	0xcf00	cfg3	pm	mp3		8125b_2	?DESCV3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-current/attachments/20210502/554cfdad/attachment.sig>

More information about the freebsd-current mailing list