From nobody Sun Sep 24 14:03:29 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RtnmG2kS2z4v1jx for ; Sun, 24 Sep 2023 14:03:42 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RtnmF4PhXz4bmj for ; Sun, 24 Sep 2023 14:03:41 +0000 (UTC) (envelope-from ctuffli@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=ctuffli@gmail.com; dmarc=none Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3ae093798c0so3099870b6e.3 for ; Sun, 24 Sep 2023 07:03:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695564220; x=1696169020; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S16MSp7RTmTq6ggci5Y0jpPK4dHhd/s9qUxde3dy1Ho=; b=q23nJUqpWipwBzYsaS2pMjfVm3RCFp145nSP7isipLceOh98go0/EY4P20lQEbu6ep co2o3p+IJmjYp2FTE0GTsC4W+WWfVKA2+uRJyKbA802uJhTUkN59Ttb2155umJuB6Oii 4NOVPsQhyPmcQKo/d8s8UELuGFUZFY9VQPuRDtrML1m+uPp58zCgS2Dl5CK36TaJihEZ e7rAH5JWkmcON9VCwHMuwSsognGR+Z0hQW9cW7I0j5HFDW4lW5JSy/2M9CcMgC0VjSVB xitGAswq1F/AaoQb59UN3pADqOuoSrvlFCtPLfb/TxvZNaVPovvdvNP3gMQh00yLT5q7 k7Ig== X-Gm-Message-State: AOJu0YwYDi7+KMuBp8ANYFp8a3kFAvIh/sNONDV5wCO60r2mSOTaLlsf UFpnVH2knVNBRg3u33si/Wh1AZ/gKXk= X-Google-Smtp-Source: AGHT+IFPERk7pvO8KOwDpgIt9AgJj4XwiQ6woeXYAfWs8NE2h76phAovYfujYVfHXAEqNCqRvuMq/g== X-Received: by 2002:a05:6808:1452:b0:3ae:18c0:dd8e with SMTP id x18-20020a056808145200b003ae18c0dd8emr8288677oiv.27.1695564220331; Sun, 24 Sep 2023 07:03:40 -0700 (PDT) Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com. [209.85.160.182]) by smtp.gmail.com with ESMTPSA id bs24-20020ac86f18000000b0041072408126sm3080946qtb.5.2023.09.24.07.03.40 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 24 Sep 2023 07:03:40 -0700 (PDT) Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-417f872fb94so24125241cf.0 for ; Sun, 24 Sep 2023 07:03:40 -0700 (PDT) X-Received: by 2002:ac8:7248:0:b0:418:1084:1f4 with SMTP id l8-20020ac87248000000b00418108401f4mr2466303qtp.48.1695564219929; Sun, 24 Sep 2023 07:03:39 -0700 (PDT) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Chuck Tuffli Date: Sun, 24 Sep 2023 07:03:29 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: how to setup the host interfacce with sriov and mlx5en(4) driver To: Benoit Chesneau Cc: "freebsd-net@FreeBSD.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: - X-Spamd-Result: default: False [-1.88 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_SPAM_SHORT(0.12)[0.116]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.180:from,209.85.160.182:received]; BLOCKLISTDE_FAIL(0.00)[209.85.167.180:server fail,209.85.160.182:server fail]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.180:from] X-Rspamd-Queue-Id: 4RtnmF4PhXz4bmj On Sat, Sep 23, 2023 at 1:14=E2=80=AFAM Benoit Chesneau wrote: > > Hi all, > > I am trying to understand how works sriov with the mlx5en(4) driver. > > In particularly should Isetup the VF-0 as passthrough false? On intel NI= C i would have to set one of the VF not passthrough for the host. It seems = that sr-iov is behaving differently on mellanox: even if all VFs are set w= ith `passthrough=3Dtrue` I can still configure the interface mce0 on the ho= st. > > Is this expected? Does it mean the VF-0 is usable too? In the past, I set up a system similarly: /etc/rc.conf had the line iovctl_files=3D"/etc/iovctl.d/*.conf" The file /etc/iovctl.d/mlx.0.conf contains: $ head -20 /etc/iovctl.d/mlx.0.conf # Creates 8 virtual functions # Assuming two physical functions and 16 enabled VF's, # VF's will be X:Y.[2-9] PF { device : "mlx5_core0"; num_vfs : 8; } DEFAULT { passthrough : true; } VF-0 { node-guid : 0x706bb09cdcc711ea port-guid : 0x706bb09cdcc711eb } ... The VM's bhyve conf contains $ grep pass /vm/jbof-00/jbof-00.conf passthru0=3D"55/0/2" HTH --chuck