From nobody Fri Feb 21 13:06:16 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Yzr4C6fD7z5pnFL for ; Fri, 21 Feb 2025 13:06:35 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Yzr4C53mdz41Yg for ; Fri, 21 Feb 2025 13:06:35 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5dee07e51aaso3912318a12.3 for ; Fri, 21 Feb 2025 05:06:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740143194; x=1740747994; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/LTrwOy2tss8lRBDQonGnNAB9BXR7L3UUfVz/Mnpr6Q=; b=lFEthRLx17y1s/A5//XgqsBNczHbc3G+vjC0P4gh/x7kFEUfQBo4+80fWc72MWfILf zoFhDEeQp2hEthnq6S/dVOtV4Ak753Mtla9ULcKOmhXImvXW2RACYPUso1Ru8Ptzd1OP C74RU2HVaHT++Bna8oFl9W0+DG6xBAfDjD9VGAMbwne/ZT2vOWToVKVoH4DKTqjLfMOl 4yKTbucqg9mboThwXkenDegjXeF4XoiBGXPHo2b568pIRdjcLMNdSC0aYKeRkLlH6KGj oTCMvKZCCP9P7UPxJXdQFdKcfj4FavpjacuDw/e8NL0UXXiQAbrPbIyLmlw1oiMX2PPK ntfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740143194; x=1740747994; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/LTrwOy2tss8lRBDQonGnNAB9BXR7L3UUfVz/Mnpr6Q=; b=aA74oCaMAv5KYsHKNxn2HrO5v6NioVNynAvd+LLEmiSroBcI8X1cX7EoHtiOrrublO nWCh/orx23SgN51Zwa9G0ExMiOGhlA0mhWgTzPV2Ehmytf0GU8Mp8WhLGrGdWdPFDj21 qpyzmdQPZkBg8u3toNgpm6ZTiWJqJJHwzkXbiihWU/LmmAEsQpFOuNfdoop6MSWaf+u3 yHqzxO1pC1HVP1InoMfmU7MCOcAZv+VYkb40uAjR8bL45rFeKuueZnmIRBrUdwZSzGIF BH9GBntSAGM1FLucaqH/vDYMTp5KKaOhkAZKNhguNcNliP39ZqicHVSFXOIt+pPVRpKS 5wPg== X-Gm-Message-State: AOJu0YwPiGSjA0nu5k1zFpJPm4bK4O1cWLUuhhj9DR3WchX+oQ2ftAQ+ NdwerhpsTiIh7FCnp0T1W8PG63VN/0cEv3RfLTgXuk7T4BdhLKAek+NK7RxShOBSIIxovnBXKCA p7/RjmzkRIq3tOEnJjXm1eC1yhw== X-Gm-Gg: ASbGncv4fMPpS2cbNGpbzmpcDChBzXj3bLkN9jmvTNrOisIXRfDR9ORrmuEs0nbcPm3 P97kdSR80xknINRsdWUQS1XoqxgmM4SSYueD3ix2zfKL3wid1cTMU6wUOrV1gl+OdFT3ETrmf7J E07O4YE9SLTiDInquu6fluQDK5kO73O4lnNNYnqtnk X-Google-Smtp-Source: AGHT+IEBbpz1dnONbAB9b45PwSps7jwhdiW3dk/JKCriD45ox8Y6I5AubuWSSBQknIPe96YxQqjI6a7jWUPREIS4eFw= X-Received: by 2002:a05:6402:c46:b0:5e0:8920:c4c5 with SMTP id 4fb4d7f45d1cf-5e0b70dac7emr2710400a12.11.1740143193533; Fri, 21 Feb 2025 05:06:33 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <862576B0-EFBF-4CC9-B99A-723125D60983@me.com> In-Reply-To: <862576B0-EFBF-4CC9-B99A-723125D60983@me.com> From: Rick Macklem Date: Fri, 21 Feb 2025 05:06:16 -0800 X-Gm-Features: AWEUYZmfMG1h8QYJGIIqPSnRlErosXuwF9R5AtF-mzHPC5H_9FnxUpCYQs-mFVI Message-ID: Subject: Re: RFC: mount_nfs failure due to dns not running yet To: Toomas Soome Cc: FreeBSD CURRENT , Steve Rikli , Gleb Smirnoff Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4Yzr4C53mdz41Yg X-Spamd-Bar: ---- On Fri, Feb 21, 2025 at 12:13=E2=80=AFAM Toomas Soome wrote= : > > > > On 21. Feb 2025, at 04:39, Rick Macklem wrote: > > On Thu, Feb 20, 2025 at 4:28=E2=80=AFPM Steve Rikli wro= te: > > > On Wed, Feb 19, 2025 at 02:40:15PM -0800, Rick Macklem wrote: > > > The subject line basically describes the problem glebius@ > ran into. When doing an NFS mount in /etc/fstab, it failed > since the DNS service was not yet working and, as such, > the DNS lookup of the server fqdn failed, causing the mount > to fail. Note that this behaviour has existed for decades. > > He feels this is a bug and that mount_nfs(8) should retry > getaddrinfo(3) calls until success, instead of failing the > mount when the first attempt fails. > The problem with just retrying getaddrinfo(3) is that it > could retry forever for simple failures like a typo in the > server fqdn. > I can see several ways this can be handled and would > like feedback from others w.r.t. these alternatives. > > 1) Simply document this case and encourage use of > host names in /etc/hosts for NFS servers along with > specifying use of file before dns in nsswitch.conf. > Doing this results in the mounts working whether or > not DNS is working. > > 2) Call it a bug and patch mount_nfs(8) to retry getaddrinfo(3) > until it succeeds. (I feel this would be a POLA violation, > given that the current behaviour has existed for decades > and for simple cases where the fqdn will never resolve > the behaviour would be to hang at the mount attempt > during boot unless "bg" is specified for the /etc/fstab entry.) > > 3) Add a new NFS mount option "retrydns=3D", which would enable > retries of getaddrinfo(3). This would avoid any POLA violation and > would allow for a convenient way to document the behaviour in > "man mount_nfs". > > 4) ??? > > So, what do you think is the preferred change? > > > I don't think I would change mount_nfs code behavior for this. > > That is, requiring services and daemons etc. to workaround missing, > misconfigured, slow, or misbehaving nameservice (whether it's DNS, > /etc/hosts, NIS, whatever) seems like more complexity, possibly not > effective, and maybe not focused on the right thing. > > Now, without meaning to be presumptuous, it may be worth re-examining > the startup sequence, e.g. to make sure NFS mounts are tried after the > known dependencies can reasonably be expected to have started, including > the network, plus local_unbound or bind (if used), possibly others. > > After a quick look, I don't see an obvious problem with the sequence, > but more knowledgeable eyes than mine are welcome. I don't quite follow > some of the output from rcorder and service -r. > > ps: I looked and the return value from getaddrinfo(3) does not > appear to be useful to discern the case of "DNS service not > running yet". (I think it replies EAI_FAIL for this case.) > > > In that area, I'll note FreeBSD rc.d has a "NETWORKING" dependency for > PROVIDE and REQUIRE, and it's included in scripts like nfsclient, > mountcritremote et al. However there seems to be no similar dependency > for something like "NAMESERVICE" (generic, as opposed to "named" > specifically), and I'm not sure how that might be implemented, even > assuming it could be useful in a situation like this. > > I.e. there are many things to potentially check for "can the system > resolve hostnames yet", and not all of them involve running a local > instance of named, unbound, etc. > > In general, if I were running into problems with nameservice not being > available by the time NFS mounts happen, I think I'd start by looking > into possible nameservice issues, then check out some mechanisms other > folks have mentioned (fstab IP addresses or late option, rc.conf > netwait_enable, etc.) rather than coding workarounds into NFS itself. > > Well, the patch I have created (it took about 15min) only changes behavio= ur > if a new "retrydns" option i used. As such, I think it might be useful fo= r some, > but doesn't change things unless someone uses it. > > I agree with you that I don't think the rc scripts have a way to check RE= QUIRE > dns working. (I, personally, always put the fqdn for NFS servers in /etc/= hosts > and make sure "files" is first in nsswitch.conf, but others argue that is= not > feasible for some deployments. (Using IP numbers works for AUTH_SYS, > but not Kerberized mounts.) > > Note that there is already "retrycnt", which specifies retry the mount, > but that retry loop doesn't include getaddrinfo(3) calls. > --> Personally, I do not like always doing retries since I often > type mount commands manually and I'm a terrible typist, so I > often mistype the server's name. > > This reply was mostly a followup on all the good comments and > not just yours. > > Thanks everyone, for your comments, rick > > > my 2cents: > > there is a difference of name service not responding and name not resolvi= ng. Agreed. Unfortunatey, the return values for getaddrinfo(3) do not clearly differentiate between them. I think Gleb's case returns EAI_FAIL, which is = also returned for other failures. I think EAI_NONAME is returned for the case wh= ere the resolver (dns, /etc/hosts or ???) does determine that the name is bogus= . I suppose the code could do retries for all return values other than EAI_NO= NAME, but to me that would still be a POLA violation, since the current behaviour has been in place for decades (as others have noted). Also, some of the feedback here has been "It is not broken, don't fix it", if I interpreted it correctly. A new option avoids changing the default behaviour. > In first case, it will go to: > > bg If an initial attempt to contact the server fails, f= ork > > off a child to keep trying the mount in the backgrou= nd. > > Useful for fstab(5), where the file system mount is = not > > critical to multiuser operation. > > > bgnow Like bg, fork off a child to keep trying the mount i= n the > > background, but do not attempt to mount in the foreg= round > > first. This eliminates a 60+ second timeout when th= e > > server is not responding. Useful for speeding up th= e > > boot process of a client when the server is likely t= o be > > unavailable. This is often the case for interdepend= ent > > servers such as cross-mounted servers (each of two > > servers is an NFS client of the other) and for clust= er > > nodes that must boot before the file servers. > > > in second case, its a failure you can not recover from. The only difference between "bg" and "bgnow" is whether or not the mount gows into background right away or after the first failed attempt. A name resolver failure still terminates the mount attempt, for both of these. "bg" does fix it for Gleb, but I think that is just timing. (Retries are of the actual NFS mount, assuming the NFS server is also booting and has not yet come up. Cross mounts between multiple systems is messy. Gleb also notes a different behaviour when "late" is used. I have not yet investigated this one rick > > > rgds, > > toomas > > > > >