From nobody Mon Oct 11 09:51:22 2021 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id ED39717FFF43 for ; Mon, 11 Oct 2021 09:51:34 +0000 (UTC) (envelope-from olivier.delande@gmail.com) Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HSYwQ09SVz3mjd for ; Mon, 11 Oct 2021 09:51:34 +0000 (UTC) (envelope-from olivier.delande@gmail.com) Received: by mail-pj1-x102d.google.com with SMTP id d13-20020a17090ad3cd00b0019e746f7bd4so14349322pjw.0 for ; Mon, 11 Oct 2021 02:51:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=fx7wsvWNJGZAnYfWwfiO/sZYIU5+8HsIAIqxEeLrBKg=; b=HFbTlJ3qiVPXXMElc3gv6t7xipj6txsufdTTkok7Ok3ir/4bLjQYsXiCMpBJrUqhMC mddzl5crgsRu1wkSDdTbZ1tixYy0pPeE0oUOyRMEVbb+HMpltpXZweMu5/r3Esra0OQG bmgN2PAbuR+0PJCXHH+rlybywH6pRBvHR34fA4UfU3QgZzoTffrKOowRExxwtoCyfkql OQlAhREn3nKCGZKplq9rzI9C/TwImZN5B2G7oPWb/GpNAildnBtnyod2SrQ9tuitxnPE 8IFhZIFQDF153+7u+ZiHd9aWRmnEjxq3OTXYuEUm+BXABhqeGOusoCjGG1V8j4tqjkWY nZNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=fx7wsvWNJGZAnYfWwfiO/sZYIU5+8HsIAIqxEeLrBKg=; b=q4aVbYb8RZ2lZc2xw4GT0c/U0jsr4blddvusvfUOkp7cFACMDMHw6z24wm9J8y0tGI H94qEiPgcxSZHFgHsPNg44kSMhA1HRVa0Vlsdh3/st00BADS7Jzaoba5IReD1JkzE1n6 OgL4fglcKQUy4JJf3Icpu1wmJR31jsdY41e9sr/jNzRlAPnrrwqi0t9jkucle2imutDo SHepDNQZK5exx0RpLC+pQrmRwXFlG3y20QXm7dn2vnQRKkFOO/szYwm/FApdK9VOZK7B rLhH6BNbcLXo9l+HfSdz+P7E8xeMgHByYLfg1vASKff7S8OXfB7iXx5qErzM2Nq0RSIz W2nw== X-Gm-Message-State: AOAM531TCrxQMhNvQtWDuTdz5MIvPGKL0myW1ZtsmKSD919x7OnI1zJp JSR95FCM+q2GxAwkgq6fwzAG29WVMqPikqTIMWuo7AZ37ahKrfXJVnc= X-Google-Smtp-Source: ABdhPJziAupdjERuMz1u5vu3rfS9uO8rpQ+D5zmJ3417MtqEFWgqWBdCXLpC55CwiPWijUA0qXbn9fq0cw1rzV0EX1E= X-Received: by 2002:a17:90a:dc81:: with SMTP id j1mr29068033pjv.155.1633945892812; Mon, 11 Oct 2021 02:51:32 -0700 (PDT) List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 From: Olivier Delande Date: Mon, 11 Oct 2021 11:51:22 +0200 Message-ID: Subject: How do I synchronize with the return from a bio_done handler? To: freebsd-geom@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4HSYwQ09SVz3mjd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=HFbTlJ3q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of olivierdelande@gmail.com designates 2607:f8b0:4864:20::102d as permitted sender) smtp.mailfrom=olivierdelande@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-0.995]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-geom@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::102d:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N (Apologies if freebsd-geom is not the most appropriate list to which to post this. This is not strictly about the GEOM framework, but rather about some of its ingredients: struct bio and asynchronous completion callbacks. That is why I chose this list over more generic ones like freebsd-drivers. Please let me know if I should move this elsewhere.) I am writing a kernel module that performs some I/O on a block device. To this end, I create struct bio instances with g_new_bio() which I then submit to the block device's d_strategy method. In my bio_done handler, I handle the result of the operation, then free the struct bio with g_destroy_bio(). This strategy seems to be working. I wonder how to properly shut down my module. I can easily arrange to stop submitting new bios, but how can I then wait until no one is running my bio_done handler any more? Keeping track of the number of in-flight bios (e.g. by incrementing a counter before posting a bio, and decrementing it right before returning from the bio_done handler) is not enough, because there is a race window between the final decrement bringing the counter to 0 and the actual return(s) from the handler. I need to synchronize with something happening after the handler has returned. Unfortunately, since freeing the bio inside the handler as I do seems acceptable (other drivers, like cam/ctl/ctl_backend_block.c, do so as well), it is likely that the bio is never touched after the handler has returned. Is there any other point where something synchronizes with the bio_done handlers? (Perhaps releasing the reference to the block device and its switch table with dev_relthread() does wait for the handlers to finish running, but this is just a wild guess.) Best regards, Olivier