From nobody Tue Feb 22 17:34:48 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B502919EB8D8 for ; Tue, 22 Feb 2022 17:35:04 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K35sN4j3bz3kXk for ; Tue, 22 Feb 2022 17:35:04 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645551304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=nzSkLicbYzHEnsxAfQ3Pxz69pVmFQw9RW8amvygQOPY=; b=U3QZ08I+3IlFp49xitTGP+PM49ZTEeCgqe+5upJWOcGnB0cGUDPBC54Rfi0JugMqWnfsSS aBOg7es4DL1a0tPYJ9n/O4LKQbcehJJ/CugTIgIivBWBlLYkyxe/SWs7XUbpFpfwxV6wSp gzVdbPiPcRXaDivsDCo18Kasx1Zn+rnE/DxvqYy8LeJIpVFbTgXExZa6sIxB/obZeLNI7A K7WUBPRnxgQmzwlbRJsb4kRBbMWDnXlKOu+bnm94w+O6DOMK3/dMUkDtmnM1PiFRUupKlS r4hxAa/AsS9yesdy0Wg91P5l1u+2HdhsUxkMJUFnsvMt8sLWOT2ruQ7fu/fnUw== Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 7D10C22247 for ; Tue, 22 Feb 2022 17:35:04 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qk1-f172.google.com with SMTP id n185so528938qke.5 for ; Tue, 22 Feb 2022 09:35:04 -0800 (PST) X-Gm-Message-State: AOAM530n5u30GaQhQnTWaM78+bb54Y4zoSUBOXJ7nHuS0DyRS1ttD5F9 B8OOaeYmi3Io3aI0IUpe3cwCZ/xHij+Aqp8Lpeo= X-Google-Smtp-Source: ABdhPJyLippYicQzpiRitAS1709HVDLDZcUJzHfc+W03j0h04Klk8nGM3GjMsFDn6ZzI2NPCi56JA0PZfQKuflUB2D8= X-Received: by 2002:a05:620a:240a:b0:648:c490:41c with SMTP id d10-20020a05620a240a00b00648c490041cmr9194066qkn.365.1645551304111; Tue, 22 Feb 2022 09:35:04 -0800 (PST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Kyle Evans Date: Tue, 22 Feb 2022 11:34:48 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: iconv changes To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645551304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=nzSkLicbYzHEnsxAfQ3Pxz69pVmFQw9RW8amvygQOPY=; b=TI4yWdExXBRkmZHOuGjOVjQ0cITPPr/zPh4z8LLHhTCNTbe4xHSUTQPYGKIrtwMsVhj8IU RUX9f1TWkB2hjzcI9opv+9zVmUh3iNTvc1cZ+zI2A+rknzKPPSA8w6Amk+kTcGW6m42Nj9 0/NE6znTbC9DCmV6i7oTo9hjPKDCXG6Nfm/gsnpglD5bNlqImsqHAIpJ5k8aWOgONwVrQ+ yKVoaTQNxp/cvuHrEjtIRtrE7jy5gn45GfNKvYs48XOT0nuI5tOkAIFpM5C1zGDEAwfRRb m/9ODipCCPvjeCoKTgDMMMbX40Dtm8K4obuTWVYeGsHX2znV/A+VoRdW8ZZa9A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1645551304; a=rsa-sha256; cv=none; b=TSh2DDSEgs2reWe8iutzzTMKT57UA1wey9Q52iBsyx/a2Ee7ul7tjw7j5fmE8LdG7c5JK0 J3QqSfkOA5bpZEFNukim5BnY2bsodNhf1NxnY7ZW/eT3QN/EZA25chB/c1pCFbscYXs7zV q8i+Xo902BFNsplkuNveRXh5Khhq1XKjZnjfTsVbdfE3yqLtDazXkb9SKe9gltbaFaItLj G9dJgi5ZB9Cs8wiuUD3CIZzXLNjviGNvKlIvoT/+PHrTDIocqDTDSaHEQAZKsrjw1FBVS3 OMQA+KetdRRMaN1RQ6nwuueaqmcXAcHIgaijLVcGg5YEYVZ09AM5YXXb/ynKiw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N Hello! Results of `git log` were somewhat inconclusive, so I'm casting a bit of a wider net here; I've got four changes just posted to Phabricator for iconv and iconv modules that could use some third party review. These largely pertain to //IGNORE support: - https://reviews.freebsd.org/D34342 ("iconv: only conditionally use ICONV_SET_DISCARD_ILSEQ") - https://reviews.freebsd.org/D34343 ("libc: iconv: push option ignore into citrus_iconv_open()") - https://reviews.freebsd.org/D34344 ("libc: iconv: add mb_cur_min for encoder traits") - https://reviews.freebsd.org/D34345 ("iconv_std: complete the //IGNORE support") The short version is that //IGNORE is only partially implemented at the moment if you consider the behavior it was trying to model: GNU libiconv. With GNU libiconv, you can use it to, e.g., sanitize a UTF-8 string by removing illegal sequences rather than truncating. These reviews aim to fix it so that it works the same for our libiconv, which would previously only ignore EILSEQ that arise as a result of csmapper failures and not those that arise in initial mbrtowc. Thanks, Kyle Evans