From nobody Mon Oct 23 01:53:32 2023 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SDJBP0HtDz4y7MV; Mon, 23 Oct 2023 01:53:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SDJBN6zZKz4CnQ; Mon, 23 Oct 2023 01:53:32 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698026013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yeZRT+34ixteF6CSHXY5UNM2al1LKgZXxntQyxt4IfU=; b=KiXrEbnnOdFZJFRC8Wa9DY/mkgB07Eq1p/FlMv6WRUbxTy8ZSa1NT3pOZ5Xc9SI0aus1Y0 ceRH6EYG2Z6fLFu+2VzTGo0AcEbYVpENGO8qPXrphf/qWhElbvYkfU6EI6sJ/p9e2vLNeP 4riZhQPfpuOEGQV7yz/wElG2i32Ap8O5DR31rD5v+JgpWHY0Ui/bovZOS7ap1G/+owPaan 0tg41krJ2ZX8xPzyrAUuTJvANuZdIrOby1NG8To20Neo68i9WwvHYr0lTH8N91vftyjVqI EF+yixpgdUFRErTjYL+RzsVS/VqtpJt46qAolMqp+K0ArF5ZLYgiUN/oMyo5NA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1698026013; a=rsa-sha256; cv=none; b=i8CPuNV+jKpMA5rrA+FU1yR/33uf5FzamL3rAhXrg6zOgjLbLBgyegnmAIztTHx6f19ou6 HfjTTKipnR4FjFp1QSSOUyR0BeQYqwY1LKlB0Dx7KDvBZ0vE7hS3h7Xf9MzkTEziN8BcBu ZFcNvlh+mE0VZpI9I1S6vbgYNYiZQKkd02rCrEograe7e+zgOOygdy2zbNXbu0c0EvN+ur OfymLFS18bw/2KiYxFSoSPXGPRwNqZzLhFzfnIjK3fJHsLezZSTKSY7KTK4zmHEjR/dJ72 UvwKTs6m1RqmFAGVlNiWDnLPr3Mvjm5PH0NBrm2Ar5HK7Xu1cPKXDoOTTBaJIA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698026013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yeZRT+34ixteF6CSHXY5UNM2al1LKgZXxntQyxt4IfU=; b=SvNUmS188qPcXpG2/tilwAbqRGLz48owm9Y1lGfROi4CCK2y9KK7LEFDvYMvSF6wckMk9G +IG6UNv893l7N3JPbFpjtVQ4403iDHndYVmrLkfpyDj2PP9A72tbt/8Y+XGDGeNLYR3x1N 0+BXHahP/YVxumoC12lPQDhmVbCuSSe+KY9GyhhOS3JFMZhL3m75qiV/Z/CNUBpP2osTW+ 16OIsoL0VFsmqyY+5JxbvIDlDpPV3Y8Ga48l/xx4egHBbAeXu7kHFw+MCDtrEVBXDTzR3R UoWNLKdINsSYhjso73GJKhQ1xtkhyq9q1WWPjkfYXDzy5SjgQ/0rafOmR6ifpQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SDJBN64R0z6RZ; Mon, 23 Oct 2023 01:53:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 39N1rWb9013000; Mon, 23 Oct 2023 01:53:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 39N1rWNR012997; Mon, 23 Oct 2023 01:53:32 GMT (envelope-from git) Date: Mon, 23 Oct 2023 01:53:32 GMT Message-Id: <202310230153.39N1rWNR012997@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Robert Clausecker Subject: git: 21a02f42b641 - main - converters/simdutf: Unicode validation and transcoding with SIMD List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-all@freebsd.org X-BeenThere: dev-commits-ports-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: fuz X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 21a02f42b64150353438318eae45aefb66d51b4d Auto-Submitted: auto-generated The branch main has been updated by fuz: URL: https://cgit.FreeBSD.org/ports/commit/?id=21a02f42b64150353438318eae45aefb66d51b4d commit 21a02f42b64150353438318eae45aefb66d51b4d Author: Robert Clausecker AuthorDate: 2023-10-20 17:07:54 +0000 Commit: Robert Clausecker CommitDate: 2023-10-23 01:51:27 +0000 converters/simdutf: Unicode validation and transcoding with SIMD This library provide fast Unicode functions such as - ASCII, UTF-8, UTF-16LE/BE and UTF-32 validation, with and without error identification, - transcoding between each of Latin1, UTF-8, UTF-16LE/BE, and UTF-32, with and without validation, with and without error identification - From an UTF-8 string, compute the size of the Latin1/UTF-16/UTF-32 equivalent string, - From an UTF-16LE/BE string, compute the size of the Latin1/UTF-8/UTF-32 equivalent string, - From an UTF-32 string, compute the size of the UTF-8 or UTF-16LE equivalent string, - UTF-8 and UTF-16LE/BE character counting. - UTF-16 endianness change (UTF16-LE/BE to UTF-16-BE/LE) The functions are accelerated using SIMD instructions (e.g., ARM NEON, SSE, AVX, AVX-512, etc.). When your strings contain hundreds of characters, we can often transcode them at speeds exceeding a billion characters per second. You should expect high speeds not only with English strings (ASCII) but also Chinese, Japanese, Arabic, and so forth. We handle the full character range (including, for example, emojis). The library compiles down to a small library of a few hundred kilobytes. Our functions are exception-free and non allocating. We have extensive tests and extensive benchmarks. WWW: https://simdutf.github.io/simdutf/ --- converters/Makefile | 1 + converters/simdutf/Makefile | 38 ++++++++++++++++++++++++++++++++++++++ converters/simdutf/distinfo | 3 +++ converters/simdutf/pkg-descr | 26 ++++++++++++++++++++++++++ converters/simdutf/pkg-plist | 18 ++++++++++++++++++ 5 files changed, 86 insertions(+) diff --git a/converters/Makefile b/converters/Makefile index 3782f6bcdac8..7b40eac0f7ae 100644 --- a/converters/Makefile +++ b/converters/Makefile @@ -168,6 +168,7 @@ SUBDIR += rubygem-url_safe_base64 SUBDIR += rubygem-xdr SUBDIR += showkey + SUBDIR += simdutf SUBDIR += ta2as SUBDIR += tnef SUBDIR += trans diff --git a/converters/simdutf/Makefile b/converters/simdutf/Makefile new file mode 100644 index 000000000000..947a871a07f6 --- /dev/null +++ b/converters/simdutf/Makefile @@ -0,0 +1,38 @@ +PORTNAME= simdutf +DISTVERSIONPREFIX= v +DISTVERSION= 4.0.0 +CATEGORIES= converters textproc + +MAINTAINER= fuz@FreeBSD.org +COMMENT= Unicode validation and transcoding with SIMD +WWW= https://simdutf.github.io/simdutf/ + +LICENSE= APACHE20 MIT +LICENSE_COMB= dual +LICENSE_FILE_APACHE20= ${WRKSRC}/LICENSE-APACHE +LICENSE_FILE_MIT= ${WRKSRC}/LICENSE-MIT + +USES= cmake:testing +USE_GITHUB= yes +USE_LDCONFIG= yes + +CMAKE_ON= BUILD_SHARED_LIBS + +OPTIONS_DEFINE= BENCHMARKS ICONV TEST TOOLS +OPTIONS_DEFAULT= ICONV TOOLS +OPTIONS_SUB= yes +BENCHMARKS_DESC= Build benchmarks +TOOLS_DESC= Build tools + +BENCHMARKS_IMPLIES= TEST +BENCHMARKS_LIB_DEPENDS= libicuuc.so:devel/icu +BENCHMARKS_CMAKE_BOOL= SIMDUTF_BENCHMARKS +ICONV_USES= iconv +ICONV_CMAKE_BOOL= SIMDUTF_ICONV +TEST_CMAKE_BOOL= SIMDUTF_TESTS +TOOLS_CMAKE_BOOL= SIMDUTF_TOOLS + +pre-test-TEST-off: + @echo Enable option TEST to run the full test suite + +.include diff --git a/converters/simdutf/distinfo b/converters/simdutf/distinfo new file mode 100644 index 000000000000..ef9cc66d6552 --- /dev/null +++ b/converters/simdutf/distinfo @@ -0,0 +1,3 @@ +TIMESTAMP = 1697849191 +SHA256 (simdutf-simdutf-v4.0.0_GH0.tar.gz) = 1a84ea8a24396ea410d1c88d3126f95956a8799d8eaea0e03dc721e7c65ff9b3 +SIZE (simdutf-simdutf-v4.0.0_GH0.tar.gz) = 1864807 diff --git a/converters/simdutf/pkg-descr b/converters/simdutf/pkg-descr new file mode 100644 index 000000000000..d8319ef380ee --- /dev/null +++ b/converters/simdutf/pkg-descr @@ -0,0 +1,26 @@ +This library provide fast Unicode functions such as + + - ASCII, UTF-8, UTF-16LE/BE and UTF-32 validation, with and without + error identification, + - transcoding between each of Latin1, UTF-8, UTF-16LE/BE, and UTF-32, + with and without validation, with and without error identification + - From an UTF-8 string, compute the size of the Latin1/UTF-16/UTF-32 + equivalent string, + - From an UTF-16LE/BE string, compute the size of the + Latin1/UTF-8/UTF-32 equivalent string, + - From an UTF-32 string, compute the size of the UTF-8 or UTF-16LE + equivalent string, + - UTF-8 and UTF-16LE/BE character counting. + - UTF-16 endianness change (UTF16-LE/BE to UTF-16-BE/LE) + +The functions are accelerated using SIMD instructions (e.g., ARM NEON, +SSE, AVX, AVX-512, etc.). When your strings contain hundreds of +characters, we can often transcode them at speeds exceeding a billion +characters per second. You should expect high speeds not only with +English strings (ASCII) but also Chinese, Japanese, Arabic, and so +forth. We handle the full character range (including, for example, +emojis). + +The library compiles down to a small library of a few hundred kilobytes. +Our functions are exception-free and non allocating. We have extensive +tests and extensive benchmarks. diff --git a/converters/simdutf/pkg-plist b/converters/simdutf/pkg-plist new file mode 100644 index 000000000000..4c801e507c91 --- /dev/null +++ b/converters/simdutf/pkg-plist @@ -0,0 +1,18 @@ +%%TOOLS%%bin/sutf +include/simdutf.h +include/simdutf/avx512.h +include/simdutf/common_defs.h +include/simdutf/compiler_check.h +include/simdutf/encoding_types.h +include/simdutf/error.h +include/simdutf/implementation.h +include/simdutf/internal/isadetection.h +include/simdutf/portability.h +include/simdutf/simdutf_version.h +lib/cmake/simdutf/simdutf-config-version.cmake +lib/cmake/simdutf/simdutf-config.cmake +lib/cmake/simdutf/simdutfTargets-%%CMAKE_BUILD_TYPE%%.cmake +lib/cmake/simdutf/simdutfTargets.cmake +lib/libsimdutf.so.5.0.0 +lib/libsimdutf.so.5 +lib/libsimdutf.so