From nobody Thu Jan 13 21:38:38 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 00E03196411E; Thu, 13 Jan 2022 21:38:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JZd8t65h1z4Qth; Thu, 13 Jan 2022 21:38:38 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642109918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MHa6UY76oZpLCN3yeEpVtcj1FmbIBycrNou5E2B3Roc=; b=W4IAXX5sFoDWvkGzGXbgiqOanW5pVCvbdETc0w9xd+kcgkpLQ6ANErWcJ6GRjTpukQfVKo aNnDvYM3vLXOdMSDiH181O+iRnAkT03/Z6TDWtZFiLFFcDCqCxfj6NeNvVX81/WPb4H7QL jxf8YsHSXg9p0M99q9EpEC6PKMPL7p06TM7FovFT0tAUUqeVf0J4GnlC8olozydT1mtj5A XsCvWxPWnQwRMggIZilmWozO6oRLq9rbTSK46dwYsXOZy3xNInGb94dBjXMYOG78CMnTXf T0ZSLWcCkrntwCl8fEdDQpjDqmDRsysRM2tZokaNfoU89KZQimlG6e8aAG66NA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B0EEA27D9F; Thu, 13 Jan 2022 21:38:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20DLccgm044308; Thu, 13 Jan 2022 21:38:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20DLccSU044307; Thu, 13 Jan 2022 21:38:38 GMT (envelope-from git) Date: Thu, 13 Jan 2022 21:38:38 GMT Message-Id: <202201132138.20DLccSU044307@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: 82bfeeff10da - main - tools/portconflicts/port_conflicts_check.lua: detect port conflicts List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 82bfeeff10da860b1ed9f03a01f3282b42b991be Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642109918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MHa6UY76oZpLCN3yeEpVtcj1FmbIBycrNou5E2B3Roc=; b=iHiF5PxIArKVP86FvkTbX89b3VNWhENEj2+bMjcq4zRHcVQETnYfI11HMMXYH54vF0EdlV PZpekzmYkaPWvj5SCx3u4odDboRqrw/VRhYi4xwmghVlpKZZ6FQQEYk3VN3cyeTta965Ab 0AdlKrrao1JdCK3TgLnq97/xfGO67JHIJ7lw+vs3MD8PZhyXQmb86qnI+l6tRg4MXWJy9F iL4Ewc1lvyXIaOSsAYlyUP05EP0VGFfI2U2ZXPwt8VSbbbeQ1gdQTNg3DVeduW1WH5V7TX e+XGvTw7Tb0BHhKFLtd+8JTlTHReke4PndQESMd/oxTlt/RoJ5DL10FMNBMdfQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642109918; a=rsa-sha256; cv=none; b=geipOcpyVfbNnK3Aeev/E6Xuwt3v9Vm3N+fgD3SYj8x6GAr21Q0i1Zze1uHMHAqFD6USOX 54eGyzY4ocXEEuoD/l1+OtsR2pJNTD/2JMu1Y2xu8cdl2wPyDP4dcclxjgJgTDhAb+mnya I77x38MXbjwcUeXGpiBpKYezUD92ZGi1QRUOUTtWfkVda8hNFSYCDVhuSVGOm5l1ffWXlO AjpIq7IgKDFwfug9S7cUdBJ71jqE6t8FkXqUlpV3AmqIyR3DKj4otj09BT4yZjyR7IteRA RDqhdhS3U4fJ+1HmdguAkz88T+FKSK5Or1Js7wNBwsjd8I/pWMHCBzknTBKpxQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=82bfeeff10da860b1ed9f03a01f3282b42b991be commit 82bfeeff10da860b1ed9f03a01f3282b42b991be Author: Stefan Eßer AuthorDate: 2022-01-13 21:36:09 +0000 Commit: Stefan Eßer CommitDate: 2022-01-13 21:36:09 +0000 tools/portconflicts/port_conflicts_check.lua: detect port conflicts This tool can be used to detect install conflicts (ports/packages that conflict with each other because of identically named files). MFC after: 3 days --- tools/tools/portconflicts/port_conflicts_check.lua | 184 +++++++++++++++++++++ 1 file changed, 184 insertions(+) diff --git a/tools/tools/portconflicts/port_conflicts_check.lua b/tools/tools/portconflicts/port_conflicts_check.lua new file mode 100755 index 000000000000..ced7cc619f4d --- /dev/null +++ b/tools/tools/portconflicts/port_conflicts_check.lua @@ -0,0 +1,184 @@ +#!/usr/libexec/flua + +--[[ +SPDX-License-Identifier: BSD-2-Clause-FreeBSD + +Copyright (c) 2022 Stefan Esser + +Generate a list of existing and required CONFLICTS_INSTALL lines +for all ports (limited to ports for which official packages are +provided). + +This script depends on the ports-mgmt/pkg-provides port for the list +of files installed by all pre-built packages for the architecture +the script is run on. + +The script generates a list of ports by running "pkg provides ." and +a mapping from package base name to origin via "pkg rquery '%n %o'". + +The existing CONFLICTS and CONFLICTS_INSTALL definitions are fetched +by "make -C $origin -V CONFLICTS -V CONFLICTS_INSTALL". This list is +only representative for the options configured for each port (i.e. +if non-default options have been selected and registered, these may +lead to a non-default list of conflicts). + +The script detects files used by more than one port, than lists by +origin the existing definition and the list of package base names +that have been detected to cause install conflicts followed by the +list of duplicate files separated by a hash character "#". + +This script uses the "hidden" LUA interpreter in the FreeBSD base +systems and does not need any port except "pkg-provides" to be run. + +The run-time on my system checking the ~32000 packages available +for -CURRENT on amd64 is 150 seconds. +--]] + +require "lfs" + +local index_file = "/usr/ports/INDEX-14" + +local function read_index () + local ORIGIN = {} + + local pipe = io.popen("pkg rquery '%n %o'") + for line in pipe:lines() do + local pkgbase, origin = string.match(line, "(%S+) (%S+)") + ORIGIN[pkgbase] = origin + end + pipe:close() + return ORIGIN +end + +local function read_files() + local FILES_TABLE = {} + + local pkgbase, version + local pipe = io.popen("pkg provides .") + for line in pipe:lines() do + local label = string.sub(line, 1, 10) + if label == "Name : " then + name = string.sub(line, 11) + pkgbase, version = string.match(name, "(.*)-([^-]*)") + elseif label == " " or label == "Filename: " then + local file = string.sub(line, 11) + if file:sub(1, 10) == "usr/local/" then + file = file:sub(11) + else + file = "/" .. file + end + local t = FILES_TABLE[file] or {} + t[#t + 1] = pkgbase + FILES_TABLE[file] = t + end + end + pipe:close() + return FILES_TABLE +end + +local PKG_PAIRS = {} + +for file, pkgbases in pairs(read_files()) do + if #pkgbases > 1 then + for i = 1, #pkgbases -1 do + local pkg_i = pkgbases[i] + for j = i + 1, #pkgbases do + local pkg_j = pkgbases[j] + if pkg_i ~= pkg_j then + p1 = PKG_PAIRS[pkg_i] or {} + p2 = p1[pkg_j] or {} + p2[#p2 + 1] = file + p1[pkg_j] = p2 + PKG_PAIRS[pkg_i] = p1 + end + end + end + end +end + +local CONFLICT_PKGS = {} +local CONFLICT_FILES = {} + +for pkg_i, p1 in pairs(PKG_PAIRS) do + for pkg_j, p2 in pairs(p1) do + CONFLICT_PKGS[pkg_i] = CONFLICT_PKGS[pkg_i] or {} + CONFLICT_PKGS[pkg_j] = CONFLICT_PKGS[pkg_j] or {} + CONFLICT_FILES[pkg_i] = CONFLICT_FILES[pkg_i] or {} + CONFLICT_FILES[pkg_j] = CONFLICT_FILES[pkg_j] or {} + table.insert(CONFLICT_PKGS[pkg_i], pkg_j) + table.insert(CONFLICT_PKGS[pkg_j], pkg_i) + for _, file in ipairs(p2) do + table.insert(CONFLICT_FILES[pkg_i], file) + table.insert(CONFLICT_FILES[pkg_j], file) + end + end +end + +local function table_sorted_keys(t) + result = {} + for k, _ in pairs(t) do + result[#result + 1] = k + end + table.sort(result) + return result +end + +local function table_sort_uniq(t) + local result = {} + local last + + table.sort(t) + for _, entry in ipairs(t) do + if entry ~= last then + last = entry + result[#result + 1] = entry + end + end + return result +end + +local ORIGIN = read_index() + +local RESULT_PATTERN = {} + +for pkg, pkgs in pairs(CONFLICT_PKGS) do + local origin = ORIGIN[pkg] + + if origin then + table.sort(pkgs) + RESULT_PATTERN[origin] = table.concat(pkgs, " ") + end +end + +local FILE_LIST = {} + +for pkg, files in pairs(CONFLICT_FILES) do + local origin = ORIGIN[pkg] + + if origin then + FILE_LIST[origin] = table.concat(table_sort_uniq(files), " ") + end +end + +for _, origin in ipairs(table_sorted_keys(RESULT_PATTERN)) do + local pipe = io.popen("make -C /usr/ports/" .. origin .. " -V CONFLICTS -V CONFLICTS_INSTALL 2>/dev/null") + local conflicts_table = {} + local seen = {} + for line in pipe:lines() do + for word in line:gmatch("(%S*)%s?") do + if word ~= "" and not seen[word] then + table.insert(conflicts_table, word) + seen[word] = true + end + end + end + pipe:close() + table.sort(conflicts_table) + conflicts_string = table.concat(conflicts_table, " ") + local conflicts_new = RESULT_PATTERN[origin] + if conflicts_string ~= conflicts_new then + print("< " .. origin, conflicts_string) + print("> " .. origin, conflicts_new .. " # " .. FILE_LIST[origin]) + print() + end +end