From nobody Tue Jul 19 23:53:28 2022 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LnbJ81c6Xz46vG2 for ; Tue, 19 Jul 2022 23:53:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LnbJ76pXzz3TZp for ; Tue, 19 Jul 2022 23:53:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4LnbJ75rkbzdRs for ; Tue, 19 Jul 2022 23:53:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 26JNrR72079405 for ; Tue, 19 Jul 2022 23:53:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 26JNrR6a079404 for bugs@FreeBSD.org; Tue, 19 Jul 2022 23:53:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 265320] diff program runs long when running diff on large files Date: Tue, 19 Jul 2022 23:53:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: shrikanth07@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658274808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=UFWxq9wGJ2vca2xik61rTIaBMwQQ24lzCNLc1mdaw48=; b=ooTLjrkEdR34Cu48MpW5qubE8pIFBSWi0QrTiJxL9uOY4O2k4IUwRDM59HU/qPMQ1AVpe9 aJhptyC0vmM8tD2yuy8zddItj8dTbyIIVrzFrLkR9YW2Jo0pjd9LX3P5N1WvNBtgDqVzL9 e9Af0Vz759b6i0aMwTOkn/KGD7sLB4rGDAbHvLyhwagNvlmuBWAuIe7DZ3mTduXNE0gTXs bxeE4aBvQCsrjpMpXYxzECq/Z+wr9tiKWVWGO9JEKRCfeBRo3a/+Ts/vcoj0QvaE2Zwawe PahG7PPGmhZuG03XAI1PahaBwlxNTF/YQ14qn1pfccg3hrDGTYJGiEHy7MorNA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1658274808; a=rsa-sha256; cv=none; b=idpWEWdllQN1zyojSPSk/JYdq302JhzQjqZIDuD916xqxHdz47xdqoNTYdSXAEHoc6WObG wAHQ45l5gk7avzmXH/mKIiHay6Cnjh6rmf4QVMlkLEg0RqejhGkjcA4PJdUI72UsZ7uie/ VYnK9TQ0LROs4/JVKpzWQq37RKpZ/DwdUUjenZTOd8DE+6teCMN26E5vO64NohaoIgxyMB CgtrncYqnFNQYdI1Lwwmly/mYyaSqrWHj16tTYwSnbwktGTIds+TJmWWrnEErfWuL/3/11 YemnZpt0+IClExD2NSLqAgQUsN1OvD27CeyH5hXVxzOXqt7GOr8nm0RU66Rw/g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265320 Bug ID: 265320 Summary: diff program runs long when running diff on large files Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Many People Priority: --- Component: bin Assignee: bugs@FreeBSD.org Reporter: shrikanth07@gmail.com The issue was found when running diff on a set of files that are nearly identical (1865319 lines / ~76MB in size) except having differences in the first line and line 1865315. # uname -a FreeBSD BSD14vm 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n255938-326a8d3e0= 85: Fri Jun 3 08:30:41 UTC 2022=20=20=20=20 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 The diff program ran for more than 3172.21 seconds (or even more) I stopped monitoring with CTRL-T after some point but here is the capture... root@BSD14vm:# ls -ltrhG total 469632 -rw-r--r-- 1 root wheel 76M Jul 19 11:46 config_edb_pre -rw-r--r-- 1 root wheel 76M Jul 19 11:46 config_edb_post root@BSD14vm:# diff config_edb_p* load: 0.21 cmd: diff 972 [running] 3.60r 3.44u 0.15s 29% 40480k load: 0.21 cmd: diff 972 [running] 4.90r 4.74u 0.15s 39% 40576k load: 1.10 cmd: diff 972 [running] 156.52r 156.31u 0.15s 100% 46564k load: 1.19 cmd: diff 972 [running] 462.20r 461.86u 0.18s 100% 49140k load: 1.19 cmd: diff 972 [running] 462.64r 462.30u 0.18s 100% 49140k load: 1.19 cmd: diff 972 [running] 462.85r 462.51u 0.18s 100% 49140k load: 1.09 cmd: diff 972 [running] 503.74r 503.39u 0.18s 100% 49140k load: 1.09 cmd: diff 972 [running] 503.96r 503.60u 0.18s 100% 49140k load: 1.09 cmd: diff 972 [running] 504.14r 503.78u 0.18s 100% 49140k load: 1.09 cmd: diff 972 [running] 504.30r 503.94u 0.18s 100% 49140k load: 1.07 cmd: diff 972 [running] 766.07r 765.61u 0.21s 98% 51196k load: 1.17 cmd: diff 972 [running] 919.11r 918.61u 0.21s 100% 51196k load: 1.13 cmd: diff 972 [running] 1014.49r 1013.96u 0.22s 100% 53252k load: 1.13 cmd: diff 972 [running] 1014.75r 1014.21u 0.22s 100% 53252k load: 1.13 cmd: diff 972 [running] 1015.05r 1014.52u 0.22s 98% 53252k load: 1.13 cmd: diff 972 [running] 1015.31r 1014.77u 0.22s 100% 53252k load: 1.13 cmd: diff 972 [running] 1015.49r 1014.96u 0.22s 100% 53252k load: 1.04 cmd: diff 972 [running] 1087.73r 1087.17u 0.22s 100% 53252k load: 1.13 cmd: diff 972 [running] 1266.02r 1265.41u 0.22s 98% 53252k load: 1.04 cmd: diff 972 [running] 1468.09r 1467.41u 0.22s 100% 59176k load: 1.02 cmd: diff 972 [running] 1499.33r 1498.63u 0.22s 100% 59176k load: 1.08 cmd: diff 972 [running] 2095.98r 2095.10u 0.23s 100% 61232k load: 1.43 cmd: diff 972 [running] 2146.53r 2145.62u 0.23s 100% 61232k load: 1.39 cmd: diff 972 [running] 2350.22r 2349.22u 0.23s 100% 63288k load: 1.24 cmd: diff 972 [running] 2874.17r 2873.05u 0.25s 100% 67392k load: 1.12 cmd: diff 972 [running] 2932.42r 2931.28u 0.25s 100% 67392k load: 1.12 cmd: diff 972 [running] 2932.67r 2931.52u 0.25s 100% 67392k load: 1.10 cmd: diff 972 [running] 2980.07r 2978.90u 0.26s 98% 85608k load: 1.19 cmd: diff 972 [running] 3173.49r 3172.21u 0.26s 100% 85608k ... 1c1 < --- > 1865315c1865315 < 2022-07-19 10:43:02.391784 --- > 2022-07-18 16:19:44.594290 As you see there is difference on line 1 and line 1865315 for the entire fi= le. If I use 'head' or 'tail' and retain only one of the difference in the file 'diff' is able to complete in less than a second.=20 The below files have only the diff on line 1 #head -n 1865310 config_edb_pre > p1_headn1865310 #head -n 1865310 config_edb_post > p2_headn1865310 # time diff p1_headn1865310 p2_headn1865310 1c1 < --- > 0.83 real 0.74 user 0.09 sys The below files have only the diff on line 1865306 # tail -n 1865310 config_edb_pre > p1_tailn1865310 # tail -n 1865310 config_edb_post > p2_tailn1865310 # time diff p1_tailn1865310 p2_tailn1865310=20 1865306c1865306 < 2022-07-18 16:19:44.594290 --- > 2022-07-19 10:43:02.391784 0.84 real 0.75 user 0.09 sys # which diff /usr/bin/diff # diff --version FreeBSD diff 20220309 # file config_edb_p* config_edb_post: ASCII text config_edb_pre: ASCII text --=20 You are receiving this mail because: You are the assignee for the bug.=