From nobody Mon Jan 23 22:12:16 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4P148d2czjz2t14y; Mon, 23 Jan 2023 22:12:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4P148d1P9tz3MLW; Mon, 23 Jan 2023 22:12:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674511937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Hr4ctser7LBBNSahpZesnlm5sqtrt8bW/92rs4IHNZg=; b=wY6r8NLI8niY7LPk1VaE0zY7KsKLHYf5+SsHAWhK/4vJza7sWlfVcIrKyvH5t+f1i85qux R5kwTeB5Gv5ZSwwOex276o4/2vcMwTL22708pTGW4I+RhK8/4VKvDf/XcnSuulBNaLFZ1c //KjoRj9611HgxFkXRGaXmCl12vBO6hni1CPq10XKr7G8KbQpKJ9uNvT8ijwDfs6/7c3SB jIh4K5LK21fBLnHhLSiPRXsJJiB00Sa64xRwAUakctRQenr9fvTEouc0WFJCM9JGdclzsI TcnwYj/Bjipde8j1e63cw7Z75CqSELtjS9iq55l/L0m2W6VSwGkG4PR5ZWKRlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1674511937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Hr4ctser7LBBNSahpZesnlm5sqtrt8bW/92rs4IHNZg=; b=s4rj3yGJ0f8kMdh0O3av/Jn4VH99HTEJppm/SW0CdNww3Ec6hWJzlLOIRcEE3QrJQggu/Y bl1ZUubF52TtQ1FDCRPvRKFLq+1nXaKzIHHN7CfPN9G9UEuxVdH2+mENwOCsvBi47XG7BF pHhHwIa2MRv23voDsIiHaHcHw58WzN1iCte9ajz6PtM03684JBM+wri/DRfIf6JL+ey23b rlCium/E43Gp1UbuU04fyMwdUy8XpLQUmjEX5uUBTBHR158Uyx8Ii+UJhQCrqcLMlRoucD bo4HQMVMarhIkjz4HA6UYkqnRKf/cR9S6eHJwAHwCHx1AeWI/bSWTH5sGPm6SA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1674511937; a=rsa-sha256; cv=none; b=FcdMDUNcP6T01YuekzpVpCbvg2yifND0q6bqBX7Cd55wh+3T9oJqW01qWSiNfoC+qaKt3T v5YSn4Uo6y92thPTtktZ6K2ICu560bAFe0vEMXb+4oGk/jVxUnG4QtENgo7H7TqCE+NY0R ut7AGudtG1TNbGsdSDoccfBg0bg+RK4Lq/Jaix/sCZJtYgbdFGk24Q2oh7GqxALEG8jA62 BqTuJuOvdHGHOYY6yp3YTYJUMZfwnNzPeM+RLCctZ9kXzmWRW5DEnuqImbgE6nkW+M6spP LW45gk1VqSqEvYrw6OJws2257NYEkPZ6HGoYYU8y+diz7Sn9SyjtzkVmVHXssw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4P148d088szlrJ; Mon, 23 Jan 2023 22:12:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 30NMCGoS017227; Mon, 23 Jan 2023 22:12:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 30NMCGXa017226; Mon, 23 Jan 2023 22:12:16 GMT (envelope-from git) Date: Mon, 23 Jan 2023 22:12:16 GMT Message-Id: <202301232212.30NMCGXa017226@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: "Alexander V. Chernikov" Subject: git: 176e0427b208 - stable/13 - testing: add python test examples List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: melifaro X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 176e0427b208a326ef8256c53c79edeea43e13dc Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by melifaro: URL: https://cgit.FreeBSD.org/src/commit/?id=176e0427b208a326ef8256c53c79edeea43e13dc commit 176e0427b208a326ef8256c53c79edeea43e13dc Author: Alexander V. Chernikov AuthorDate: 2023-01-01 14:35:41 +0000 Commit: Alexander V. Chernikov CommitDate: 2023-01-23 22:09:04 +0000 testing: add python test examples Simplify the adoption of python tests by proving some examples, utilising commonly-used patterns. Differential Revision: https://reviews.freebsd.org/D37902 Reviewed by: asomers MFC after: 2 weeks (cherry picked from commit 8161b823d77f9d89ffabd47444a83d693f74c515) --- etc/mtree/BSD.tests.dist | 2 + tests/Makefile | 1 + tests/examples/Makefile | 10 ++ tests/examples/test_examples.py | 198 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 211 insertions(+) diff --git a/etc/mtree/BSD.tests.dist b/etc/mtree/BSD.tests.dist index 0dabe96110d8..136877db00d6 100644 --- a/etc/mtree/BSD.tests.dist +++ b/etc/mtree/BSD.tests.dist @@ -263,6 +263,8 @@ rc.d .. .. + examples + .. games .. gnu diff --git a/tests/Makefile b/tests/Makefile index b406b8dc6c17..47fc9488f772 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -9,6 +9,7 @@ ${PACKAGE}FILES+= README __init__.py conftest.py KYUAFILE= yes SUBDIR+= etc +SUBDIR+= examples SUBDIR+= sys SUBDIR+= atf_python diff --git a/tests/examples/Makefile b/tests/examples/Makefile new file mode 100644 index 000000000000..7a5d84a98dfe --- /dev/null +++ b/tests/examples/Makefile @@ -0,0 +1,10 @@ +# $FreeBSD$ + +PACKAGE= tests + +TESTSDIR= ${TESTSBASE}/examples + +ATF_TESTS_PYTEST += test_examples.py + +.include + diff --git a/tests/examples/test_examples.py b/tests/examples/test_examples.py new file mode 100644 index 000000000000..13fdcc420f0e --- /dev/null +++ b/tests/examples/test_examples.py @@ -0,0 +1,198 @@ +import pytest +from atf_python.utils import BaseTest +from atf_python.sys.net.tools import ToolsHelper +from atf_python.sys.net.vnet import SingleVnetTestTemplate +from atf_python.sys.net.vnet import VnetTestTemplate +from atf_python.sys.net.vnet import VnetInstance + +import errno +import socket +import subprocess +import json + +from typing import List + + +# Test classes should be inherited +# from the BaseTest + + +class TestExampleSimplest(BaseTest): + @pytest.mark.skip(reason="comment me to run the test") + def test_one(self): + assert ToolsHelper.get_output("uname -s").strip() == "FreeBSD" + + +class TestExampleSimple(BaseTest): + # List of required kernel modules (kldstat -v) + # that needs to be present for the tests to run + REQUIRED_MODULES = ["null"] + + @pytest.mark.skip(reason="comment me to run the test") + def test_one(self): + """Optional test description + This and the following lines are not propagated + to the ATF test description. + """ + pass + + @pytest.mark.skip(reason="comment me to run the test") + # List of all requirements supported by an atf runner + # See atf-test-case(4) for the detailed description + @pytest.mark.require_user("root") + @pytest.mark.require_arch(["amd64", "i386"]) + @pytest.mark.require_files(["/path/file1", "/path/file2"]) + @pytest.mark.require_machine(["amd64", "i386"]) + @pytest.mark.require_memory("200M") + @pytest.mark.require_progs(["prog1", "prog2"]) + @pytest.mark.timeout(300) + def test_two(self): + pass + + @pytest.mark.skip(reason="comment me to run the test") + @pytest.mark.require_user("unprivileged") + def test_syscall_failure(self): + s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) + with pytest.raises(OSError) as exc_info: + s.bind(("::1", 42)) + assert exc_info.value.errno == errno.EACCES + + @pytest.mark.skip(reason="comment me to run the test") + @pytest.mark.parametrize( + "family_tuple", + [ + pytest.param([socket.AF_INET, None], id="AF_INET"), + pytest.param([socket.AF_INET6, None], id="AF_INET6"), + pytest.param([39, errno.EAFNOSUPPORT], id="FAMILY_39"), + ], + ) + def test_parametrize(self, family_tuple): + family, error = family_tuple + try: + s = socket.socket(family, socket.SOCK_STREAM) + s.close() + except OSError as e: + if error is None or error != e.errno: + raise + + # @pytest.mark.skip(reason="comment me to run the test") + def test_with_cleanup(self): + print("TEST BODY") + + def cleanup_test_with_cleanup(self, test_id): + print("CLEANUP HANDLER") + + +class TestVnetSimple(SingleVnetTestTemplate): + """ + SingleVnetTestTemplate creates a topology with a single + vnet and a single epair between this vnet and the host system. + Additionally, lo0 interface is created inside the vnet. + + Both vnets and interfaces are aliased as vnetX and ifY. + They can be accessed via maps: + vnet: VnetInstance = self.vnet_map["vnet1"] + iface: VnetInterface = vnet.iface_alias_map["if1"] + + All prefixes from IPV4_PREFIXES and IPV6_PREFIXES are + assigned to the single epair interface inside the jail. + + One can rely on the fact that there are no IPv6 prefixes + in the tentative state when the test method is called. + """ + + IPV6_PREFIXES: List[str] = ["2001:db8::1/64"] + IPV4_PREFIXES: List[str] = ["192.0.2.1/24"] + + def setup_method(self, method): + """ + Optional pre-setup for all of the tests inside the class + """ + # Code to run before vnet setup + # + super().setup_method(method) + # + # Code to run after vnet setup + # Executed inside the vnet + + @pytest.mark.skip(reason="comment me to run the test") + @pytest.mark.require_user("root") + def test_ping(self): + assert subprocess.run("ping -c1 192.0.2.1".split()).returncode == 0 + assert subprocess.run("ping -c1 2001:db8::1".split()).returncode == 0 + + @pytest.mark.skip(reason="comment me to run the test") + def test_topology(self): + vnet = self.vnet_map["vnet1"] + iface = vnet.iface_alias_map["if1"] + print("Iface {} inside vnet {}".format(iface.name, vnet.name)) + + +class TestVnetDual1(VnetTestTemplate): + """ + VnetTestTemplate creates topology described in the self.TOPOLOGY + + Each vnet (except vnet1) can have a handler function, named + vnetX_handler. This function will be run in a separate process + inside vnetX jail. The framework automatically creates a pipe + to allow communication between the main test and the vnet handler. + + This topology contains 2 VNETs connected with 2 epairs: + + [ VNET1 ] [ VNET2 ] + if1(epair) 2001:db8:a::1/64 <-> 2001:db8:a::2/64 if1(epair) + if2(epair) 2001:db8:b::1/64 <-> 2001:db8:b::2/64 if2(epair) + lo0 lo0 + + """ + + TOPOLOGY = { + "vnet1": {"ifaces": ["if1", "if2"]}, + "vnet2": {"ifaces": ["if1", "if2"]}, + "if1": {"prefixes6": [("2001:db8:a::1/64", "2001:db8:a::2/64")]}, + "if2": {"prefixes6": [("2001:db8:b::1/64", "2001:db8:b::2/64")]}, + } + + def _get_iface_stat(self, os_ifname: str): + out = ToolsHelper.get_output( + "{} -I {} --libxo json".format(ToolsHelper.NETSTAT_PATH, os_ifname) + ) + js = json.loads(out) + return js["statistics"]["interface"][0] + + def vnet2_handler(self, vnet: VnetInstance): + """ + Test handler that runs in the vnet2 as a separate process. + + This handler receives an interface name, fetches received/sent packets + and returns this data back to the parent process. + """ + while True: + # receives 'ifX' with an infinite timeout + iface_alias = self.wait_object(vnet.pipe, None) + # Translates topology interface name to the actual OS-assigned name + os_ifname = vnet.iface_alias_map[iface_alias].name + self.send_object(vnet.pipe, self._get_iface_stat(os_ifname)) + + @pytest.mark.skip(reason="comment me to run the test") + @pytest.mark.require_user("root") + def test_ifstat(self): + """Checks that RX interface packets are properly accounted for""" + second_vnet = self.vnet_map["vnet2"] + pipe = second_vnet.pipe + + # Ping neighbor IP on if1 and verify that the counter was incremented + self.send_object(pipe, "if1") + old_stat = self.wait_object(pipe) + assert subprocess.run("ping -c5 2001:db8:a::2".split()).returncode == 0 + self.send_object(pipe, "if1") + new_stat = self.wait_object(pipe) + assert new_stat["received-packets"] - old_stat["received-packets"] >= 5 + + # Ping neighbor IP on if2 and verify that the counter was incremented + self.send_object(pipe, "if2") + old_stat = self.wait_object(pipe) + assert subprocess.run("ping -c5 2001:db8:b::2".split()).returncode == 0 + self.send_object(pipe, "if2") + new_stat = self.wait_object(pipe) + assert new_stat["received-packets"] - old_stat["received-packets"] >= 5