From nobody Fri Dec 09 01:18:57 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NStTK1CvZz4k0Zs; Fri, 9 Dec 2022 01:19:01 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NStTK0GGyz44GQ; Fri, 9 Dec 2022 01:19:01 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670548741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+E8ZcyJ9aijZIqqNz4MeUXiORiGFtV737xbEIbf7OY8=; b=csW7ZW9ZHyQnc7wInvYoQCRvcQnwQnKuOPCamYSXylj5bNdllNsIAKBE9kyyoSJCVVP3D+ aVYoEMj8vkwmNOgvHen0eSnik3c1S5LEYBOGa+ag9LoDo34NBcTD3oH769ZGQzSNHw/qjp Z5U+6nFulIJqntaZrNEVVm0delTZsic/8jpX9wVqoc+Ftx1FfBb68Hybe6eexT2ORUrglu p7M32fHBu7I+DjxgmqVB/T9HYSU3pzFYeMEPI/9NHSzNeUQlGriPMDJwi1YYvn6ybRo98z Njp6rcLC/U7PZHFJYTBV0GV7rlP4qD2B7DDtwzgHlWjH2bjaXZXMuYvSMJ9c2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670548741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+E8ZcyJ9aijZIqqNz4MeUXiORiGFtV737xbEIbf7OY8=; b=W/VamZuQVWfq3an13LOf3L+LCPhnNYsA7cXQYs5MIUnzGRjdC7BmOFMee3lr8uClkOE4ZT kh9m7j3F1OjRlWqi0NlOKZjIjOKYaCfRs8NL+TdAjxVhKw2CtuuFRGRyCDQetgnUjCd8SQ z2i1GLc3ccfk+GwlylInYy0PwLq2Tt7kiRh9NQKfeTl9PGbvxqLOuMqnzRDO3i0AyJsuHk DvY6AonfYtMxFvaK9dJLv52yPZRx3rIEk3ZiH8xba7KFTUPh1RL3QXSvj+0nmJVjqPHhp9 rmQWciplfYeBhJ4KH/CqWxzz9+FPOQBiRT3KeMYqK5GSh6w2y53U+lUPrp7Tzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1670548741; a=rsa-sha256; cv=none; b=qiXPbnBSuTPXF+Vqf8VkAZs2tyZmYgTjLYtFQ7s/4S0wGrNjyzJe51IyjM3j28cyfkIk7c KIssMncNvIw3cBKAhtpnLxR3VqW8gm75fuWb3ipGm/4qdEWbIVmG6hg/iTfTXezoErB8Aa 8ne0K7ecH40ESoW2p6lgpJopQMnQDap4wNxNl96tZUrOTdCg18AU5hjkhpYP0/zPaTHY79 tXxnBB1toUdZgvDK/ozEDbjAlmLleJxTxgNdT7xVGVqwpIqj77ipz3T9DZwfMlETsKIiPS PAvaldjUWFM88fQAnXT4l22AyxlPnJ4XyrO8J9cfD3pD2cyuOxhjWyxrzVaFhQ== Received: from [10.0.1.4] (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NStTJ2cywzG5M; Fri, 9 Dec 2022 01:19:00 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <365ed69d-79c4-4beb-50ef-1da2678fbc57@FreeBSD.org> Date: Thu, 8 Dec 2022 17:18:57 -0800 List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.4.0 Content-Language: en-US To: Eric Joyner Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org References: <202205232354.24NNs69N032987@gitrepo.freebsd.org> From: John Baldwin Subject: Re: git: cdcd52d41e24 - main - irdma: Add RDMA driver for Intel(R) Ethernet Controller E810 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N On 12/8/22 5:00 PM, Eric Joyner wrote: > Yeah, I was able to repro it and get what I think is the error in a > nicer-to-read set of lines: > > Building > /root/build/head/obj/root/build/head/src/amd64.amd64/lib/ofed/libirdma/libirdma.a > > building static irdma library > > Building > /root/build/head/obj/root/build/head/src/amd64.amd64/lib/ofed/libirdma/libirdma.so.1.full > building shared library libirdma.so.1 > > /usr/local/bin/x86_64-unknown-freebsd14.0-ld: duplicate expression `*' in > version information > collect2: error: ld returned 1 exit status > > *** [libirdma.so.1.full] Error code 1 > > > What does that even mean? I don't know anything about building libraries. For libraries with versioned symbols, we use a separate file that lists the versions assigned to exported symbols. You can find several examples in our tree named Symbol.map. There are also several examples you can look at in the ofed tree: > find contrib/ofed/ -name '*.map' contrib/ofed/libmlx5/libmlx5.map contrib/ofed/libibumad/libibumad.map contrib/ofed/libibnetdisc/libibnetdisc.map contrib/ofed/opensm/libvendor/libosmvendor.map contrib/ofed/opensm/complib/libosmcomp.map contrib/ofed/opensm/opensm/libopensm.map contrib/ofed/libibverbs/libibverbs.map contrib/ofed/librdmacm/librdmacm.map contrib/ofed/librdmacm/librspreload.map contrib/ofed/libibcm/libibcm.map contrib/ofed/libibmad/libibmad.map contrib/ofed/libirdma/libirdma.map The difference here is that the maps for other libraries explicitly list the symbols being exported and what versions to export them under, but the map for libirdma just uses the global '*' wildcard in multiple places, which seems wrong. > cat contrib/ofed/libirdma/libirdma.map /* Export symbols should be added below according to Documentation/versioning.md document. */ IRDMA_1.0 { global: *; local: *; }; IRDMA_1.1 { global: *; } IRDMA_1.0; Contrast this with the map for libmlx5: > cat contrib/ofed/libmlx5/libmlx5.map /* Export symbols should be added below according to Documentation/versioning.md document. */ MLX5_1.0 { global: mlx5dv_query_device; mlx5dv_init_obj; local: *; }; MLX5_1.1 { global: mlx5dv_create_cq; } MLX5_1.0; Here the wildcard is only used for the local rule (which I assume is for private symbols not really exported?) Looking at the library linked with lld, lld seems to be assigning all of the symbols to the first namespace (IRDMA_1.0) and ignoring the duplicates. I suspect that's probably an lld bug. A proper map should be explicitly listing which symbols to export (and only exporting those). I can't find a corresponding .map file for Linux anywhere (has this library been upstreamed anywhere else?) The simplest fix might be to just explicitly list the intended public symbols under version 1.0 and delete the clause for version 1.1 for now. I suspect you weren't meaning to export '_init' and '_fini' as public symbols at least. Here is the current list of exported symbols: > readelf -s /usr/lib/libirdma.so | grep -v UND Symbol table (.dynsym) contains 117 entries: Num: Value Size Type Bind Vis Ndx Name 56: 0000000000008440 968 FUNC GLOBAL DEFAULT 14 irdma_uresize_cq@@IRDMA_1.0 (2) 57: 0000000000007c40 59 FUNC GLOBAL DEFAULT 14 irdma_uquery_qp@@IRDMA_1.0 (2) 58: 0000000000008430 10 FUNC GLOBAL DEFAULT 14 irdma_udetach_mcast@@IRDMA_1.0 (2) 59: 000000000000a6c0 35 FUNC GLOBAL DEFAULT 14 irdma_uk_cq_set_resized_cnt@@IRDMA_1.0 (2) 60: 000000000000b110 84 FUNC GLOBAL DEFAULT 14 irdma_get_sqdepth@@IRDMA_1.0 (2) 61: 000000000000b4b0 107 FUNC GLOBAL DEFAULT 14 irdma_uk_clean_cq@@IRDMA_1.0 (2) 62: 00000000000058f0 264 FUNC GLOBAL DEFAULT 14 irdma_ubind_mw@@IRDMA_1.0 (2) 63: 0000000000008a20 102 FUNC GLOBAL DEFAULT 14 irdma_uk_qp_post_wr@@IRDMA_1.0 (2) 64: 000000000000a690 33 FUNC GLOBAL DEFAULT 14 irdma_uk_cq_resize@@IRDMA_1.0 (2) 65: 0000000000008d90 123 FUNC GLOBAL DEFAULT 14 irdma_qp_get_next_recv_wqe@@IRDMA_1.0 (2) 66: 0000000000005680 154 FUNC GLOBAL DEFAULT 14 irdma_ualloc_pd@@IRDMA_1.0 (2) 67: 0000000000006030 80 FUNC GLOBAL DEFAULT 14 irdma_ucreate_cq@@IRDMA_1.0 (2) 68: 0000000000008310 213 FUNC GLOBAL DEFAULT 14 irdma_ucreate_ah@@IRDMA_1.0 (2) 69: 000000000000b0b0 89 FUNC GLOBAL DEFAULT 14 irdma_get_wqe_shift@@IRDMA_1.0 (2) 70: 000000000000b6b0 35 FUNC GLOBAL DEFAULT 14 irdma_fragcnt_to_wqesize_rq@@IRDMA_1.0 (2) 71: 0000000000005fd0 85 FUNC GLOBAL DEFAULT 14 irdma_udealloc_mw@@IRDMA_1.0 (2) 72: 0000000000007390 38 FUNC GLOBAL DEFAULT 14 irdma_munmap@@IRDMA_1.0 (2) 73: 000000000000b1c0 658 FUNC GLOBAL DEFAULT 14 irdma_uk_qp_init@@IRDMA_1.0 (2) 74: 0000000000008b80 513 FUNC GLOBAL DEFAULT 14 irdma_qp_get_next_send_wqe@@IRDMA_1.0 (2) 75: 0000000000005810 36 FUNC GLOBAL DEFAULT 14 irdma_udereg_mr@@IRDMA_1.0 (2) 76: 0000000000009e10 657 FUNC GLOBAL DEFAULT 14 irdma_uk_inline_send@@IRDMA_1.0 (2) 77: 000000000000b460 79 FUNC GLOBAL DEFAULT 14 irdma_uk_cq_init@@IRDMA_1.0 (2) 78: 0000000000007330 94 FUNC GLOBAL DEFAULT 14 irdma_mmap@@IRDMA_1.0 (2) 79: 000000000000a4c0 453 FUNC GLOBAL DEFAULT 14 irdma_uk_post_receive@@IRDMA_1.0 (2) 80: 00000000000089d0 73 FUNC GLOBAL DEFAULT 14 irdma_clr_wqes@@IRDMA_1.0 (2) 81: 0000000000008420 10 FUNC GLOBAL DEFAULT 14 irdma_uattach_mcast@@IRDMA_1.0 (2) 82: 0000000000009270 35 FUNC GLOBAL DEFAULT 14 irdma_fragcnt_to_quanta_sq@@IRDMA_1.0 (2) 83: 0000000000007230 153 FUNC GLOBAL DEFAULT 14 irdma_uarm_cq@@IRDMA_1.0 (2) 84: 00000000000065f0 770 FUNC GLOBAL DEFAULT 14 irdma_flush_thread@@IRDMA_1.0 (2) 85: 000000000000b170 78 FUNC GLOBAL DEFAULT 14 irdma_get_rqdepth@@IRDMA_1.0 (2) 86: 0000000000007f60 334 FUNC GLOBAL DEFAULT 14 irdma_udestroy_qp@@IRDMA_1.0 (2) 87: 00000000000054d0 227 FUNC GLOBAL DEFAULT 14 irdma_uquery_device_ex@@IRDMA_1.0 (2) 88: 000000000000a6f0 81 FUNC GLOBAL DEFAULT 14 irdma_uk_cq_request_notification@@IRDMA_1.0 (2) 89: 000000000000b95c 0 FUNC GLOBAL DEFAULT 16 _fini@@IRDMA_1.0 (2) 90: 0000000000006a70 78 FUNC GLOBAL DEFAULT 14 irdma_upoll_cq@@IRDMA_1.0 (2) 91: 0000000000007c80 723 FUNC GLOBAL DEFAULT 14 irdma_umodify_qp@@IRDMA_1.0 (2) 92: 0000000000009750 1099 FUNC GLOBAL DEFAULT 14 irdma_uk_send@@IRDMA_1.0 (2) 93: 000000000000e1e0 9 OBJECT GLOBAL DEFAULT 25 libirdma_version@@IRDMA_1.0 (2) 94: 0000000000008e10 1112 FUNC GLOBAL DEFAULT 14 irdma_uk_rdma_write@@IRDMA_1.0 (2) 95: 0000000000009ba0 620 FUNC GLOBAL DEFAULT 14 irdma_uk_inline_rdma_write@@IRDMA_1.0 (2) 96: 000000000000a2b0 524 FUNC GLOBAL DEFAULT 14 irdma_uk_mw_bind@@IRDMA_1.0 (2) 97: 000000000000a750 1808 FUNC GLOBAL DEFAULT 14 irdma_uk_cq_poll_cmpl@@IRDMA_1.0 (2) 98: 0000000000008a90 229 FUNC GLOBAL DEFAULT 14 irdma_qp_push_wqe@@IRDMA_1.0 (2) 99: 000000000000b94c 0 FUNC GLOBAL DEFAULT 15 _init@@IRDMA_1.0 (2) 100: 0000000000005720 36 FUNC GLOBAL DEFAULT 14 irdma_ufree_pd@@IRDMA_1.0 (2) 101: 00000000000055c0 119 FUNC GLOBAL DEFAULT 14 irdma_uquery_device@@IRDMA_1.0 (2) 102: 00000000000072d0 91 FUNC GLOBAL DEFAULT 14 irdma_cq_event@@IRDMA_1.0 (2) 103: 000000000000a0b0 512 FUNC GLOBAL DEFAULT 14 irdma_uk_stag_local_invalidate@@IRDMA_1.0 (2) 104: 0000000000006f10 209 FUNC GLOBAL DEFAULT 14 irdma_ibvcq_ex_fill_priv_funcs@@IRDMA_1.0 (2) 105: 000000000000e4a0 4 OBJECT GLOBAL DEFAULT 27 irdma_dbg@@IRDMA_1.0 (2) 106: 0000000000005640 59 FUNC GLOBAL DEFAULT 14 irdma_uquery_port@@IRDMA_1.0 (2) 107: 0000000000006900 367 FUNC GLOBAL DEFAULT 14 irdma_udestroy_cq@@IRDMA_1.0 (2) 108: 0000000000005a00 1477 FUNC GLOBAL DEFAULT 14 irdma_upost_send@@IRDMA_1.0 (2) 109: 00000000000083f0 36 FUNC GLOBAL DEFAULT 14 irdma_udestroy_ah@@IRDMA_1.0 (2) 110: 0000000000005840 161 FUNC GLOBAL DEFAULT 14 irdma_ualloc_mw@@IRDMA_1.0 (2) 111: 00000000000065c0 40 FUNC GLOBAL DEFAULT 14 irdma_ucreate_cq_ex@@IRDMA_1.0 (2) 112: 0000000000005750 179 FUNC GLOBAL DEFAULT 14 irdma_ureg_mr@@IRDMA_1.0 (2) 113: 00000000000073c0 1179 FUNC GLOBAL DEFAULT 14 irdma_ucreate_qp@@IRDMA_1.0 (2) 114: 00000000000092a0 1189 FUNC GLOBAL DEFAULT 14 irdma_uk_rdma_read@@IRDMA_1.0 (2) 115: 00000000000080b0 604 FUNC GLOBAL DEFAULT 14 irdma_upost_recv@@IRDMA_1.0 (2) 116: 000000000000b520 385 FUNC GLOBAL DEFAULT 14 irdma_nop@@IRDMA_1.0 (2) > > - Eric > > On Wed, Dec 7, 2022 at 1:05 PM John Baldwin wrote: > >> On 6/13/22 2:18 PM, Eric Joyner wrote: >>> On Fri, Jun 10, 2022 at 3:29 PM John Baldwin wrote: >>> >>>> On 5/23/22 4:54 PM, Eric Joyner wrote: >>>>> The branch main has been updated by erj: >>>>> >>>>> URL: >>>> >> https://cgit.FreeBSD.org/src/commit/?id=cdcd52d41e246ba1c0fcfad0769bd691487355ef >>>>> >>>>> commit cdcd52d41e246ba1c0fcfad0769bd691487355ef >>>>> Author: Bartosz Sobczak >>>>> AuthorDate: 2022-05-23 23:39:27 +0000 >>>>> Commit: Eric Joyner >>>>> CommitDate: 2022-05-23 23:52:49 +0000 >>>>> >>>>> irdma: Add RDMA driver for Intel(R) Ethernet Controller E810 >>>>> >>>>> This is an initial commit for RDMA FreeBSD driver for Intel(R) >>>> Ethernet >>>>> Controller E810, called irdma. Supporting both RoCEv2 and iWARP >>>>> protocols in per-PF manner, RoCEv2 being the default. >>>>> >>>>> Testing has been done using krping tool, perftest, ucmatose, >> rping, >>>>> ud_pingpong, rc_pingpong and others. >>>>> >>>>> Signed-off-by: Eric Joyner >>>>> >>>>> Reviewed by: #manpages (pauamma_gundo.com) [documentation] >>>>> MFC after: 1 week >>>>> Relnotes: yes >>>>> Sponsored by: Intel Corporation >>>>> Differential Revision: https://reviews.freebsd.org/D34690 >>>> >>>> I'm trying to get the tree building again with GCC and ld.bfd does not >> like >>>> libirdma.map at all complaining with an error: >>>> >>>> duplicate expression '*' in version information >>>> >>>> I looked at the version map, and it seems rather bogus to me and not >> like >>>> version maps used anywhere else. It seems to be exporting all symbols >> in >>>> two different versions? That would seem to defeat the point of having >>>> symbol versions? >>>> >>>> /* Export symbols should be added below according to >>>> Documentation/versioning.md document. */ >>>> IRDMA_1.0 { >>>> global: *; >>>> local: *; >>>> }; >>>> >>>> IRDMA_1.1 { >>>> global: *; >>>> } IRDMA_1.0; >>>> >>>> (I'm also not sure why lld accepts such a version map.) >>>> >>>> -- >>>> John Baldwin >>>> >>> >>> Hi, >>> >>> I don't know how to answer this; I'm going to forward this to the FreeBSD >>> RDMA guy and another one (the first guy might still be out on vacation) >> and >>> see if they can answer this. >> >> FYI, this is now the current breakage for the new GCC 12 job in FreeBSD's >> CI: >> >> https://ci.freebsd.org/job/FreeBSD-main-amd64-gcc12_build/47/console >> >> The error is buried in the log there, but should be obtainable by doing >> 'pkg install amd64-gcc12' and 'make CROSS_TOOLCHAIN=amd64-gcc12 buildworld' >> on main. >> >> -- >> John Baldwin >> >> > -- John Baldwin