From nobody Wed Jun 08 12:34:29 2022 X-Original-To: freebsd-xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3C1C483AF09 for ; Wed, 8 Jun 2022 12:34:46 +0000 (UTC) (envelope-from prvs=151432dd3=roger.pau@citrix.com) Received: from esa6.hc3370-68.iphmx.com (esa6.hc3370-68.iphmx.com [216.71.155.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.hc3370-68.iphmx.com", Issuer "HydrantID Server CA O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJ69x13qXz3klX for ; Wed, 8 Jun 2022 12:34:45 +0000 (UTC) (envelope-from prvs=151432dd3=roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1654691685; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=4q9D//ea4VXUD0gF/Rv8OUUZnKSQXajHa37Gzbrkx0A=; b=CJptygzM3jxoU6u9tDAP1+d87CKjdGnv0eIph/dWsjx0roKXln2qVTwe HVpw4mHJUEfT93jSWujhD5b04qFC8GDnbatXHUM8BlwxFky+su7zQOFSq DHRq97llREEmr/6AVONocMUmTDzjiXe0LdD6cb4e53OsvBTk2qZIX4MBH Y=; X-IronPort-RemoteIP: 104.47.58.102 X-IronPort-MID: 72979601 X-IronPort-Reputation: None X-IronPort-Listener: OutboundMail X-IronPort-SenderGroup: RELAY_O365 X-IronPort-MailFlowPolicy: $RELAYED IronPort-Data: A9a23:s+yTkaB8e3OKJhVW/3Liw5YqxClBgxIJ4kV8jS/XYbTApGkr1DQDz WdLXmqAafyLMGChLYhza4izoxgDvJeHmN42QQY4rX1jcSlH+JHPbTi7wuYcHM8wwunrFh8PA xA2M4GYRCwMZiaA4E/raNANlFEkvU2ybuOU5NXsZ2YgHGeIdA970Ug5w7Bh0tYy6TSEK1jlV e3a8pW31GCNg1aYAkpMg05UgEoy1BhakGpwUm0WPZinjneH/5UmJMt3yZWKB2n5WuFp8tuSH I4v+l0bElTxpH/BAvv9+lryn9ZjrrT6ZWBigVIOM0Sub4QrSoXfHc/XOdJFAXq7hQllkPh+x YtQsLeTcjwPFbLAk8ssChpnMzlXaPguFL/veRBTsOS15mifKz7A5qsrC0s7e4oF5uxwHGdCs +QCLywAZQyCgOTwx6+nTu5rhYIoK8yD0IE34yk8i22GS6t2B8mbGc0m5vcBtNs0rtpJEvvEI dIQdBJkbQjaYg0JMVASYH47tLjy1yauLWUBwL6TjfYe+1OJnB1d6rTSP8P7UYWYTphruG/N8 woq+Ey8WHn2Lue3wyeF6HWzrunBlyjyXMQVDrLQ3vBwjVCZ3WBWGBAQfVWhrPSzkUL4XMhQQ 3H44QIrpKk2sUCuEN/0WkTgpGbe50ZGHd1ND+c99QeBjLLO5BqUDXQFSTgHb8E6sMgxRnoh0 Vrhc87VOAGDeYa9ERq1nop4ZxvoUcTJBQfuvRM5cDY= IronPort-HdrOrdr: A9a23:ZK9ErKOyJyNRlMBcT13155DYdb4zR+YMi2TDiHoddfUFSKalfp 6V98jztSWatN/eYgBEpTmlAtj5fZq8z+8N3WB1B9uftWbd2FdAQLsSjrcKhgeQYBEWldQtqZ uIEZIOb+EYZGIS5aia3OD7KadH/DDuytHUuQ609QYIcegFUdAD0+8vYTzraHGeCTM2c6YRJd 653I5qtjCgcXMYYoCSAWQEZfHKo5numIj9aRALKhY74E3W5AnYoILSIly95FMzQjlPybAt/S zslBH43Lyqt7WexgXH32HewpxKkJ/Ky8dFBuaLls8JQw+cwzqAVcBEYfmvrTo1qOag5BIDl8 TNmQ4pO4BJ53bYbgiO0G7Q8jil9Axrx27pyFeej3emi9f+XigGB81Igp8cWgfF6mI71esMnJ 5j7ia8jd56HBnAlCPy65zjTBdxjHe5pnIkjKo6k2Ffa40Dc7VcxLZvsX+9KK1wUh4S1bpXUd WHVKrnlbZrmBKhHjrkV1BUsZORti9ZJGbEfqAA0vbloQS+0koJjXfw//Zv4EvoxKhNNKWs2N 60Q5iAtIs+OvP+PpgNc9vof6OMexzwaCOJFl6uCnLaM4xCE07xivfMkcYIDaeRCdc18Kc= X-IronPort-AV: E=Sophos;i="5.91,286,1647316800"; d="scan'208";a="72979601" Received: from mail-dm6nam10lp2102.outbound.protection.outlook.com (HELO NAM10-DM6-obe.outbound.protection.outlook.com) ([104.47.58.102]) by ob1.hc3370-68.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 08 Jun 2022 08:34:38 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ESFildMV3v0HOOUnUcGeH+Iu5gfGyTVsOBVejtHfWbhuEk4J+Q99ycAFRx3auqEeI7ji0Ig/usyerjny6d7dQE3WpkJXVN9DeR25h3wtoN5qnlNmA9SdCMYx9ukSwCwlvc9U4+aG7gBs+xaaxReaXASYrR1B3G5CZWrPSyJPZyERO+ax8xMJlgZPlg5z+orNsXjOB3bNEw86yK2UWweuJT4GUhS2EHaZIIPyJgPLsOKFjCqN4BSJlO7Cf86qQ1mCjQhLUXhCVvGlm5r2XQ0tQFTf7AuQQGztQdbbCMIsI1cs/4ezaaAL/QmYfLbvi4PFEjakLTfh0UreSqF93tV11A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iySmQsg0gj/Tui/7mu363lF1r6kfSGBYs+RMFHK5v5U=; b=kUcVIxhan9LiiX1N4UZV3iVMpsBTMv0rJ1ZcWVK7RkWAi9spxdVMN+vpNb2BYitmkhI9ZMHRLBRWrN/LdDYYrbrkfYp6lwA+uMY/Ofamfi3XfcLjwFOAi6WLccsL1BUEkvG0YXng99w+GazZX7rwB72Fic7ziCwywQat6JY7esx3KOXJb4dvDI+uqBW/n7eWlVtqcofJnWL6K/N8IXns1ONmv0zWZX1ERYnTg70g8CgqLtBTu+30x9N2YFfo4dbSafCY/XTzhbpKphU+ibL/UYf0vC5HZU2IpTEJcs7T227aLi7me8+85BodDkVKmPnKCK3oWcliDj7RczINdWJMcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iySmQsg0gj/Tui/7mu363lF1r6kfSGBYs+RMFHK5v5U=; b=d1M+dKFPItHlfrNqWyfTQ4ttKQsewbOSLJrjuSDIONIcDJ/4CvSldAFcOgnyXGZRS3EoDDoGAgJsE07+SqHujd1OD+O9+xy70kfE8Kei7aRen7YZUAG95OJpTg2sb1sj64L+h6TBK90h0E7/VSivQDzoqMUMdcdDISGHwxyDIpc= Received: from DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) by SJ0PR03MB6762.namprd03.prod.outlook.com (2603:10b6:a03:409::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.12; Wed, 8 Jun 2022 12:34:34 +0000 Received: from DS7PR03MB5608.namprd03.prod.outlook.com ([fe80::5df3:95ce:4dfd:134e]) by DS7PR03MB5608.namprd03.prod.outlook.com ([fe80::5df3:95ce:4dfd:134e%4]) with mapi id 15.20.5314.019; Wed, 8 Jun 2022 12:34:34 +0000 Date: Wed, 8 Jun 2022 14:34:29 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Brian Buhrow Cc: freebsd-xen@freebsd.org Subject: Re: Stable/13 doesn't boot with xen Message-ID: References: <202206071919.257JJ1aB027047@nfbcal.org> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <202206071919.257JJ1aB027047@nfbcal.org> X-ClientProxiedBy: LO4P123CA0310.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:197::9) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b630e624-4fcf-4e58-5fb4-08da494b3e02 X-MS-TrafficTypeDiagnostic: SJ0PR03MB6762:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zcPRrJ6JpUehywyzxQ7PVnwz1u0rCO/3rk0Aifp6noikT/dkrTfiLNHhY8+OveKB7R7BuMx6/opVycO3G5XZD1a1Z2WYbtrPw4EhKWNgfodRJB+XissF+FhdQQlk3HeWoAtzPRWrQqAK316cysDm5VOs2ux0Vq2fmqfzM5goXAsDO7PXd3AgvvJUiy8hb3lLM28wpv9uCJTdbSj/SSNsBiqCG6qWrj0Z+bHtqfVTPyTZ1UX3aXLHpSsOo/PRu3ouNB/IotHTTyuEoqpdz5fZDcVmBmH21ZFpzZSfzXfwZIfIQPSJiIgMztASSOlML8sDmv8YSi+eR7uMoY4wP0jYcHmVJfNFnTlkyly58zuQk6lvUPGkYhcDbVgiEWRGgjvJEFUpmOlge2nbuF+1Qt1iNv6oVH/g4jtyzOEUbcf6UBoabnkP657jovW56+J71UDEq/5aYw7YAz18tToggcWEHhW0PIBD3RtOGGK4YGfm9h/rEONLGTfx7rVlVwXfIVTUqc4kzJWrw7Us/X/POIXm/XgeZC2RbWoUVC2eeGm0LiGVIgmhgfF8BFTBPh4uwqd5QtAgWT7SPtq2E0VZI88DncQ2pfRyFsx4+xak5VEaN2ba5virqdDvvmQGKnXxhtZaRbOK/LEX00k0o+a1ZIk4V72g/7JmaoqqNKFGLmS29iVZNA8hP0FWuYlFADjUycu2G0tyvon6/vFrV0sXb1DOTN6xc40TKZhw1eJENKjHPXSYG+3EMQrw4Nml1CrNYDmWK4WaxPNvKIhbSomgWlJ8uVc8KmGMx0/WHGPZTA9HSv0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(7916004)(366004)(66556008)(4326008)(6916009)(85182001)(38100700002)(66476007)(316002)(6512007)(5660300002)(33716001)(8676002)(66946007)(966005)(6486002)(82960400001)(86362001)(508600001)(186003)(6506007)(8936002)(26005)(2906002)(9686003)(6666004)(340984004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eGplbEdSbWtuS2tIaG0xSW9CWWRWZ2FFZFB6Nnc3SjB2NzArTElEUnNzYkNi?= =?utf-8?B?aXRrVnc3ejVOdnZWSWJPS3R3UHduaHdqY2xtZ05uNlNMd1BnY1l5dW9jL1Q3?= =?utf-8?B?NDhKUWZ2NWV4Y2VTMkp4WG9jY0VDMHdEOHBORkJUNi95TzFtNnVwT25WYzNJ?= =?utf-8?B?TjJaMHRWVjZ0YUdWYTBlR01HcW5kaXAvaklIc1NRMDNtMEdVVlF4Zzg3SEt5?= =?utf-8?B?Wk9LNlZpOTNrMmtkdmUwR2t1cWdLYXVTZUJicWhMT0UrMEIwTUZVdXBoMnZQ?= =?utf-8?B?OFlYcHdNZzl1M09MeVk3VlN4UUNBNVhZRThhczdGejd0N1pJZXZvTExJVHVs?= =?utf-8?B?NDZGL2UySHFXNWhjemxuU0VveXh3TmxYeXUzNkg1eXZ4S3B4cWlkYnZKanlP?= =?utf-8?B?OGV1Q000MUVUU1p0RklmZ0ZmVCtqbGpvckRjblZ4WmlhSmhVYXZjS0JJM0sx?= =?utf-8?B?RUpiU1VOZkd0cHhxRzdnN21TVlIrdFRSWWRBc0M5OTk4OTBGaVJTLzhuOE9C?= =?utf-8?B?TlBXYUtTUHJFQkR6Qk4zWVpvWTd5blFPU0ZoMW9KM0czTklRQ2NBRHJHUG9W?= =?utf-8?B?ZWgwS0dVK3l4OXV4cFBOWjg1L3AvRjNUay9IMkhOcHZCUUxjdDBPSDJaeUlU?= =?utf-8?B?OGYxQWFsNnJvQ05lQkl2MDFKNHQ4RU01aFV5VWczUktLbTlIL3l4V205SW4y?= =?utf-8?B?S1VmK0ZuZUM1VUhORE9lSmtLTDBrL3I5WitEYmZtWFBZdmN3UFpwSWk4Z2J3?= =?utf-8?B?YnQ1ZW85Q1hCbC9aK1ovTkFoODFVdlo1b2FuWFR5bXNCRHJGbHF5ZGV6WnBp?= =?utf-8?B?b3YvZW9NYzhYeWpJSzBDdGhIMEc0dE54UjhEc3dHTWZ5b3FIdWpiamMxR0Ra?= =?utf-8?B?ZE16SklWTzJDZTgxVVVlREh0U2twYkFKdytOZTFaV3NvRWxyb3FndHF5QmYx?= =?utf-8?B?RWxFQklpejJUUlZsWWZGUFBDT2I4c0J0TGhlWmluMkVEalBNZ1MrUGZsOWt1?= =?utf-8?B?enMxV0MvUVFyc2Y4M0tObVo0OUNWdUhiV2pmOE9XL0lYbzFadUFvQ0pZYmVH?= =?utf-8?B?NWg2cWJVVFhIY3JrZnhxV1lLSDZiZFRZYVZFS0ZUVVFGR29DMmNjQURoVkZ1?= =?utf-8?B?c3JNSXJLTXQ2MUdmR1dBcDdPR0d4YUtFdVJJR25NWkZQUy9WSFJ4V3g0RGxZ?= =?utf-8?B?dHd6R3lveG43VHYxZ2k2aHZpTnVISzgrYURCNWgrb0RkODRlMExvZVA1R0pn?= =?utf-8?B?U0dEelBUTEZHZkM0SzNaZ3hCaHIvT1JEbjBraUhMQy9WSW9ZZTBtaEJoYzAy?= =?utf-8?B?SHU1aytNRlZkZ0dHV3dlQURQenFUKzV1OEs2S0t5NUJ0WFNvYmMrQ1NlN0hL?= =?utf-8?B?VzFZOWZzanBtZjNLYUxzVkh4K25JQ2Q2M2w4bElrVlZPQ205T1NPaWF4ZjZN?= =?utf-8?B?K2RBY3NlSXRDbzdydGNpWFMraFBMaEdpaXBFWHBpdC9EaXVTWGk3RHR0eDdC?= =?utf-8?B?RjF3MXliZDFmNjE1TUpocDRGWjYxQmUyaTZoU0wxbnpaQVFHSE1JR1hxZGtW?= =?utf-8?B?ZUl0Zk1iZWlZeWp0b0ZHMXZQVFJoOU5vTkhnYXo0RkowUWtER28rMkhRMDNO?= =?utf-8?B?STZUeHA2Umh4Tk5udkxEdkdKVEQ0bCtZc0l0YUM1REszQTFkU2NxN21udmJI?= =?utf-8?B?Tk5LZVIvY3RPQkN3WEFNa3V1QjVONGRYaFZ0eDUrWVBXR0JUaFVKQmNtRmZ2?= =?utf-8?B?dEo2Z2ZPL0VsTkUwdVFsRTNENXlyT0JnTDlIMWNJWGo0V1I5QlFLb0RHV3V5?= =?utf-8?B?VUxzcm9xdFBPZmdMZXl3bFZJYzREbUVhR3FuTXB6QzBENy9PTWxETGNmOE12?= =?utf-8?B?N0VXOGF3V21jYStYaUhuL25XU0p3RkhGaU1WdkxLbmd2dGc4SUpZb3VnV2Nz?= =?utf-8?B?aVZsRUJ6Rzk4RC9RSlV0aGxyeTdQdC9TNkhtRDZsUVRvTTQ4K2tpK0RncGxR?= =?utf-8?B?Vllxd3RjSjVLKzRKQzBJenpHUmNNZDNhUDZDY3RmVG9KWEVYY0c2bjFYajh1?= =?utf-8?B?UjBDZzJqNlhXWnZDOUZrU3BQeGV6ZFRsaUhackZjQXJkZEEzUmtUYVdCUzN1?= =?utf-8?B?Q01EU2EyN2ZHUXZ2bWZ6MkY3L3NOYXVnM3M4cytEbmE2MHhmQkV0ckdRMU14?= =?utf-8?B?S2t2M2dyV3VTeHdXNXZzSi96MHpMelJWK0lSeG41dGtYQUd2TmhCZ0xBL3Np?= =?utf-8?B?WVhBSDM2TXJmcXF0N3dQR3B6QjByT1pkclpZVEdIWUduQXJZNkdNVVJYUUpv?= =?utf-8?B?dHNWS0N3a2lXdjFYeENWb0FCZDNSVW05eTlPK3F3aVhQbzBadCtUaW82RHlL?= =?utf-8?Q?gIcc4r//oPauUZxc=3D?= X-OriginatorOrg: citrix.com X-MS-Exchange-CrossTenant-Network-Message-Id: b630e624-4fcf-4e58-5fb4-08da494b3e02 X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2022 12:34:34.0720 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KgbZfY7S+PB6ju8E5L3qJ+ncKlGp5wD+gBA/Ljcp4h44LzK0l+5wpAL+dfS0JDYyOYhe8K9YE8Rb3QpfiyFU2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR03MB6762 X-Rspamd-Queue-Id: 4LJ69x13qXz3klX X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=citrix.com header.s=securemail header.b=CJptygzM; dkim=pass header.d=citrix.onmicrosoft.com header.s=selector2-citrix-onmicrosoft-com header.b=d1M+dKFP; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=citrix.com; spf=pass (mx1.freebsd.org: domain of "prvs=151432dd3=roger.pau@citrix.com" designates 216.71.155.175 as permitted sender) smtp.mailfrom="prvs=151432dd3=roger.pau@citrix.com" X-Spamd-Result: default: False [-6.20 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+exists:216.71.155.175.spf.hc3370-68.iphmx.com]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[citrix.com:+,citrix.onmicrosoft.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,reject]; RCVD_IN_DNSWL_MED(-0.20)[216.71.155.175:from]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; FROM_NEQ_ENVFROM(0.00)[roger.pau@citrix.com,prvs=151432dd3=roger.pau@citrix.com]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail,citrix.onmicrosoft.com:s=selector2-citrix-onmicrosoft-com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FORGED_SENDER_VERP_SRS(0.00)[]; DWL_DNSWL_LOW(-1.00)[citrix.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-xen]; MID_RHS_NOT_FQDN(0.50)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, Jun 07, 2022 at 12:19:01PM -0700, Brian Buhrow wrote: > hello Roger. I'm happy to do what I can, but it is difficult to get a lot of time to > bring this server down to get diagnostic data. Having said that, I'm happy to look at the code > via bisection if you can help me by providing sample commands to git to assist me in getting > the idea. I'm pretty new to git and am still learning my way around it. `git bisect` [0] will simply point to commits that you build and then tell git whether the code work or not, it's basically a binary search over a range of commits to find the commit that introduced the issue. For example in this case you would use, from a src.git checkout: $ git fetch $ git bisect start 0bbbd845fd32ef07707c03d1dc4e4b27f6ebef83 fc952ac2212b121aa6eefc273f5960ec3e0a466d $ make -j8 kernel $ reboot <... test whether the kernel boots ...> $ git bisect [bad|good] Select either bad or good from the command above depending on whether the proposed commit works. Git will then propose a new commit for you to test until it can pin the commit that introduced the regression. If a commit fails to build you can also use `git bisect skip` to select a different commit for test. > Another question which you may or may not know the answer to. Since the server in question > runs FreeBSD fine when it's running on bare metal, do you think the bad commit would be xen > specific or do you think it's some change related to the difference in the way the hardware > looks to FreeBSD when it's looking at it through xen versus looking at the bare machine? I don't think there have been any? Xen related changes to stable/13 since 13.1 was branched, so it's unlikely to be a Xen specific commit I would think. It's also weird that HEAD works for you, which means it's likely either a commit that changed when backported to stable-13, or a commit that had some prerequisites to work correctly on Xen, and those prerequisites haven't been backported to stable-13. Thanks, Roger. [0] https://git-scm.com/docs/git-bisect