Re: Only 1TB RAM out of 1.5TB on amd64

On Wed, Mar 27, 2024 at 03:01:11AM +0100, Bernd Walter wrote:
> Same Problem with current:
> reeBSD 15.0-CURRENT #0 main-n268793-220ee18f1964: Thu Mar 14 02:58:39 UTC 2024
>  
> real memory  = 1649265344512 (1572862 MB)
> avail memory = 1057118396416 (1008146 MB)
Real memory is really the max physical address of the valid byte.
If you have holes in the phys map, real memory should reflect these
holes.  Avail memory is the total memory as reported by EFI map.

Verify all that by looking at sysctl machdep.efi_map output.

> 
> On Fri, Mar 22, 2024 at 07:54:06PM +0100, Bernd Walter wrote:
> > ..
> > real memory  = 1649265344512 (1572862 MB)
> > avail memory = 1057138888704 (1008166 MB)
> > ..
> > 
> > I suspect address space issues, but don't know for sure.
> > Changing NUMA nodes per socket in the BIOS had an influence to make it
> > worse, but not better.
> > 
> > 
> > Copyright (c) 1992-2021 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> >         The Regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 13.3-RELEASE releng/13.3-n257428-80d2b634ddf0 GENERIC amd64
> > FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367)
> > VT(efifb): resolution 1024x768
> > CPU: AMD EPYC 7352 24-Core Processor                 (2300.07-MHz K8-class CPU)
> >   Origin="AuthenticAMD"  Id=0x830f10  Family=0x17  Model=0x31  Stepping=0
> >   Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
> >   Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
> >   AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
> >   AMD Features2=0x75c237ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX,ADMSKX>
> >   Structured Extended Features=0x219c91a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA>
> >   Structured Extended Features2=0x400004<UMIP,RDPID>
> >   XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
> >   AMD Extended Feature Extensions ID EBX=0x18cf757<CLZERO,IRPerf,XSaveErPtr,RDPRU,MCOMMIT,WBNOINVD,IBPB,IBRS,STIBP,PREFER_IBRS,PPIN,SSBD>
> >   SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
> >   TSC: P-state invariant, performance statistics
> > real memory  = 1649265344512 (1572862 MB)
> > avail memory = 1057138888704 (1008166 MB)
> > Event timer "LAPIC" quality 600
> > ACPI APIC Table: <SUPERM SMCI--MB>
> > FreeBSD/SMP: Multiprocessor System Detected: 96 CPUs
> > FreeBSD/SMP: 2 package(s) x 8 cache groups x 3 core(s) x 2 hardware threads
> > random: registering fast source Intel Secure Key RNG
> > random: fast provider: "Intel Secure Key RNG"
> > random: unblocking device.
> > ioapic0 <Version 2.1> irqs 0-23
> > ioapic1 <Version 2.1> irqs 24-55
> > ioapic2 <Version 2.1> irqs 56-87
> > ioapic3 <Version 2.1> irqs 88-119
> > ioapic4 <Version 2.1> irqs 120-151
> > ioapic5 <Version 2.1> irqs 152-183
> > ioapic6 <Version 2.1> irqs 184-215
> > ioapic7 <Version 2.1> irqs 216-247
> > ioapic8 <Version 2.1> irqs 248-279
> > Launching APs: 32 30 34 13 14 16 23 21 19 24 28 9 86 10 87 7 26 37 39 41 45 46 43 66 71 69 54 57 59 80 82 78 49 53 51 60 65 62 84 88 91 95 74 73 76 92 33 40 11 2 27 22 20 5 44 18 15 36 6 8 25 89 79 85 93 56 48 70 31 81 12 50 38 77 55 63 75 58 1 90 29 94 52 72 67 83 35 42 61 64 68 47 4 17 3
> > random: entropy device external interface
> > kbd1 at kbdmux0
> > ..
> > 
> > kern.maxphys: 1048576
> > hw.physmem: 1099322220544
> > hw.usermem: 1091323355136
> > hw.realmem: 1649265344512
> > vm.phys_locality: 
> > 0: 10 32 
> > 1: 32 10 
> > 
> > vm.phys_segs: 
> > SEGMENT 0:
> > 
> > start:     0x10000
> > end:       0xa0000
> > domain:    0
> > free list: 0xffffffff81ec7ea0
> > 
> > SEGMENT 1:
> > 
> > start:     0x100000
> > end:       0x1000000
> > domain:    0
> > free list: 0xffffffff81ec7ea0
> > 
> > SEGMENT 2:
> > 
> > start:     0x1000000
> > end:       0x74000000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 3:
> > 
> > start:     0x74043000
> > end:       0x75db0000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 4:
> > 
> > start:     0x76000000
> > end:       0x963b8000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 5:
> > 
> > start:     0x963c2000
> > end:       0x963f9000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 6:
> > 
> > start:     0x96500000
> > end:       0xa57f7000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 7:
> > 
> > start:     0xa8e96000
> > end:       0xac000000
> > domain:    0
> > free list: 0xffffffff81ec7c30
> > 
> > SEGMENT 8:
> > 
> > start:     0x10000e000
> > end:       0x7d07800000
> > domain:    0
> > free list: 0xffffffff81ec79c0
> > 
> > SEGMENT 9:
> > 
> > start:     0x10000019000
> > end:       0x1797de00000
> > domain:    1
> > free list: 0xffffffff81ec8110
> > 
> > SEGMENT 10:
> > 
> > start:     0x17ffdc00000
> > end:       0x17ffdde7000
> > domain:    1
> > free list: 0xffffffff81ec8110
> > 
> > 
> > 
> > -- 
> > B.Walter <bernd@bwct.de> https://www.bwct.de
> > Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
> > 
> 
> -- 
> B.Walter <bernd@bwct.de> https://www.bwct.de
> Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.