svn commit: r303583 - head/sys/amd64/amd64

Sun Jul 31 15:26:33 UTC 2016

On Mon, Aug 01, 2016 at 12:30:14AM +1000, Bruce Evans wrote:

> On Sun, 31 Jul 2016, Slawa Olhovchenkov wrote:
> 
> > On Sun, Jul 31, 2016 at 11:11:25PM +1000, Bruce Evans wrote:
> >
> >> Misalignment of this loop made it almost twice as slow on old Turion2 with
> >> slow DDR2 memory.  It made no difference on Haswell.  I added an extra
> >> movnti, but that makes little or no differences.  2 more movnti's wouldn't
> >> fit in a 16-byte cache line so are slower unless even more care is taken
> >> with alignment (or with less care, 4 with misalignment are not less than
> >> twice as slow as 1 with alignment).
> >>
> >> I thought that alignment and unrolling didn't matter here, because movnti
> >> has to wait for memory and almost any loop runs fast enough to keep up.
> >> The timing on my old system is something like: CPUs at 2 GHz; main memory
> >> at 4 GB/sec; movnti is only 4 bytes wide on i386 (so this problem
> >> only affects i386, at least with slow memory).  So sustaining 4 GB/sec
> >> requires 1 G movnti's/sec, so the loop needs to run at 2 cycles/iteration
> >> to keep up.  But when it is misaligned, it runs at 3-4 cycles/iteration.
> >> Alignment makes it take about 2, and the extra movnti is for safety and
> >> to work with faster memory.
> >>
> >> On Haswell with CPUs at 4 GHz, 2 cycles/iteration gives 8 GB/sec on
> >> i386 and 16 GB/sec on amd64 with wider movnti.  IIRC, 16 GB/sec is about
> >> the main memory speed so nothing better is possible but just 1 extra
> >> movnti gives more with faster memory.  This is just worse than bzero()
> >
> > What about modern system with 120 GB/sec main memory speed?
> 
> Is there such a system?  It would have main memory almost twice as fast
> as Haswell L2 and almost half as fast as Haswell L1.

http://ark.intel.com/products/family/93797/Intel-Xeon-Processor-E7-v4-Family#@Server

102 GB/s (sorry, 120 is misprint)

> My fastest memory actually does 20001 MB/s according to old memtest
> and that is about right according to other tests.

Some short time I am have free 1650v4
http://ark.intel.com/products/92994/Intel-Xeon-Processor-E5-1650-v4-15M-Cache-3_60-GHz
with up to 76.8 GB/s (by datasheet, at DDR4-2400).
With installed DDR4-2133 -- up to 68.2 GB/s (teoretical)
After short time system put into production.

I am unable to boot UEFI Memtest86 7.0, old version (4.3.7) show 15 GB/s.

# ramspeed -b 18 -p 4
RAMspeed/SMP (FreeBSD) v3.5.0 by Rhett M. Hollander and Paul V.
Bolotoff, 2002-09

8Gb per pass mode, 4 processes

SSE (nt)  Copy:      54176.91 MB/s  [NTA prefetch]
SSE (nt)  Scale:     54241.98 MB/s  [NTA prefetch]
SSE (nt)  Add:       48945.60 MB/s  [T0 prefetch]
SSE (nt)  Triad:     50102.80 MB/s  [T0 prefetch]
---
SSE (nt)  AVERAGE:   51866.82 MB/s

# ramspeed -b 16 -p 4
RAMspeed/SMP (FreeBSD) v3.5.0 by Rhett M. Hollander and Paul V.
Bolotoff, 2002-09

8Gb per pass mode, 4 processes

SSE & WRITING (nt)        1 Kb block: 55913.18 MB/s
SSE & WRITING (nt)        2 Kb block: 60819.02 MB/s
SSE & WRITING (nt)        4 Kb block: 58662.37 MB/s
SSE & WRITING (nt)        8 Kb block: 57165.14 MB/s
SSE & WRITING (nt)       16 Kb block: 56310.22 MB/s
SSE & WRITING (nt)       32 Kb block: 56407.22 MB/s
SSE & WRITING (nt)       64 Kb block: 58200.44 MB/s
SSE & WRITING (nt)      128 Kb block: 59213.49 MB/s
SSE & WRITING (nt)      256 Kb block: 59047.57 MB/s
SSE & WRITING (nt)      512 Kb block: 59158.01 MB/s
SSE & WRITING (nt)     1024 Kb block: 59140.03 MB/s
SSE & WRITING (nt)     2048 Kb block: 59165.49 MB/s
SSE & WRITING (nt)     4096 Kb block: 59714.68 MB/s
SSE & WRITING (nt)     8192 Kb block: 59926.68 MB/s
SSE & WRITING (nt)    16384 Kb block: 59100.03 MB/s
SSE & WRITING (nt)    32768 Kb block: 58268.52 MB/s

# ramspeed -b 16 -p 2
RAMspeed/SMP (FreeBSD) v3.5.0 by Rhett M. Hollander and Paul V.
Bolotoff, 2002-09

8Gb per pass mode, 2 processes

SSE & WRITING (nt)        1 Kb block: 32131.03 MB/s
SSE & WRITING (nt)        2 Kb block: 41851.23 MB/s
SSE & WRITING (nt)        4 Kb block: 41848.02 MB/s
SSE & WRITING (nt)        8 Kb block: 41640.80 MB/s
SSE & WRITING (nt)       16 Kb block: 41640.60 MB/s
SSE & WRITING (nt)       32 Kb block: 41639.89 MB/s
SSE & WRITING (nt)       64 Kb block: 41849.65 MB/s
SSE & WRITING (nt)      128 Kb block: 41848.74 MB/s
SSE & WRITING (nt)      256 Kb block: 41847.87 MB/s
SSE & WRITING (nt)      512 Kb block: 41846.14 MB/s
SSE & WRITING (nt)     1024 Kb block: 41835.69 MB/s
SSE & WRITING (nt)     2048 Kb block: 41815.94 MB/s
SSE & WRITING (nt)     4096 Kb block: 41717.39 MB/s
SSE & WRITING (nt)     8192 Kb block: 41575.85 MB/s
SSE & WRITING (nt)    16384 Kb block: 41295.03 MB/s
SSE & WRITING (nt)    32768 Kb block: 40735.83 MB/s