Another Test Run with Alternative pmap Implementation

Wed Nov 26 11:58:43 UTC 2014

Hello,

yesterday I have made two testruns with your updated source tree.

In the FIRST run no debug options were activated. Only ARM_NEW_PMAP was added to the
kernel configuration.

The _build_ (make -j20) hung after about 6 to 7 hours here (see my email from 2014-11-22,
the same situation, I think):

--- cpp_helpers ---

c++   -O -pipe -I/usr/local/DEVEL/STREJDA/freebsd/contrib/atf -Qunused-arguments  -Wno-c+
+11-extensions  -L/usr/obj/usr/local/DEVEL/STREJDA/freebsd/tmp/usr/lib/private
-rpath /usr/lib/private -o cpp_helpers
cpp_helpers.o /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c++/libatf-c+
+.so /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c/libatf-c.so

I tried a second run with make -DNO_CLEAN buildworld, but that hung here:

===> share/termcap (all)

TERM=dumb TERMCAP=dumb: ex - /usr/local/DEVEL/STREJDA/freebsd/share/termcap/termcap.src
< /usr/local/DEVEL/STREJDA/freebsd/share/termcap/reorder
##

To prepare the SECOND run, I build a new kernel with debug options and ARM_NEW_PMAP:

# Debugging support.  Always need this:
options         KDB                     # Enable kernel debugger support.
# For minimum debugger support use KDB_TRACE, for interactive use DDB.
options         KDB_TRACE               # Print a stack trace for a panic.
options         DDB                     # Support DDB.
# For full debugger support use this instead:
#options        GDB                     # Support remote GDB.
# Other debugging options...
makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols
options         ALT_BREAK_TO_DEBUGGER   # Use <CR><tilde><ctrl-b> to enter debugger.
options         DEBUG
options         DEADLKRES               # Enable the deadlock resolver
options         INVARIANTS              # Enable calls of extra sanity checking
options         INVARIANT_SUPPORT       # Extra sanity checks of internal structures,
                                        # required by INVARIANTS 
options         WITNESS                 # Enable
                                        # checks to detect deadlocks and cycles
options         DIAGNOSTIC
#

I build a new kernel, checked all filesystems twice, deleted /usr/obj/* and cleaned the
source tree twice.

Then I did:
root at quad:/usr/src # make -j20 buildworld

the _build_ hung after ca. 8 - 9 hours. top -P on another terminal works, the system
works furthermore. I have waited some time, then breaked to debugger.

--- cpp_helpers ---

c++   -O -pipe -I/usr/local/DEVEL/STREJDA/freebsd/contrib/atf -Qunused-arguments  -Wno-c+
+11-extensions  -L/usr/obj/usr/local/DEVEL/STREJDA/freebsd/tmp/usr/lib/private
-rpath /usr/lib/private -o cpp_helpers
cpp_helpers.o /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c++/libatf-c+
+.so /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c/libatf-c.so

It is the same place, I think.
##

root at quad:/usr/src/sys/arm/include # 

from file vm.h:

#define VM_MEMATTR_UNCACHEABLE  VM_MEMATTR_NOCACHE /*name is misused by DMA */

regards
Ulrich
----------------------------------
On Wed, 26 Nov 2014 11:54:42 +0100
Svatopluk Kraus <onwahe at gmail.com> wrote:

> On Tue, Nov 25, 2014 at 10:54 PM, Ulrich Grey <usenet at ulrich-grey.de> wrote:
> 
> > Hello,
> >
> > I updated the source tree from Svatopluk Kraus and build an Image (crochet,
> > wandboard-quad). The kernel is compiled with ARM_NEW_PMAP:
> >
> > root at quad:/usr/home/gwgpi # uname -a
> > FreeBSD quad 11.0-CURRENT FreeBSD 11.0-CURRENT #0 428e9d2(master)-dirty:
> > Tue Nov 25
> > 09:45:07 UTC 2014
> > root at quad:/usr/local/DEVEL/obj/usr/local/DEVEL/STREJDA/freebsd/sys/WANDBOARD-QUAD
> > arm
> >
> > root at quad:/usr/home/gwgpi # sysctl vm.pmap.
> > vm.pmap.pv_entry_max: 1745184
> > vm.pmap.shpgperproc: 200
> > vm.pmap.nkpt2pg: 32
> > vm.pmap.sp_enabled: 1
> > vm.pmap.pte1.demotions: 22
> > vm.pmap.pte1.mappings: 0
> > vm.pmap.pte1.p_failures: 122
> > vm.pmap.pte1.promotions: 38
> > vm.pmap.pv_entry_count: 12369
> > vm.pmap.pc_chunk_count: 43
> > vm.pmap.pc_chunk_allocs: 1981
> > vm.pmap.pc_chunk_frees: 1938
> > vm.pmap.pc_chunk_tryfail: 0
> > vm.pmap.pv_entry_frees: 417470
> > vm.pmap.pv_entry_allocs: 429839
> > vm.pmap.pv_entry_spare: 2079
> >
> > #
> > Then I did:
> > root at quad:/usr/src # make -j20 buildworld
> >
> > #
> > The build hangs here (not for the first time):
> >
> > --- cpp_helpers ---
> >
> > c++   -O -pipe -I/usr/local/DEVEL/STREJDA/freebsd/contrib/atf
> > -Qunused-arguments  -Wno-c+
> > +11-extensions
> > -L/usr/obj/usr/local/DEVEL/STREJDA/freebsd/tmp/usr/lib/private
> > -rpath /usr/lib/private -o cpp_helpers
> > cpp_helpers.o
> > /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c++/libatf-c+
> > +.so /usr/obj/usr/local/DEVEL/STREJDA/freebsd/lib/atf/libatf-c/libatf-c.so
> >
> > #
> > I did a break into the debugger and this is the output:
> > [...]
> >
> 
> Please tell me:
> (0) Does system hang or only the build? Can you terminate the build when it
> hangs by ctrl c?
> (1) Invariants was on or off?
> (2) Memory attributes for DMA buffers was VM_MEMATTR_NOCACHE or
> VM_MEMATTR_SO?
> (3) How long does the build run before it hangs?
> (4) Does it really hang on same place?
> 
> Can you run the test with invariants on if it was off and with memory
> attributes for DMA buffers VM_MEMATTR_SO if it was VM_MEMATTR_NOCACHE? If
> it helps, set invariants off and try again.
> 
> It's always worth to try it with vm.pmap.sp_enabled=0.
> 
> Considering debug terminal, after you type "show all pcpu", look at output
> and type "where #pid" for current threads on all cpus except idle ones. For
> example, in sent debug output, there is only one non idle current thread on
> cpu #3, so it would be "where 92540".
> 
> Svatopluk Kraus