[Differential] [Request, 895 lines] D1711: Changes to the callout code to restore active semantics and also add a test-framework and test to validate thecallout code (and potentially for use by other tests).

[rrs (Randall Stewart)]

rrs created this revision.
rrs added reviewers: gnn, rwatson, imp, hselasky, adrian, sbruno, lstewart.
rrs added a subscriber: freebsd-net.

REVISION SUMMARY
  The callout code had two specific bugs within it after the new CPU migration
  feature was added.
  1) The callout_active() call at times could lie to the KPI user, this is because
      during a migration the ACTIVE flag would have been removed so that 
      the semantics were broken. This could cause problems for KPI users 
      since the active flag is a key feature used by those declaring themselves MPSAFE
      and using there own lock (think TCP).
  2) There was a bug in the callout code where if a migration was going on and
      two calls were made to callout_reset (the first starting the migration) the second
      call would possibly corrupt the linked list (if it was the active callout about to execute) and
      thus we would spin forever in soft clock and this would result in a panic, spin lock held to long.

TEST PLAN
  I have added the kernel test framework and a callout_test module. This
  allows us to pound on the callout code with multiple threads resetting, draining
  and stopping callouts all at about the same time. This test without the fixes would
  lock up and panic with a spin lock held to long vs now it does not.
  
  I have also begun testing this on Netflix Caches to further validate that it does no harm
  and instead gets rid of a panic or two ;-)

REVISION DETAIL
  https://reviews.freebsd.org/D1711

AFFECTED FILES
  sys/amd64/conf/GENERIC
  sys/conf/files
  sys/conf/options
  sys/kern/kern_testfrwk.c
  sys/kern/kern_timeout.c
  sys/modules/Makefile
  sys/modules/callout_test/Makefile
  sys/modules/callout_test/callout_test.c
  sys/sys/callout.h
  sys/sys/callout_test.h
  sys/sys/kern_testfrwk.h

To: rrs, gnn, rwatson, imp, hselasky, adrian, sbruno, lstewart
Cc: freebsd-net