amd64/87977: amd64 busdma dflt_lock called (by ata) if physmem>4G

[Jacques Caron]

>Number:         87977
>Category:       amd64
>Synopsis:       amd64 busdma dflt_lock called (by ata) if physmem>4G
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-amd64
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 25 10:30:15 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Jacques Caron
>Release:        5.4
>Organization:
Oxado
>Environment:
FreeBSD db4.fr1.oxado.com 5.4-RELEASE FreeBSD 5.4-RELEASE #0: Thu Oct 20 23:46:49 UTC 2005     root at artemis.oxado.com:/usr/obj/usr/src/sys/SMP  amd64           
>Description:
FreeBSD/amd64 5.4-RELEASE on Tyan K8SD Pro (S2882D) with two Opteron processors and 8 GB RAM, on-board SiI 3114 SATA controller and additional Promise SATA300TX4 SATA controller.

5.4-RELEASE was patched to add support for the Promise controller:

diff -c sys/dev/ata/ata-chipset.c.orig sys/dev/ata/ata-chipset.c
*** sys/dev/ata/ata-chipset.c.orig      Wed May 11 17:15:35 2005
--- sys/dev/ata/ata-chipset.c   Fri Sep  2 16:19:05 2005
***************
*** 1312,1317 ****
--- 1312,1320 ----
       { ATA_PDC20621,  0, PRMIO, PRSX4X,  ATA_UDMA5, "Promise PDC20621" },
       { ATA_PDC20622,  0, PRMIO, PRSX4X,  ATA_SA150, "Promise PDC20622" },
       { ATA_PDC40518,  0, PRMIO, PRSATA2, ATA_SA150, "Promise PDC40518" },
+      { ATA_PDC40519,  0, PRMIO, PRSATA2, ATA_SA150, "Promise PDC40519" },
+      { ATA_PDC40718,  0, PRMIO, PRSATA2, ATA_SA150, "Promise PDC40718" },
+      { ATA_PDC40719,  0, PRMIO, PRSATA2, ATA_SA150, "Promise PDC40719" },
       { 0, 0, 0, 0, 0, 0}};
      char buffer[64];
      uintptr_t devid = 0;
diff -c sys/dev/ata/ata-pci.h.orig sys/dev/ata/ata-pci.h
*** sys/dev/ata/ata-pci.h.orig  Wed May 11 17:15:35 2005
--- sys/dev/ata/ata-pci.h       Fri Sep  2 16:19:56 2005
***************
*** 180,185 ****
--- 180,188 ----
  #define ATA_PDC20579          0x3574105a
  #define ATA_PDC20580          0x3570105a
  #define ATA_PDC40518          0x3d18105a
+ #define ATA_PDC40519            0x3519105a
+ #define ATA_PDC40718            0x3d17105a
+ #define ATA_PDC40719            0x3515105a
  #define ATA_PDC20617          0x6617105a
  #define ATA_PDC20618          0x6626105a
  #define ATA_PDC20619          0x6629105a

Any kind of serious I/O involving several of the Promise-based disks or a combination of the Promise-based disks and the SiI-based ones will cause a panic "busdma dflt_lock called".

Any level of I/O on the SiI only apparently never causes the problem.

Setting hw.physmem=3g in /boot/loader.conf apparently makes the problem go away, but I need the additional memory...

Similar issues with other drivers are found here:
http://www.monkey.org/freebsd/archive/freebsd-amd64/200411/msg00070.html

and possibly here:
http://www.mail-archive.com/freebsd-hardware@freebsd.org/msg00536.html

It seems the issue is either in the busdma code, or the way the busdma code is used by those drivers.

As an aside, fxp has a long delay during boot with 8G, but this goes away when hw.physmem is set to 3g.

>How-To-Repeat:
amd64 with 8G RAM and 4 disks on a Promise SATA300TX4,

dd of=/dev/null if=/dev/ad12 bs=256000 &
dd of=/dev/null if=/dev/ad14 bs=256000 &
dd of=/dev/null if=/dev/ad16 bs=256000 &
dd of=/dev/null if=/dev/ad18 bs=256000 &

You will never get to the last one.

>Fix:
Workaround (not fix): set hw.physmem to less than 4G.


>Release-Note:
>Audit-Trail:
>Unformatted: