Server locks up if more than one jail in running
Robert Watson
rwatson at freebsd.org
Sat Jan 8 13:38:53 PST 2005
On 8 Jan 2005 matrix at itlegion.ru wrote:
> I have a very bad problem. I need to seetup large number of jails (about
> 100) to do virtual hosting and i am doing it on 5.3-STABLE (not
> 5.3-RELEASE, i mean -STABLE which is cvsupped). So, when i setup one
> jail - everything runs fine for many days. If i setup one more jail or 3
> or more - server locks up (freezes, hangs, becomes totally
> unsresponsive) within 48 hours. Sometime it runs only a couple hours,
> sometimes almost a couple of days. So, to diagnose the problem i setup a
> cron process which saves a lot of system statistics every minutes. The
> last working minute stats are included below as well as basic system
> configuration. Jails are complete distributions, but very basic
> services are running: cron, sshd. adjtimezone is disabled. All other
> records on /etc/crontab are left just like in distro. I really need
> help with this problem because i canno migrate to 5.3 because of it. And
> i really need 5.3 because of some new features. Maybe somone could help.
If you haven't already: could you compile the kernel with "options DDB",
"options KDB", and "options BREAK_TO_DEBUGGER" and see if you can get into
the debugger using a console or (preferably) a serial break? That way we
can attempt to diagnose the state of the system once it is wedged. It
would also be helpful if you could set the system up to run with "options
INVARIANTS", which will have some performance impact but turn on
additional self-checking in the OS. Finally, depending on whether you can
tolerate the performance hit, running with "options WITNESS" (and "options
WITNESS_SKIPSPIN" to help a bit with performance) will allow us to
generate additional debugging information. If you are able to get into
the debugger, it would be helpful if you could run the following, and
ideal send the output copied using a serial console:
- "show pcpu" "show pcpu 0" "show pcpu 1" "show pcpu 2" "show pcpu 3"
- "ps"
- "show threads"
- "show alllocks" # will only work with witness
- "show lockedvnods"
This is a substantial volume of output, so if you don't have access to a
serial console so you can copy and paste on another box, it may be
somewhat inconvenient.
Robert N M Watson
>
>
> SYSTEM:
>
> Multiprocessor. Dual XEON 2.667Ghz with HTT. 4GB DDR RAM.
> RAID 5 (4 SATA disks x 120gb) on 3Ware SATA controller in
> PCI 64 slot on Supermicro MB.
>
> ------------ dmesg --------------------------------------
>
> Copyright (c) 1992-2004 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD 5.3-STABLE #4: Wed Dec 22 10:25:48 MSK 2004
> root at new.itlegion.ru:/usr/obj/usr/src/sys/OMNI
> ACPI APIC Table: <PTLTD APIC >
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Xeon(TM) CPU 2.66GHz (2665.92-MHz 686-class CPU)
> Origin = "GenuineIntel" Id = 0xf25 Stepping = 5
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> Hyperthreading: 2 logical CPUs
> real memory = 4160225280 (3967 MB)
> avail memory = 4077481984 (3888 MB)
> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
> cpu0 (BSP): APIC ID: 0
> cpu1 (AP): APIC ID: 1
> cpu2 (AP): APIC ID: 6
> cpu3 (AP): APIC ID: 7
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
> ioapic2 <Version 2.0> irqs 48-71 on motherboard
> ioapic3 <Version 2.0> irqs 72-95 on motherboard
> ioapic4 <Version 2.0> irqs 96-119 on motherboard
> npx0: [FAST]
> npx0: <math processor> on motherboard
> npx0: INT 16 interface
> acpi0: <PTLTD RSDT> on motherboard
> acpi0: Power Button (fixed)
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
> cpu0: <ACPI CPU (2 Cx states)> on acpi0
> cpu1: <ACPI CPU (2 Cx states)> on acpi0
> cpu2: <ACPI CPU (2 Cx states)> on acpi0
> cpu3: <ACPI CPU (2 Cx states)> on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pci0: <unknown> at device 0.1 (no driver attached)
> pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
> pci1: <ACPI PCI bus> on pcib1
> pci1: <base peripheral, interrupt controller> at device 28.0 (no driver attached)
> pcib2: <ACPI PCI-PCI bridge> at device 29.0 on pci1
> pci2: <ACPI PCI bus> on pcib2
> pci1: <base peripheral, interrupt controller> at device 30.0 (no driver attached)
> pcib3: <ACPI PCI-PCI bridge> at device 31.0 on pci1
> pci3: <ACPI PCI bus> on pcib3
> em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0x3000-0x303f mem 0xfc200000-0xfc21ffff irq 28 at device 2.0 on
> pci3
> em0: Ethernet address: 00:30:48:2a:2d:bc
> em0: Speed:N/A Duplex:N/A
> em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0x3040-0x307f mem 0xfc220000-0xfc23ffff irq 29 at device 2.1 on
> pci3
> em1: Ethernet address: 00:30:48:2a:2d:bd
> em1: Speed:N/A Duplex:N/A
> pcib4: <ACPI PCI-PCI bridge> at device 3.0 on pci0
> pci4: <ACPI PCI bus> on pcib4
> pci4: <base peripheral, interrupt controller> at device 28.0 (no driver attached)
> pcib5: <ACPI PCI-PCI bridge> at device 29.0 on pci4
> pci5: <ACPI PCI bus> on pcib5
> pci4: <base peripheral, interrupt controller> at device 30.0 (no driver attached)
> pcib6: <ACPI PCI-PCI bridge> at device 31.0 on pci4
> pci6: <ACPI PCI bus> on pcib6
> twe0: <3ware Storage Controller. Driver version 1.50.01.002> port 0x4000-0x400f mem 0xfc800000-0xfcffffff irq 72 at device 1.0 on
> pci6
> twe0: [GIANT-LOCKED]
> twe0: 4 ports, Firmware FE7S 1.05.00.063, BIOS BE7X 1.08.00.048
> uhci0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> port 0x2000-0x201f irq 16 at device 29.0 on pci0
> uhci0: [GIANT-LOCKED]
> usb0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> on uhci0
> usb0: USB revision 1.0
> uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub0: 2 ports with 2 removable, self powered
> uhci1: <Intel 82801CA/CAM (ICH3) USB controller USB-B> port 0x2020-0x203f irq 19 at device 29.1 on pci0
> uhci1: [GIANT-LOCKED]
> usb1: <Intel 82801CA/CAM (ICH3) USB controller USB-B> on uhci1
> usb1: USB revision 1.0
> uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub1: 2 ports with 2 removable, self powered
> uhci2: <Intel 82801CA/CAM (ICH3) USB controller USB-C> port 0x2040-0x205f irq 18 at device 29.2 on pci0
> uhci2: [GIANT-LOCKED]
> usb2: <Intel 82801CA/CAM (ICH3) USB controller USB-C> on uhci2
> usb2: USB revision 1.0
> uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub2: 2 ports with 2 removable, self powered
> pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci7: <ACPI PCI bus> on pcib7
> pci7: <display, VGA> at device 1.0 (no driver attached)
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel ICH3 UDMA100 controller> port 0x2060-0x206f,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
> ata0: channel #0 on atapci0
> ata2: channel #1 on atapci0
> pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
> acpi_button0: <Power Button> on acpi0
> atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> sio0: type 16550A
> sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
> sio1: type 16550A
> fdc0: <floppy drive controller> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
> fdc0: [FAST]
> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
> orm0: <ISA Option ROMs> at iomem 0xe0000-0xe3fff,0xc9000-0xc9fff,0xc8000-0xc8fff,0xc0000-0xc7fff on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> Timecounters tick every 10.000 msec
> acd0: CDRW <TOSHIBA DVD-ROM SD-R2412/1015> at ata0-slave UDMA33
> twed0: <Unit 0, RAID5, Normal> on twe0
> twed0: 343417MB (703318656 sectors)
> SMP: AP CPU #2 Launched!
> SMP: AP CPU #1 Launched!
> SMP: AP CPU #3 Launched!
> Mounting root from ufs:/dev/twed0s1a
> em0: Link is up 100 Mbps Full Duplex
>
> ------------ rc.conf --------------------------------------
>
> font8x14="cp866-8x14"
> font8x16="cp866b-8x16"
> font8x8="cp866-8x8"
> keymap="ru.koi8-r"
> keyrate="fast"
> linux_enable="YES"
> mousechar_start="3"
> saver="no"
> scrnmap="koi8-r2cp866"
>
> gateway_enable="YES"
> hostname="omni2.itlegion.ru"
> defaultrouter="213.252.68.1"
> ifconfig_em0="inet 213.252.68.2 netmask 255.255.255.0"
>
> sshd_enable="YES"
> named_enable="YES"
> enable_quotas="YES"
> usbd_enable="NO"
>
> healthd_enable="NO"
> healthd_flags="-6 -l"
>
> syslogd_flags="-ss"
>
> background_fsck="NO"
>
> rpcbind="NO"
> sendmail_enable="NO"
> sendmail_submit_enable="NO"
> sendmail_outbound_enable="NO"
> sendmail_msp_queue_enable="NO"
> inetd_enable="NO"
>
> jail_enable="YES"
> jail_list="213_252_68_6 213_252_68_5"
>
> jail_213_252_68_5_rootdir="/hosts/213.252.68.5"
> jail_213_252_68_5_hostname="jail1.itlegion.ru"
> jail_213_252_68_5_ip="213.252.68.5"
> jail_213_252_68_5_exec="/bin/sh /etc/rc"
> jail_213_252_68_5_devfs_enable="YES"
> jail_213_252_68_5_fdescfs_enable="NO"
> jail_213_252_68_5_procfs_enable="NO"
> jail_213_252_68_5_devfs_ruleset="devfsrules_jail"
>
> jail_213_252_68_6_rootdir="/hosts/213.252.68.6"
> jail_213_252_68_6_hostname="jail2.itlegion.ru"
> jail_213_252_68_6_ip="213.252.68.6"
> jail_213_252_68_6_exec="/bin/sh /etc/rc"
> jail_213_252_68_6_devfs_enable="YES"
> jail_213_252_68_6_fdescfs_enable="NO"
> jail_213_252_68_6_procfs_enable="NO"
> jail_213_252_68_6_devfs_ruleset="devfsrules_jail"
>
> -------- KERNEL CONFIGURATION FILE -------------
> machine i386
> cpu I686_CPU
> ident OMNI2
>
> options SMP
>
> options SCHED_4BSD # 4BSD scheduler
> options INET # InterNETworking
> options INET6 # IPv6 communications protocols
> options FFS # Berkeley Fast Filesystem
> options SOFTUPDATES # Enable FFS soft updates support
> options UFS_ACL # Support for access control lists
> options UFS_DIRHASH # Improve performance on big directories
> options MD_ROOT # MD is a potential root device
> options NFSCLIENT # Network Filesystem Client
> options NFSSERVER # Network Filesystem Server
> options NFS_ROOT # NFS usable as /, requires NFSCLIENT
> options MSDOSFS # MSDOS Filesystem
> options CD9660 # ISO 9660 Filesystem
> options PROCFS # Process filesystem (requires PSEUDOFS)
> options PSEUDOFS # Pseudo-filesystem framework
> options GEOM_GPT # GUID Partition Tables.
> options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!]
> options COMPAT_FREEBSD4 # Compatible with FreeBSD4
> options SCSI_DELAY=15000 # Delay (in ms) before probing SCSI
> #options KTRACE # ktrace(1) support
> options SYSVSHM # SYSV-style shared memory
> options SYSVMSG # SYSV-style message queues
> options SYSVSEM # SYSV-style semaphores
> options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
> options KBD_INSTALL_CDEV # install a CDEV entry in /dev
> options ADAPTIVE_GIANT # Giant mutex is adaptive.
>
> device apic # I/O APIC
>
> # Bus support. Do not remove isa, even if you have no isa slots
> device isa
> device pci
>
> # Floppy drives
> device fdc
>
> # ATA and ATAPI devices
> device ata
> device atadisk # ATA disk drives
> device ataraid # ATA RAID drives
> device atapicd # ATAPI CDROM drives
> device atapifd # ATAPI floppy drives
> device atapist # ATAPI tape drives
> options ATA_STATIC_ID # Static device numbering
>
> # SCSI peripherals
> device scbus # SCSI bus (required for SCSI)
> device ch # SCSI media changers
> device da # Direct Access (disks)
> device sa # Sequential Access (tape etc)
> device cd # CD
> device pass # Passthrough device (direct SCSI access)
> device ses # SCSI Environmental Services (and SAF-TE)
>
> device twe # 3ware ATA RAID
>
> # atkbdc0 controls both the keyboard and the PS/2 mouse
> device atkbdc # AT keyboard controller
> device atkbd # AT keyboard
> device psm # PS/2 mouse
>
> device vga # VGA video card driver
>
> device splash # Splash screen and screen saver support
>
> # syscons is the default console driver, resembling an SCO console
> device sc
>
> device agp # support several AGP chipsets
>
> # Floating point support - do not disable.
> device npx
>
> # Power management support (see NOTES for more options)
> #device apm
> # Add suspend/resume support for the i8254.
> #device pmtimer
>
> # Serial (COM) ports
> device sio # 8250, 16[45]50 based serial ports
>
> # Parallel port
> device ppc
> device ppbus # Parallel port bus (required)
> device lpt # Printer
> device ppi # Parallel port interface device
> #device vpo # Requires scbus and da
>
>
> device miibus # MII bus support
> device fxp # Intel EtherExpress PRO/100B (82557, 82558)
> device em
>
>
> device loop # Network loopback
> device mem # Memory and kernel memory devices
> device io # I/O device
> device random # Entropy device
> device ether # Ethernet support
> #device sl # Kernel SLIP
> #device ppp # Kernel PPP
> device tun # Packet tunnel.
> device pty # Pseudo-ttys (telnet etc)
> device md # Memory "disks"
> #device gif # IPv6 and IPv4 tunneling
> #device faith # IPv6-to-IPv4 relaying (translation)
>
> device bpf # Berkeley packet filter
>
> # USB support
> device uhci # UHCI PCI->USB interface
> device ohci # OHCI PCI->USB interface
> device usb # USB Bus (required)
> #device udbp # USB Double Bulk Pipe devices
> device ugen # Generic
> device uhid # "Human Interface Devices"
> device ukbd # Keyboard
> device ulpt # Printer
> device umass # Disks/Mass storage - Requires scbus and da
> device ums # Mouse
>
>
> # FireWire support
> device firewire # FireWire bus code
> device sbp # SCSI over FireWire (Requires scbus and da)
> device fwe # Ethernet over FireWire (non-standard!)
>
>
> ------- SCRIPT TO SAVE SYSTEM STATUS ------------------
>
> echo '---------------------------------------------------' >> /var/log/_stat
> date >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> uptime >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> ps -axwlH >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> iostat -c 2 -w 10 >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> jls >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> df >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> vmstat -f >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> vmstat -a >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> vmstat -i >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> netstat -an >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
> top -b -d1 >> /var/log/_stat
> echo '---------------------------------------------------' >> /var/log/_stat
>
>
> ----------- LAST WORKING MINUTE STATS ---------------------------------------
>
>
> ---------------------------------------------------
> Thu Jan 6 19:56:00 MSK 2005
> ---------------------------------------------------
> 7:56PM up 4:28, 0 users, load averages: 0.06, 0.03, 0.01
> ---------------------------------------------------
> UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND
> 0 0 0 0 -16 0 0 4 sched DLs ?? 0:00.04 [swapper]
> 0 1 0 0 8 0 740 376 wait ILs ?? 0:00.01 /sbin/init --
> 0 2 0 0 -8 0 0 12 - DL ?? 0:01.67 [g_event]
> 0 3 0 0 -8 0 0 12 - DL ?? 0:01.79 [g_up]
> 0 4 0 0 -8 0 0 12 - DL ?? 0:01.78 [g_down]
> 0 5 0 0 8 0 0 12 - DL ?? 0:00.00 [thread taskq]
> 0 6 0 0 8 0 0 12 - DL ?? 0:00.00 [kqueue taskq]
> 0 7 0 0 -84 0 0 12 actask IL ?? 0:00.00 [acpi_task0]
> 0 8 0 0 -84 0 0 12 actask IL ?? 0:00.00 [acpi_task1]
> 0 9 0 0 -84 0 0 12 actask IL ?? 0:00.00 [acpi_task2]
> 0 10 0 45 97 0 0 12 - RL ?? 267:38.74 [idle: cpu3]
> 0 11 0 44 97 0 0 12 - RL ?? 267:24.91 [idle: cpu2]
> 0 12 0 45 97 0 0 12 - RL ?? 267:22.85 [idle: cpu1]
> 0 13 0 44 97 0 0 12 - RL ?? 267:23.30 [idle: cpu0]
> 0 14 0 0 -60 0 0 12 - WL ?? 0:00.00 [irq1: atkbd0]
> 0 15 0 0 -60 0 0 12 - WL ?? 0:00.00 [irq3: sio1]
> 0 16 0 0 -60 0 0 12 - WL ?? 0:00.00 [irq4: sio0]
> 0 17 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq5:]
> 0 18 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq6: fdc0]
> 0 19 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq7:]
> 0 20 0 0 -84 0 0 12 - WL ?? 0:00.00 [irq8: rtc]
> 0 21 0 0 -52 0 0 12 - WL ?? 0:00.00 [irq9: acpi0]
> 0 22 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq10:]
> 0 23 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq11:]
> 0 24 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq12:]
> 0 25 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq13:]
> 0 26 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq14: ata0]
> 0 27 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq15: ata2]
> 0 28 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq16: uhci0]
> 0 29 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq17:]
> 0 30 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq18: uhci2]
> 0 31 0 0 -64 0 0 12 - WL ?? 0:00.00 [irq19: uhci1]
> 0 32 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq20:]
> 0 33 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq21:]
> 0 34 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq22:]
> 0 35 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq23:]
> 0 36 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq24:]
> 0 37 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq25:]
> 0 38 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq26:]
> 0 39 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq27:]
> 0 40 0 0 -68 0 0 12 - WL ?? 0:00.40 [irq28: em0]
> 0 41 0 0 -68 0 0 12 - WL ?? 0:00.00 [irq29: em1]
> 0 42 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq30:]
> 0 43 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq31:]
> 0 44 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq32:]
> 0 45 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq33:]
> 0 46 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq34:]
> 0 47 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq35:]
> 0 48 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq36:]
> 0 49 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq37:]
> 0 50 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq38:]
> 0 51 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq39:]
> 0 52 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq40:]
> 0 53 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq41:]
> 0 54 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq42:]
> 0 55 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq43:]
> 0 56 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq44:]
> 0 57 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq45:]
> 0 58 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq46:]
> 0 59 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq47:]
> 0 60 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq48:]
> 0 61 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq49:]
> 0 62 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq50:]
> 0 63 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq51:]
> 0 64 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq52:]
> 0 65 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq53:]
> 0 66 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq54:]
> 0 67 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq55:]
> 0 68 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq56:]
> 0 69 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq57:]
> 0 70 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq58:]
> 0 71 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq59:]
> 0 72 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq60:]
> 0 73 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq61:]
> 0 74 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq62:]
> 0 75 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq63:]
> 0 76 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq64:]
> 0 77 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq65:]
> 0 78 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq66:]
> 0 79 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq67:]
> 0 80 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq68:]
> 0 81 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq69:]
> 0 82 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq70:]
> 0 83 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq71:]
> 0 84 0 0 -64 0 0 12 - WL ?? 0:00.50 [irq72: twe0]
> 0 85 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq73:]
> 0 86 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq74:]
> 0 87 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq75:]
> 0 88 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq76:]
> 0 89 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq77:]
> 0 90 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq78:]
> 0 91 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq79:]
> 0 92 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq80:]
> 0 93 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq81:]
> 0 94 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq82:]
> 0 95 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq83:]
> 0 96 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq84:]
> 0 97 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq85:]
> 0 98 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq86:]
> 0 99 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq87:]
> 0 100 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq88:]
> 0 101 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq89:]
> 0 102 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq90:]
> 0 103 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq91:]
> 0 104 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq92:]
> 0 105 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq93:]
> 0 106 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq94:]
> 0 107 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq95:]
> 0 108 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq96:]
> 0 109 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq97:]
> 0 110 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq98:]
> 0 111 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq99:]
> 0 112 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq100:]
> 0 113 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq101:]
> 0 114 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq102:]
> 0 115 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq103:]
> 0 116 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq104:]
> 0 117 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq105:]
> 0 118 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq106:]
> 0 119 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq107:]
> 0 120 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq108:]
> 0 121 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq109:]
> 0 122 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq110:]
> 0 123 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq111:]
> 0 124 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq112:]
> 0 125 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq113:]
> 0 126 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq114:]
> 0 127 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq115:]
> 0 128 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq116:]
> 0 129 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq117:]
> 0 130 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq118:]
> 0 131 0 0 -21 0 0 12 - WL ?? 0:00.00 [irq119:]
> 0 132 0 0 -84 0 0 12 - WL ?? 0:00.00 [irq0: clk]
> 0 133 0 0 -44 0 0 12 - WL ?? 0:00.06 [swi1: net]
> 0 134 0 0 -28 0 0 12 - WL ?? 0:22.54 [swi5: clock sio]
> 0 135 0 0 -32 0 0 12 - WL ?? 0:00.00 [swi4: vm]
> 0 136 0 0 -16 0 0 12 - DL ?? 0:01.62 [yarrow]
> 0 137 0 0 -24 0 0 12 - WL ?? 0:00.00 [swi6:+]
> 0 138 0 0 -40 0 0 12 - WL ?? 0:00.00 [swi2: camnet]
> 0 139 0 0 -36 0 0 12 - WL ?? 0:00.00 [swi3: cambio]
> 0 140 0 0 -24 0 0 12 - WL ?? 0:00.00 [swi6: acpitaskq]
> 0 141 0 0 -24 0 0 12 - WL ?? 0:00.00 [swi6: task queue]
> 0 142 0 0 -24 0 0 12 - WL ?? 0:00.00 [swi6:+]
> 0 143 0 0 8 0 0 12 usbevt DL ?? 0:00.00 [usb0]
> 0 144 0 0 8 0 0 12 usbtsk DL ?? 0:00.00 [usbtask]
> 0 145 0 0 8 0 0 12 usbevt DL ?? 0:00.00 [usb1]
> 0 146 0 0 8 0 0 12 usbevt DL ?? 0:00.00 [usb2]
> 0 147 0 0 -48 0 0 12 - WL ?? 0:00.00 [swi0: sio]
> 0 148 0 0 -8 0 0 12 - DL ?? 0:00.17 [fdc0]
> 0 149 0 0 -16 0 0 12 psleep DL ?? 0:00.04 [pagedaemon]
> 0 150 0 0 20 0 0 12 psleep DL ?? 0:00.00 [vmdaemon]
> 0 151 0 0 171 0 0 12 pgzero DL ?? 0:02.31 [pagezero]
> 0 152 0 0 -16 0 0 12 psleep DL ?? 0:00.60 [bufdaemon]
> 0 153 0 0 -4 0 0 12 vlruwt DL ?? 0:00.58 [vnlru]
> 0 154 0 0 20 0 0 12 syncer DL ?? 0:00.66 [syncer]
> 0 155 0 0 8 0 0 12 - IL ?? 0:00.00 [nfsiod 0]
> 0 156 0 0 8 0 0 12 - IL ?? 0:00.00 [nfsiod 1]
> 0 157 0 0 8 0 0 12 - IL ?? 0:00.00 [nfsiod 2]
> 0 158 0 0 8 0 0 12 - IL ?? 0:00.00 [nfsiod 3]
> 0 159 0 0 -16 0 0 12 - DL ?? 0:01.52 [schedcpu]
> 0 278 1 611 20 0 1188 640 pause ILs ?? 0:00.00 adjkerntz -i
> 0 339 1 611 115 0 516 360 select ILs ?? 0:00.00 /sbin/devd
> 0 363 1 0 96 0 1320 864 select SLs ?? 0:00.10 /usr/sbin/syslogd -l /var/run/log -l /var/named/var/run/log -ss
> 53 377 1 0 20 0 5552 4288 kserel ILs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 53 377 1 0 20 0 5552 4288 kserel ILs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 53 377 1 0 20 0 5552 4288 ksesig SLs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 53 377 1 0 20 0 5552 4288 kserel ILs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 53 377 1 0 96 0 5552 4288 select ILs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 53 377 1 0 20 0 5552 4288 kserel ILs ?? 0:00.50 /usr/sbin/named -u bind -t /var/named
> 0 488 1 0 96 0 3364 2516 select ILs ?? 0:00.01 /usr/sbin/sshd
> 0 505 1 1 8 0 1360 1052 nanslp SLs ?? 0:00.17 /usr/sbin/cron -s
> 0 619 1 0 96 0 1320 872 select ILsJ ?? 0:00.03 /usr/sbin/syslogd -ss
> 0 671 1 0 96 0 3364 2492 select ILsJ ?? 0:00.00 /usr/sbin/sshd
> 0 678 1 0 8 0 1384 1060 nanslp SLsJ ?? 0:00.04 /usr/sbin/cron -s
> 0 737 1 0 96 0 1320 888 select ILsJ ?? 0:00.05 /usr/sbin/syslogd -s
> 0 789 1 0 96 0 3364 2508 select ILsJ ?? 0:00.00 /usr/sbin/sshd
> 0 796 1 69 8 0 1384 1060 nanslp ILsJ ?? 0:00.06 /usr/sbin/cron -s
> 0 801 1 0 96 0 1668 1216 select ILsJ ?? 0:00.00 /usr/libexec/ftpd -D -S
> 0 4756 1 0 4 0 4880 2856 sbwait ILJ ?? 0:00.00 sshd: rooter [pam] (sshd)
> 0 5869 505 1 -8 0 1360 1136 piperd SL ?? 0:00.00 cron: running job (cron)
> 0 5870 5869 1 8 0 1640 1160 wait SLs ?? 0:00.00 /bin/sh -c /root/logstatus.sh
> 0 5871 5870 5 8 0 1644 1164 wait SL ?? 0:00.00 /bin/sh -c /root/logstatus.sh
> 0 5874 5871 5 96 0 1496 988 - RL ?? 0:00.00 ps -axwlH
> 0 835 1 567 5 0 1284 924 ttyin ILs+ v0 0:00.01 /usr/libexec/getty Pc ttyv0
> 0 836 1 567 5 0 1284 928 ttyin ILs+ v1 0:00.01 /usr/libexec/getty Pc ttyv1
> 0 837 1 567 5 0 1284 928 ttyin ILs+ v2 0:00.01 /usr/libexec/getty Pc ttyv2
> 0 838 1 567 5 0 1284 928 ttyin ILs+ v3 0:00.01 /usr/libexec/getty Pc ttyv3
> 0 839 1 567 5 0 1284 928 ttyin ILs+ v4 0:00.01 /usr/libexec/getty Pc ttyv4
> 0 840 1 567 5 0 1284 928 ttyin ILs+ v5 0:00.01 /usr/libexec/getty Pc ttyv5
> 0 841 1 567 5 0 1284 928 ttyin ILs+ v6 0:00.01 /usr/libexec/getty Pc ttyv6
> 0 842 1 567 5 0 1284 928 ttyin ILs+ v7 0:00.01 /usr/libexec/getty Pc ttyv7
> ---------------------------------------------------
> tty twed0 cpu
> tin tout KB/t tps MB/s us ni sy in id
> 0 2 14.46 1 0.01 0 0 0 0 100
> 0 0 16.00 0 0.01 0 0 0 0 100
> ---------------------------------------------------
> JID IP Address Hostname Path
> 2 213.252.68.6 jail2.itlegion.ru /hosts/213.252.68.6
> 1 213.252.68.5 jail1.itlegion.ru /hosts/213.252.68.5
> ---------------------------------------------------
> Filesystem 512-blocks Used Avail Capacity Mounted on
> /dev/twed0s1a 2025948 112836 1751040 6% /
> devfs 2 2 0 100% /dev
> /dev/twed0s1f 650705836 1939120 596710252 0% /hosts
> /dev/twed0s1e 20308316 5340096 13343556 29% /usr
> /dev/twed0s1d 6090012 98196 5504616 2% /var
> devfs 2 2 0 100% /var/named/dev
> devfs 2 2 0 100% /hosts/213.252.68.5/dev
> devfs 2 2 0 100% /hosts/213.252.68.6/dev
> ---------------------------------------------------
> 5182 forks, 247213 pages, average 47.71
> 537 vforks, 32666 pages, average 60.83
> 0 rforks, 0 pages, average 0.00
> ---------------------------------------------------
> procs memory page disk faults cpu
> r b w avm fre flt re pi po fr sr tw0 in sy cs us sy id
> 0 1 0 40420 3920948 30 0 0 0 28 0 0 330 50 621 0 0 100
> ---------------------------------------------------
> interrupt total rate
> irq6: fdc0 9 0
> irq8: rtc 2057141 127
> irq13: npx0 1 0
> irq14: ata0 47 0
> irq28: em0 8184 0
> irq72: twe0 9739 0
> irq0: clk 1607249 99
> Total 3682370 229
> ---------------------------------------------------
> Active Internet connections (including servers)
> Proto Recv-Q Send-Q Local Address Foreign Address (state)
> tcp4 0 0 213.252.68.5.22 195.90.153.20.3119 CLOSE_WAIT
> tcp4 0 0 213.252.68.6.21 *.* LISTEN
> tcp4 0 0 213.252.68.6.22 *.* LISTEN
> tcp4 0 0 213.252.68.5.22 *.* LISTEN
> tcp4 0 0 213.252.68.2.22 *.* LISTEN
> tcp6 0 0 ::1.953 *.* LISTEN
> tcp4 0 0 127.0.0.1.953 *.* LISTEN
> tcp4 0 0 127.0.0.1.53 *.* LISTEN
> tcp4 0 0 213.252.68.2.53 *.* LISTEN
> udp4 0 0 213.252.68.6.514 *.*
> udp6 0 0 *.61254 *.*
> udp4 0 0 *.61253 *.*
> udp4 0 0 127.0.0.1.53 *.*
> udp4 0 0 213.252.68.2.53 *.*
> Active UNIX domain sockets
> Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr
> c5854d20 stream 0 0 0 c585dd20 0 0
> c585dd20 stream 0 0 0 c5854d20 0 0
> c5854b7c stream 0 0 0 0 0 0
> c5855000 stream 0 0 c5850108 0 0 0 /var/run/devd.pipe
> c5854e38 dgram 0 0 0 c596c118 0 0
> c596c118 dgram 0 0 c59be948 0 c5854e38 0 /var/run/logpriv
> c596c1a4 dgram 0 0 c5962948 0 0 0 /var/run/log
> c596c230 dgram 0 0 c5930c60 0 0 0 /var/run/logpriv
> c596c2bc dgram 0 0 c5930d68 0 0 0 /var/run/log
> c5854ec4 dgram 0 0 0 c585d08c 0 0
> c5855ec4 dgram 0 0 c584f948 0 0 0 /var/named/var/run/log
> c585d000 dgram 0 0 c5782a50 0 0 0 /var/run/log
> c585d08c dgram 0 0 c5782b58 0 c5854ec4 0 /var/run/logpriv
> c585d118 dgram 0 0 c5782c60 0 0 0 /var/run/log
> ---------------------------------------------------
> last pid: 5882; load averages: 0.05, 0.02, 0.00 up 0+04:28:24 19:56:10
> 31 processes: 1 running, 30 sleeping
>
> Mem: 9280K Active, 15M Inact, 40M Wired, 1508K Cache, 28M Buf, 3827M Free
> Swap: 1024M Total, 1024M Free
>
>
> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
> 377 bind 20 0 5552K 4288K kserel 2 0:01 0.00% 0.00% named
> 505 root 8 0 1360K 1052K nanslp 0 0:00 0.00% 0.00% cron
> 363 root 96 0 1320K 864K select 0 0:00 0.00% 0.00% syslogd
> 796 root 8 0 1384K 1060K nanslp 0 0:00 0.00% 0.00% cron
> 737 root 96 0 1320K 888K select 0 0:00 0.00% 0.00% syslogd
> 678 root 8 0 1384K 1060K nanslp 0 0:00 0.00% 0.00% cron
> 619 root 96 0 1320K 872K select 0 0:00 0.00% 0.00% syslogd
> 837 root 5 0 1284K 928K ttyin 1 0:00 0.00% 0.00% getty
> 835 root 5 0 1284K 924K ttyin 0 0:00 0.00% 0.00% getty
> 836 root 5 0 1284K 928K ttyin 0 0:00 0.00% 0.00% getty
> 841 root 5 0 1284K 928K ttyin 0 0:00 0.00% 0.00% getty
> 838 root 5 0 1284K 928K ttyin 1 0:00 0.00% 0.00% getty
> 839 root 5 0 1284K 928K ttyin 3 0:00 0.00% 0.00% getty
> 842 root 5 0 1284K 928K ttyin 2 0:00 0.00% 0.00% getty
> 5871 root 8 0 1644K 1164K wait 1 0:00 0.00% 0.00% sh
> 840 root 5 0 1284K 928K ttyin 2 0:00 0.00% 0.00% getty
> 488 root 96 0 3364K 2516K select 0 0:00 0.00% 0.00% sshd
> 5870 root 8 0 1640K 1160K wait 2 0:00 0.00% 0.00% sh
>
> ---------------------------------------------------
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>
More information about the freebsd-stable
mailing list