Reduced java/tomcat performance 6-beta3 -> 6-stable ?

[Eirik Øverby]

On Nov 28, 2005, at 14:45 , Joseph Koshy wrote:

> On 11/26/05, Eirik Øverby <ltning at anduin.net> wrote:
> EØ> [Cross-posting after lack of response on -stable]
>
> The first step would be do some performance debugging.

Yep.

>  - What do top/vmstat/systat say about what the OS and
>    apps are doing?  Is the CPU pegged at 100%?  What's
>    the load seen by the disks?  Is the RAID in good health?

vmstat during system idle times are found below. I think they are  
rather interesting. To your other questions: The CPU usage is  
comparable on both systems. Not pegged at 100%, but load seems to  
stabilize around 0.5. Disk load is minimal on the application  
servers, somewhat more on the database servers, but they are not  
interesting here (they are not the bottle neck, and they perform  
equally). The RAIDs are in good health on both systems.

The vmstat output is interesting.
 From the "fast" system (6.0-BETA3, ~idle):
[root at app_host01] ~# vmstat -w 5
procs      memory      page                    disks     faults      cpu
r b w     avm    fre  flt  re  pi  po  fr  sr da0 pa0   in   sy  cs  
us sy id
1 0 0 2439220  38048   14   0   0   0  14   0   0   0  170  141 437   
0  0 100
0 0 0 2439220  38028    2   0   0   0   3   0   2   0  192   94 475   
0  0 100
0 0 0 2439220  37916    1   0   0   0   6   0   1   0  291  925 926   
5  0 94
0 0 0 2439220  37916    0   0   0   0   0   0   0   0  185   91 458   
0  0 100
0 0 0 2439220  37820    1   0   0   0   6   0   3   0  289 1163 1124   
6  0 94
0 0 0 2439220  37820    0   0   0   0   0   0   0   0  183   91 454   
0  0 100

 From the "slow" system (6.0-BETA3, ~idle):
[root at app_host02] ~# vmstat -w 5
procs      memory      page                    disks     faults      cpu
r b w     avm    fre  flt  re  pi  po  fr  sr da0 pa0   in   sy  cs  
us sy id
0 0 1 2468180  51660   15   0   0   0  18   4   0   0 1048 3200 5130   
0  0 100
0 0 0 2468180  51660    1   0   0   0   0   0   0   0 1004 3068 5063   
0  0 100
0 0 0 2468180  51660    0   0   0   0   0   0   0   0 1003 3094 5057   
0  0 100
0 0 0 2468180  51660    0   0   0   0   0   0   1   0 1005 3068 5065   
0  0 100
0 0 0 2468180  51656    1   0   0   0   0   0   0   0 1002 3090 5054   
0  1 99
0 0 0 2468180  51656    0   0   0   0   0   0   0   0 1002 3064 5053   
0  0 100

*loads* more context switches than on the BETA-3 system. I have not  
yet tried this during load; I have to wait for the testing window for  
that. But perhaps this helps? What do I look for next?

>  - Any unusual messages in /var/log/messages?  Any errors
>    shown by the network interfaces (I'm assuming the
>    application is using the network).

No errors shown that I can determine.

>  - A brief description of the workload presented by
>    the app would help.

This is a web application (payment gateway) that receives a HTTP  
POST, does some processing, asks an external service for a piece of  
information, then returns the gathered information to the client. The  
call to the external service can be eliminated, but does not change  
the performance profile.
How the application works internally is impossible for me to say;  
it's 3rd party. I can say, after asking them, that it is "moderately"  
threaded. Whatever "moderately" threaded. My interpretation is that  
the heaviest threading happens in tomcat itself, with up to 150  
concurrent connection threads running.

Thanks,
/Eirik

>
> --
> FreeBSD Volunteer,     http://people.freebsd.org/~jkoshy
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable- 
> unsubscribe at freebsd.org"
>
>