Brocade Fibre Channel Networking Community

Expand all | Collapse all

strange vmstat output

  • 1.  strange vmstat output

    Posted 03-22-2013 06:44 AM

    Hello,

    I am experiencing a strange situation on my DCX-8510.

    When I start monitoring CPU performance no matter with vmstat or top command cpu usage peaks up to 100%

    It seems cnmd is using the most part of cpu. Do you have any idea about this issue ?

    My FabOS is : v6.4.2b

    Here is vmstat output:

    procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----

    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa

    3  0      0 582148  28328 839412    0    0     0     1    1    0 19 49 32  0

    1  0      0 582148  28328 839412    0    0     0     0   10  233 29 71  0  0

    2  0      0 582156  28328 839412    0    0     0     0   13  226 30 70  0  0

    1  0      0 582156  28328 839412    0    0     0     0   14  230 29 71  0  0

    1  0      0 582156  28328 839412    0    0     0     0   12  224 33 67  0  0

    Here is top output:

    top - 15:20:32 up 366 days,  2:06,  2 users,  load average: 4.37, 3.63, 3.07

    Tasks: 112 total,   2 running, 110 sleeping,   0 stopped,   0 zombie

    Cpu(s): 38.2%us, 60.5%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  1.3%si,  0.0%st

    Mem:   1863172k total,  1424000k used,   439172k free,    29536k buffers

    Swap:        0k total,        0k used,        0k free,   796864k cached

      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

    2068 root      16   0  139m 7504 5088 S 87.0  0.4 312895:23 cnmd

    1718 root      16   0 92240 6940 4384 S  6.3  0.4  19730:38 emd

    2014 root      16   0 93096  17m 3676 S  3.3  0.9   5314:40 trafd

    1982 root      16   0  114m  50m  47m S  2.0  2.8   1198:43 iswitchd

    3929 root      16   0  2440 1196  940 R  0.3  0.1   0:00.04 top

        1 root      16   0  1696  592  524 S  0.0  0.0   0:32.44 init

        2 root      34  19     0    0    0 S  0.0  0.0   4:28.77 ksoftirqd/0

    Regards,

    Niyazi


    #BrocadeFibreChannelNetworkingCommunity


  • 2.  Re: strange vmstat output

    Posted 03-25-2013 05:22 AM

    Hi,

    Do you see any errors in errdump?

    in how many switchs you see this behavior?

    rgds


    #BrocadeFibreChannelNetworkingCommunity


  • 3.  Re: strange vmstat output

    Posted 03-25-2013 05:51 AM

    Hi,

    I don't see any errors in errdump.

    In all of 4 san directors I see this behavior. They are all at the same FOS level and same hw type.

    Sometime I see other processes in top as follows:

      PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
    2013 root  16   0  121m  14m 9548 S 39.1  0.8   5316:18 snmpd
    1954 root  16   0  153m  23m 9.8m S 27.4  1.3   8871:08 cald
    2068 root  16   0  139m 7504 5088 S 16.0  0.4 316041:23 cnmd
    1718 root  16   0 92240 6940 4384 S  4.9  0.4  19894:47 emd

    Regards,

    Niyazi


    #BrocadeFibreChannelNetworkingCommunity


  • 4.  Re: strange vmstat output

    Posted 03-25-2013 09:12 AM

    Hi,

    I reckon that your switchs are experiencing the behavior reported in DEFECT000370943. The fix to this defect has been added to FOS 6.4.3.

    Kind regards,

    Felipe


    #BrocadeFibreChannelNetworkingCommunity


  • 5.  Re: strange vmstat output

    Posted 03-27-2013 12:33 AM

    Hi,

    I upgraded to 6.4.3d on DR site and yes cpu problem seems to resolved.

    Thanks a lot !

    Regards,

    Niyazi


    #BrocadeFibreChannelNetworkingCommunity