11-10-2009 02:59 PM
I do have a single server which is failing quite often. All others seem to be stable. I am using a healthck as health check for this server. Is there anything at the ServerIron to troubleshot problems with healthck's?
Nov 6 08:30:36:N:L4 server 10.111.23.44 SRV2 port 80 is up
Nov 6 08:30:30:N:L4 server 10.111.23.44 SRV2 port 80 is down due to healthcheck
Nov 6 08:07:51:N:L4 server 10.111.23.44 SRV2 port 80 is up
Nov 6 08:07:38:N:L4 server 10.111.23.44 SRV2 port 80 is down due to healthcheck
Nov 6 07:44:32:N:L4 server 10.111.23.44 SRV2 port 80 is up
server port 80
tcp keepalive 5 5
healthck SRV2 tcp
protocol http url "HEAD /health.html"
server real SRV2 10.111.23.44
port http healthck SRV2
port http url "HEAD /"
11-11-2009 12:06 PM
I would suggest to start with the command "show healthck policy SRV2" - part of the output should be something like the following:
TCP send fail = 0 TCP send buffer alloc fail = 0
TCP request sent = 206 TCP response received = 206
TCP mesg received OK = 0 TCP mesg received ERROR = 206
TCP response timeout = 0 TCP closed by remote = 0
Bring port DOWN = 0 Strtok return null = 0
206 requests so far and 206 responses from a TCP point of view but 206 of the requests went wrong due to the Layer 7 check (HTTP response code of the server not OK).
Additionally (after having checked what I have mentioned above) you could use "debug filter" at the ServerIron to capture the health check traffic at the packet level ( ). Another option would be a to take a trace at the real server itself using tcpdump, wireshark or whatever you have.
On top of that you could use the following to get some more debug messages at the ServerIron:
# ptrace term
# conf t
(config)# server debug boolean SRV2 2
SOME OUTPUTS WILL APPEAR HERE LOOKING LIKE SHOWN BELOW
(config)# server debug boolean SRV2 0
Last command disables debugging again.
Bool in TEST TCP PORT: policy SRV2, dest ip a.b.c.d, port 80
Bool TEST TCP PORT: sent tcp connect, sent=1
Bool SUCC CB: healthck-policy SRV2, dest_ip a.b.c.d, port 80
Bool SUCC CB: send HTTP keepalive HEAD /testme.html HTTP/1.0
Bool RECV CB: healthck-policy SRV2 dest-ip a.b.c.d port 80
Bool RECV CB: status code = 404
Bool RECV CB: HTTP mesg received error, Bring port DOWN