is there please any possibility to set a limit of generated alerts? = if it exceeds i.e 30alert/min, then it would send just 1 alert and generate some i.e. "alert storm" message.
we'd need it as we have APM interconnected with SOI and SOI was overloaded by Nimsoft alerts couple of days ago (=hundreds alerts/1min), so we'd like to prevent this scneraio in the future.
the limit could be set either on the alert level or action (to dont send it to SOI in case of "alert storm")
There is an alert action delay at the bottom of both the caution and danger configuration for just this purpose, you could set the delay to 30 minutes then another alert would only be fired off to SOI if the problem situation persisted.
thank you David,
i havent noticed this functionality before
Not sure if that is relevant to this topic, but if you alert on individual agents then all the alerts for all the agents in that group are ignored for 30 min delay not just the agent that tripped the delay first..
See and vote up Fix or redesign Alert and Event Management
I would suggest you explore and utilize the action delay feature of APM alerts.
I think what he is after is one alert with a count of how many other alerts got suppressed in the 1 minute time interval. I can see that being useful. We do it with other monitoring products we use in our environment.
Are you looking for something like this?
One alert tied to Bytes In Use has 50 agents in its metric group and the alert is set to notify individually. Let's say in one minute, 30 of those go above the danger line. Since the notify individually is selected, 30 alerts will be sent out. So if the action is email, then 30 emails will be sent out.
If I have the above scenario correct, then we currently do not have this functionality and this would be an enhancement request. I would recommend to open an idea on here.
Now if you have 30 separate alerts (regardless of in the same management module or different management modules) and in the same minute they all go above Danger, then we currently do not have this functionality and this would be an enhancement request.
You have received a variety of answers. Please let us know if additional assistance is needed or the answers are sufficient.