So they recommed a tiered baseline_engine, but don't say how. Anyone know?
This is just a guess since there is no documentation!
dist_hub runs baseline_engine
get_queue for the basline_engine:
<baseline_engine.BASELINE_CONFIG>active = yestype = attachsubject = BASELINE_CONFIG</baseline_engine.BASELINE_CONFIG>
Primary also runs baseline_engine for direct attached robots and probes. By not passing the BASELINE_CONFIG past dist hub, the primary won't do redundant calculations for metrics. QOS_BASELINE is the calculated output stored by qos_processor.
Looks like maybe prediction engine could work the same way. Maybe ppm is source of BASELINE_CONFIG?
What about HA setups? If you failover QOS to a different hub, will it still calculate the baseline with continuity? Will it be configured. Should pared HA distribution layer hubs republish BASELINE_CONFIG to eachother?
So many questions. I'm finding if it's not in the docs, support probably doesn't know yet either.
Updates from support on how this allegedly works.
So admin console contacts the baseline_engine on the primary to setup a baseline. This is either a callback or the BASELINE_CONFIG message. Not sure which method is used yet. The latter may just be a way of setting up baselines asynchroniusly so tools don't block while baseline_engine figures it out.
The primary then allegedly contacts the database to lookup the met.id of corresponding baseline in SLM as published by the discovery server. It uses this to track back through relations to what the primary hub is. It then sets the baseline_engine on the primary hub to calculate the baseline through unknown means, probably a callback, though it could be a remote hubpost to use the async delegation?
If there is no baseline engine on the remote edge hub, the primary baseline engine configures itself to calculate the baseline.
Baseline engines on middle tiers are thus useless. There is no information yet on how the calculation would fail over to the secondary hub if there is one set. Also, the implication is that the edge hub requirements are growing significantly. It's no longer a simple communication piece. So add baseline_engine and ppm to the required to be installed on all hubs but not documented list for 8.1. Don't even bother looking for a hardware recomendation for edge hubs or HA setup information. There is still no information on what happens if you add a bunch of baselines before adding an edge hub. Maybe the primary will attempt to re-delegate it. Maybe you will have to figure out how to delete them all and re-add them on your own?
If you have a small environment, maybe one baseline engine centrally will scale. If you don't need redundancy, and you have tons of extra memory on all hubs in your environment, deploying it to every hub is the recommendation. For the rest of us, this looks like more vaporware.
New update from support. If you configure baselines while baseline_engine is only on the primary hub, it will calculate baselines. If you later deploy baseline_engine to the edge, it will delegate calculation to the edge and stop calculating those baselines.
So I hate dredging up old posts but there's not much here on the forum with regards to prediction_engine and its configuration.
So, I'll start off with saying that this description is directly in contradiction to information I've received from support and at least in version 8.31, also contradicts the documentation. That in no way reflects on the accuracy of this conversation but is a data point.
As it was related to me, in order to use prediction_engine, one has to have prediction and baseline installed on the hub closest to the robot for which predictions are being made. This is supported by the observation that if you do not have these probes installed/enabled then the admin console gui (at least for CDM) will no allow one to select the check boxes to turn time to threshold on. That could simply be because PPM only looks for prediction and baseline on the same hub.
And baseline is a prerequisite of prediction so if you have prediction on a hub then you have to have baseline.
So, does anyone have experiences to share with this feature?
Also, has anyone encountered the behavior where the alarms from prediction do not include the user tag or hub/hop fields?
And if you need to have those prediction alarms, then you also need to have alarm_enrichment probe (that means you have to install nas) into remote hubs. OK, you can deactivate nas, but alarm_enrichment probe is what is used for at least managing the ToT rules.
And then you are in trouble. Because ae running in remote hub that reads all "alarm" messages and transforms them to "alarm2" messages. Well, those can be sent to primary hub no problem but if you have then ae also running in primary hub doing what that probe is for, so enriching alarm content, then there is currently no real supported way to make that work. Alarm_enrichment probe can read only one type of messages and now we are in situation that both "alarm" and "alarm2" messages come in, and both them should be enriched.
Make UIM components compatible with each other, case ToT + ae
This is a "works as designed" situation, because no-one actually has designed these components to work together.
I just got a document about how to make that alarm flow work in 8.4+ using trellis, have not yet tested and the solution is "should work, not yet tested in production" state. Well, based on document study, there is no reason it should not work.
About those usertags in prediction alarms, yes that is also kind of WAD as I think. Those alarms are generated in remote hub, not in that robot where the situation is happening and then that prediction_engine is not knowing actually that information, it just tells that this QOS from that host / target has this kind of situation happening.
This idea is a bit of same concept: Add capability to set user tags on devices discovered by proxy probes such as rsp
Hmm, maybe we need to raise an idea if alarm_enrichment should be used out of the box to fulfill that information. Oh, I forgot that it cannot yet be used officially... (sarc)
Regarding the usertags and hub comment, in my situation, it's not getting the info from the hub that created the alert, it's getting it from the next hub on the path to the hub running nas. So, if you have robot R generating the QoS, hub Ah running prediction for R, then tunnel proxy hub Ph retrieving that prediction QoS via a queue, and Nas hub Nh actually processing the alarm data, the QoS and Alerts appear to come from Ph based on the hub but the source indicates R. Every other probe reports Ah as the hub tagging the data and alerts. This appears to be a defect.
We are also facing issues with the tiered baseline deployment and other components discussed in this thread. We have basically the same issues with HA capabilities of baseline engine as well if we move the robot to different hubs manually the baseline config doesn't follow the robot. In our configuration we have ppm + AE on each of our edge hubs where robots are connected.
Can we get an update from CA Product Management (vacro02) or Devs on how to create the recommended configuration or indeed what direction is the baseline engine heading? With the current instructions/documentation/implementation we can't use any baselining in our environment. vacro02
I absolutely agree with what you described.
By the way, here is just an idea how we can automate replication BASELINE_CONFIG rules for HA hub.
1. Create an ATTACH queue (Let's name BASELINE_CFG_For_HA) which subscribes BASELINE_CONFIG in a hub which your robots are talking.
2. Create a GET queue in HA hub which subscribes BASELINE_CFG_For_HA
Here is another option if you don't want to push the baseline configuration to the other hubs as Yu describes, using queues.
You can use the command line utility to create the baselines and thresholds but run it on the primary (and secondary) hub at the top of your architecture so that where ever your robots are, the baselines will be captured (as all QOS goes through the primary).
hope this helps