And if you need to have those prediction alarms, then you also need to have alarm_enrichment probe (that means you have to install nas) into remote hubs. OK, you can deactivate nas, but alarm_enrichment probe is what is used for at least managing the ToT rules.
And then you are in trouble. Because ae running in remote hub that reads all "alarm" messages and transforms them to "alarm2" messages. Well, those can be sent to primary hub no problem but if you have then ae also running in primary hub doing what that probe is for, so enriching alarm content, then there is currently no real supported way to make that work. Alarm_enrichment probe can read only one type of messages and now we are in situation that both "alarm" and "alarm2" messages come in, and both them should be enriched.
Make UIM components compatible with each other, case ToT + ae
This is a "works as designed" situation, because no-one actually has designed these components to work together.
I just got a document about how to make that alarm flow work in 8.4+ using trellis, have not yet tested and the solution is "should work, not yet tested in production" state. Well, based on document study, there is no reason it should not work.
About those usertags in prediction alarms, yes that is also kind of WAD as I think. Those alarms are generated in remote hub, not in that robot where the situation is happening and then that prediction_engine is not knowing actually that information, it just tells that this QOS from that host / target has this kind of situation happening.
This idea is a bit of same concept: Add capability to set user tags on devices discovered by proxy probes such as rsp
Hmm, maybe we need to raise an idea if alarm_enrichment should be used out of the box to fulfill that information. Oh, I forgot that it cannot yet be used officially... (sarc)