Stabilization Announcement for DX NetOps Network Flow Analysis (NFA)

[James Brunner]

Hi all,

I'm after some advice - as per the subject of this post I've just received the "Stabilization Announcement for DX NetOps Network Flow Analysis (NFA)" email.

In our setup we have a single NFA console and three harvesters (one in each client network/tenant due to overlapping IPs etc) - very lightweight and easy to maintain, one Windows server and three RHEL servers. These client networks also contain their own SpectroSERVER and a Data Collector for completeness.

With the move from NFA to NetOps Flow the requirements jump dramatically - there seems to be no simple option for installation except for a multi-server cluster for the 'microservice' components.

So, for each of our client networks we would have to replace the single harvester server with 3 larger boxes to run the per-customer cluster. This is a massive cost increase and a steep learning curve (the documentation is pretty nasty).

Is there any path to install Flow onto a standard server like we do for all other DX NetOps components or is this the point where we are going to have to re-evaluate our continued use of DX NetOps before we are forced to containerise everything?

Thanks in advance.

JB.

[Helen Burke]

Hi James,

It is a common architecture to have separate NFA  harvesters per client. From 25.4.7 NetOps Flow supports a distributed/hybrid architecture allowing for the deployment of Flow Collectors at the remote customer site whilst the centralised service remains at your central site similar to your setup in NFA. 

If you opt for a non HA deployment the flow collector is a single node. I would recommend playing around with our sizing tool. 

Currently that remote site does require a separate install of Kafka, it is actively being worked on to include Kafka within the cluster reducing that requirement.

Please do get in touch if you would like a roadmap of our Flow monitoring or to discuss further

Helen Burke

Product Manager

[James Brunner]

Hi Helen,

The issue isn't the worker node count which is comparable with the current set-up, but the underlying K8 cluster requirements. These requirements push the costs of the Flow component by a factor of 3 to replace each NFA server.

- that is unless there is an approved/documented way to use K8 in a single server deployment running both the Worker and Kafka nodes that we are not aware of?

As Catalin mentioned in his thread "Strategic Direction DX NetOps Components" it seems that while Spectrum is not going to be containerised going forward and CAPM and VNA are only available as a full server installs, will Flow follow the same full server install path to keep consistency across the suite?

Best regards,

James.

[Edward Royston]

James,

There is actually more than just this. AFAIK, the DX Flow collectors (essentially the component that gets deployed to replace the Harvesters in a local K8s cluster) more-or-less act as Netflow forwarders. Basically they convert the incoming Netflow into a Kafka stream that is sent back to your "central" DX Flow cluster for further processing (and interaction with the DRs and Portal). This would give the user flexibility to adjust types of traffic being sent back (supposedly). However, the volume of traffic is substantially more than the old Harvester setup (whereby in the old setup only summarised 15 min data was pushed from the Harvester back to the Console but in the new, not sure as to whether raw netflow is being sent but 1 minute is). This increase in traffic does have an impact on your WAN links - in particular if you are doing things on a global basis.

Whether this Kafka stream of netflow is compressed or not, I do not know - though I would hope that it is. However, this doesn't stop the volume of traffic being sent across the link.

The main thing that the new DX Flow setup addresses is the customer request for fault tolerance within the Flow components. Under NFA, there wasn't any tolerance - though this was justifiable due to the volume of data that was involved (and there can be a LOT). However, "boxes need to be ticked" and so customers were always after this.

I know that this doesn't answer your issue - it extends it - but it does raise more questions around architectures involving DX Flow that, unfortunately, I can't see too many answers to at the moment.

Regards,

Ed

[Helen Burke]

Hi James and Ed,

The distributed flow collector package also includes a flow common services which handles aggregation as per the documentation. The aggregation interval is configurable, by default it is 5 minutes but this can be configured to 15 minutes if required to reduce load on the WAN.

Currently we support two Kubernetes Distributions for NetOps Flow: OpenShift and RKE 2. Both of these support a single node deployment with the control plane and worker node on the same node.

Once again please do get in touch if you would like a full roadmap and to discuss your individual situations

Helen

[e-mail]