2-tier vs. 3-tier

[Anon Anon]

Hi,

I understand that sizing an NFA solution is not that easy. I'd have a few basic questions:

In a 2-tier setup, what impact does the additional task of managing aggregated data have on a harvester?
Is the 'new', as it is sometimes called, harvester a combination of some 'old' harverster plus DSA ?

I assume, I can move my NFA 2-tier solution to a 3-tier solution at any time by adding the first DSA. Is that correct?
If so, what happens to the data that was already stored on the harvester(s)? Can I move the data into the DSA?

As I can have multiple harvesters and DSAs; is there a mapping between the two?
I believe I've read somewhere that a DSA also has some storage space limitations. How is the data distributed among DSAs?


Thanks for sharing some insights.

Bernd

[Christopher Walsh]

[color=#4205eb][color][color=#4205eb][color][color=#1017ea][color][color=#1017ea][color]Answers below:

In a 2-tier setup, what impact does the additional task of managing aggregated data have on a harvester?
Is the 'new', as it is sometimes called, harvester a combination of some 'old' harverster plus DSA ?

Yes the Harvester is now doing more work in a 2 Tier Architecture, so it is storing 1 minute and now also 15 minute data which used to be on the DSA. However this prevents the bottleneck of the RA Console sending the data to the DSA's which was often a problem in earlier releases.

I assume, I can move my NFA 2-tier solution to a 3-tier solution at any time by adding the first DSA. Is that correct?
If so, what happens to the data that was already stored on the harvester(s)? Can I move the data into the DSA?

Currently there is no procedure that I have seen to migrate from a 2 tier to a 3 tier or vice versa.

As I can have multiple harvesters and DSAs; is there a mapping between the two?

You can have multiple Harvesters and DSA's and routers/interfaces are mapped to a specific Harvester/DSA, but I am not sure if that answers your question. Can you clarify what you mean by 'mapping' between the DSA's/Harvesters?


I believe I've read somewhere that a DSA also has some storage space limitations. How is the data distributed among DSAs?

DSA's have a "Capcity" setting in their database which can store between 500 and I believe 2500 interfaces. If you have multiple DSA's, as more interfaces come online they will start being sent to the second or third DSA's

[Anon Anon]

Christopher,

thanks for your quick reply.


In a 2-tier setup, what impact does the additional task of managing aggregated data have on a harvester?
Is the 'new', as it is sometimes called, harvester a combination of some 'old' harverster plus DSA ?

Yes the Harvester is now doing more work in a 2 Tier Architecture, so it is storing 1 minute and now also 15 minute data which used to be on the DSA. However this prevents the bottleneck of the RA Console sending the data to the DSA's which was often a problem in earlier releases.

It's perhaps difficult to quantify this additional effort? I found one of your answers pointing to the EM Architecture Guide; it suggests that each Harvester could handle about 500 routers. Would you think the impact could be significant to that number?

I assume, I can move my NFA 2-tier solution to a 3-tier solution at any time by adding the first DSA. Is that correct?
If so, what happens to the data that was already stored on the harvester(s)? Can I move the data into the DSA?

Currently there is no procedure that I have seen to migrate from a 2 tier to a 3 tier or vice versa.

Can I read this as 'Yes, you may add a DSA later and get to a 3-tier architecture, but you'll probably loose data collected so far'?

As I can have multiple harvesters and DSAs; is there a mapping between the two?

You can have multiple Harvesters and DSA's and routers/interfaces are mapped to a specific Harvester/DSA, but I am not sure if that answers your question. Can you clarify what you mean by 'mapping' between the DSA's/Harvesters?

With mapping I was thinking of: data from Harvester1 always goes to DSA1 and so on. Probably that's not the case; any newly apearing router (or interface) will be stored on an arbitrary DSA with some capacity left, I guess.

I believe I've read somewhere that a DSA also has some storage space limitations. How is the data distributed among DSAs?

DSA's have a "Capcity" setting in their database which can store between 500 and I believe 2500 interfaces. If you have multiple DSA's, as more interfaces come online they will start being sent to the second or third DSA's

I was trying to understand, how 15-min data is handled when it is stored in more than one DSA; is there some form of 'load balancing' trying to utilise all DSAs equally?

[Christopher Walsh]

1) The more significant statistic that affects what a Harvester can handle is the number of what Flows Per Minute the Harvesters are receiving and from how many interfaces.
There is no set number, but the guide you mentioned gives you some guidelines to work within. Once you are getting close to those limits you should consider using another Harvester especially if you have a lot of custom application definitions, aggregations and anything else that would add additional over head. It is hard to exactly quantify though how much impact each or any of these would have.

2) As for converting from a 2 tier to a 3 tier, like I said there is no procedure for this currently.
So NO you cannot add a DSA to an existing 2 Tier model, you can add more Harvesters if need be.

3) As for mapping data, it doesn't necessarily go from 1 Harvester to 1 DSA.
Each interface will have a harvesterID and a DataServerAddress in the reporter.agent_definitions table on the NFA console server.

4) As for how data is balanced across multiple DSA's, I am not 100% sure how it is distributed across multiple DSA's this is a good question, maybe someone else out there can help with this.

[Stuart Weenig]

It sort of fills up one DSA then moves on to the next. The exact method for balancing the load hasn't ever really been clear.