DX NetOps

 View Only
Expand all | Collapse all

Finding models with Network Address not in a list

  • 1.  Finding models with Network Address not in a list

    Posted Mar 30, 2016 12:37 PM

    I have a large list of IP addresses that constitute the body of device that should be in Spectrum. I'm looking to engineer a locator search that would help me find all models with Network Addresses not in my list (so i can delete them). It's been a long day already and i'm not getting anywhere. Does anyone have any slick way of doing this? I have the IP addresses in a line feed separated text file, so it can easily be imported at search launch time. But I can't figure out how to design the search. When i build one from scratch, i can't get it to prompt for a list of addresses (or import a file).



  • 2.  Re: Finding models with Network Address not in a list

    Posted Mar 30, 2016 03:34 PM

    Hi Stuart,

     

    I tried to find a solution for you in Spectrum OC but got stuck as well. Pretty interesting that users can't create searches for IP addresses with the option to enter a list of IP addresses.

    I'm pretty sure you wanted to find a solution in OC to have a comfortable way, but while this doesn't seem to work at all, a workaround might be to export all IP addresses from Spectrum into a second file and just make a "diff" of both files to get all the IPs that are not in your list?

    Do you think this might work for you for now?

     

    Best regards

    Kristian



  • 3.  Re: Finding models with Network Address not in a list

    Posted Mar 30, 2016 04:30 PM

    Your assumption that i'm looking for a built in way to do it because this will be a recurring search that analysts in my offshore team will be doing so i need something not coded.

    Yeah, i always knew i could do this, in fact, i had started to work out a solution before i even go to spectrum so i could just provide the list of IPs for models to delete. I was trying to do it in an inefficient manner though, perhaps if i ran it through python first...



  • 4.  Re: Finding models with Network Address not in a list

    Posted Mar 31, 2016 12:05 PM

    Perhaps you could use a coded solution (a perl/whatever script to do your export/compare/import) and then add a custom menu item in OC that could just execute the script... just a thought...


    Chris



  • 5.  Re: Finding models with Network Address not in a list

    Posted Mar 31, 2016 03:15 PM

    Stuart,


    This may be a bug. When I create a search using the "Equal To" Comparison Type and check the "Prompt when Launched" check box, I get the List button when I launch the search. However, if I change the Comparison Type to "Not Equal To" for the same search, I DO NOT get the List button when I launch the search.

     

    Recommend opening a support case.

     

    Joe



  • 6.  Re: Finding models with Network Address not in a list

    Posted Apr 01, 2016 04:04 AM

    I doubt that this will be accepted as a bug. It will surely be said, that this is FAD.

    The problem will be that the current "not equal to" implementation simply doesn't work with a list, because it's more complicated to implement right than "equal to" against a list.

    This will probably lead to an idea. Either "not equal to" will be improved or there comes a "doesn't contain".



  • 7.  Re: Finding models with Network Address not in a list

    Posted Apr 05, 2016 02:19 PM

    I did run this by the dev team and it is functioning as designed.



  • 8.  Re: Finding models with Network Address not in a list

    Posted Apr 05, 2016 02:40 PM

    Ok. kiran_diwakar, you asked me to come to you when i came across another situation where i asked myself, "Spectrum is how old and it still can't do something this basic?" This is one of those times.

     

    Agile is great and everything, but if you only build the things your users ask for, your product is never going to be where the puck is going to be (search for 'where do you skate').



  • 9.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Apr 06, 2016 02:20 AM

    Hey Stuart,

     

    Completely understand the sentiment.

    At the expense of sounding slightly defensive , I am going to go ahead and say the following..

    All things have some cost associated - not talking for you as a customer and user, but for us as the product team.

    Being a product that has been around for over so many years, and the fact that we are adding newer capabilities, we have a consistent set of things coming from customers, support, sales, market and our own QA team.

    Now the big problem is, like any other team, we have limited resources and what we try to do is balance them across the key initiatives.

     

    Keep customers happy, Improve Operational Efficiency, Keep Competition Away, Create to Markets to sell are few key themes we have and try to balance our work around this.

     

    Just wanted to give you a perspective that we have all the intent to do a lot of things for the tool - but are restricted by bandwidth and hence in some of the cases, resort to taking a sort of poll.

    There many more obvious things we want to do with the product and you will see we will do that. Even within the ideas, we have 300+ of them - and that votes give us a direction of which ones the customers broadly are looking for.

    That is NOT the only metric we use, as you can see we sometimes pick less voted ones too, as we see it can be done rather quickly. It is all about balancing..

     

    We are doing a community webcast and demo next week. We can absolutely talk about this and any other concerns the members might have.

    We have tried to be as transparent and open as possible and we will continue to do so  - we request you to continue to challenge us and keep us on our toes (more)

     

    We listen..

     

    Regards,

    Kiran Diwakar



  • 10.  Re: Finding models with Network Address not in a list

    Posted Apr 07, 2016 05:12 AM

    Sadly I hear this 'functioning as designed' a lot and it seems in some cases the developers don't have any clue how these things work in the real world (or they haven't been told correctly). I must admit the Agile method does seem to be making this happen less and it's more legacy items. These days it seems Kiran and his team do proper investigation and get a lot of feedback on this before implementing something.



  • 11.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Apr 08, 2016 08:22 AM

    Thanks Frank and I admit that in many cases the "functioning as designed" is a quick response. We are pushing our teams to do a bit more thorough reviews and from that perspective the surveys that you guys get after the ticket is closed are very critical. The teams and their management review these surveys very seriously and if you think an engineer has done a good job with the review and then communicated to you the reason for not being able to fix it as a bug, would suggest and request you call it out in the survey - would be a good data point for the management.


    Also, one of the commitments I have made on the SOI webcast and will do the same here - regressions are absolutely a bug - even if it means we need to redesign, we will do that in the backend.Would appreciate if you guys can help with calling out some of these workflows that are problematic wherever possible - specifically the office hours.


    Regards,Kiran Diwakar



  • 12.  Re: Finding models with Network Address not in a list

    Posted Apr 08, 2016 10:39 AM

    Hi Kiran (kiran_diwakar),

     

    The surveys could be a very helpful tool against the quick responses of "this is functioning as designed" and to improve CA's support, but I think these surveys are not used in the best way they could.

    We had an eye on the surveys and closely looked at the cases where we received a survey request for and these were only cases with easy and quick solutions. There is no survey request at all for more complicated cases where things didn't go smoothly.

    Sure, somehow I can understand that. First level engineer has to justify himself when there is bad feedback and he wants to avoid that his boss gets a bad impression. Bad feedback is also bad for the satisfactory report about the support. However, good feedback all the time will probably only result in everybody is going on like before. Constructive criticism - so somehow bad feedback in the meaning of pointing to the things which weren't that good - is the thing that helps to improve. And unfortunately CA doesn't give us the opportunity to do: to criticize constructively the support cases where we have something to say, where it would matter.

    To be honest, I'm ignoring the survey requests for a long time now, because they always come for the "wrong" cases. I already commended the cases and the engineers that where very good, but I won't repeat that other and other again, being deprived the chance to tell someone when it didn't go so well.

     

    So to be constructive here as well I want to make a suggestion. I think "randomly" sent out surveys won't get there where it's necessary and helpful, however you turn it. Giving the users the option to give feedback directly in the supported case whenever he wants, for all cases, this could make the big difference. I mean, not every user who opens a case will give feedback to all of his cases, but when he does, this will really mean something and this might be something CA could and should listen to. People tend to criticize more than they commend, but there will definitely still be good feedback for really good work. But mainly there will be helpful criticism then as well.

     

    I hope this is something to consider.

     

    Best regards

    Kristian



  • 13.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Apr 08, 2016 10:59 AM
      |   view attached

    Hi Kristian,

                    The surveys are not controlled by any of us at CA.  We utilize 3rd party Medallia<http://www.medallia.com/> to initiate which cases surveys are sent out for.  This ensures we are keeping with industry standards and not trying to “control” results.

    Cheers

    Jay



  • 14.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Apr 08, 2016 12:26 PM

    Right, thanks Jay!

     

    This is a serious process and if you haven't received a survey, it is probably coincidental.

    It is a very unbiased process and the results go to CA management as well.

     

    Regards,

    Kiran Diwakar



  • 15.  Re: Finding models with Network Address not in a list

    Posted Apr 11, 2016 09:09 AM

    You guys miss Kristian_S's point. Your survey asks how the support engineer did and how well the interaction was with support. Unfortunately, I don't care to give feedback on that interaction, outside the feedback i give to my sales rep. The feedback I think Kristian is referring to is the feedback about why the support case was needed in the first place. As I see it, all support cases stem from something not working as expected. That has two possible roots: 1) the thing not functioning as designed or 2) the design not being as expected. Unfortunately, when we have an issue with something not working as we expect, but working as designed, we still end up not being able to accomplish what we expected we could accomplish. Often, this feedback would result in an opportunity for development to identify a story and deliver on an (undeclared) expectation.

     

    I like Kristian's idea of having a 'Give Feedback' option in the case at any point. This could launch a more exhaustive survey asking questions first to determine the reason of the failure (1 or 2) and if the reason is the design not being as expected, it could kick off an idea creation workflow.



  • 16.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Apr 13, 2016 01:24 PM

    Stuart, Kristian - sure - let me pass on this feedback to the teams as well.

     

    Regards,
    Kiran Diwakar



  • 17.  Re: Finding models with Network Address not in a list

    Posted Apr 08, 2016 10:44 AM
      |   view attached

    Stuart - Not sure if this is helpful, but here's a very simple Excel example that may do what you need, without scripting/coding, etc.

    Regards.

    Attachment(s)

    xlsx
    IP Address Match.xlsx   12 KB 1 version


  • 18.  Re: Finding models with Network Address not in a list

    Posted Apr 11, 2016 08:53 AM

    Yeah, dumping the data into excel was always an option. Unfortunately, it's not what i'm looking for since I am looking for functionality within the product. This is very similar to how we do things today. We dump the spectrum list into Excel, dump the in-scope IPs into excel, then do a diff to identify removals.



  • 19.  Re: Finding models with Network Address not in a list

    Posted Apr 11, 2016 09:13 AM
      |   view attached

    Yeah, I wondered if maybe you were past this point.

    Best regards.



  • 20.  Re: Finding models with Network Address not in a list

    Broadcom Employee
    Posted Sep 16, 2016 11:35 AM

    Hi Stuart, I know this wasn't the answer you were hoping/looking for...but can you mark this as answered when you have a chance?  

    Thanks

    Jay



  • 21.  Re: Finding models with Network Address not in a list
    Best Answer

    Posted Sep 17, 2016 02:05 PM

    I guess the answer to my question is, "No, there isn't a slick way to do this." There also appears to be no way to do this at all as the product is so poorly designed (on purpose) that nobody even thinks it worth the effort to build a good way of accomplishing it. 

     

    Look guys, i get the "everything has a cost" argument, but it's lazy. Yep, everything has a cost. I'm going into analogy mode: If you build a car that has 4 wheels, an engine, and a steering wheel, yes, you have a car. However, if you don't have side air bags, seat belts, air conditioning, a decent sound system, a locking trunk, nobody is going to buy your car. If you advertise that you have a car, you need to understand why people get pissed off when they purchase it only to find that it has some major components missing that are standard on other cars. You also have to understand that they only way you're going to stay in business is to continue duping new customers. Existing customers will eventually get fed up enough with the "everything has a cost, and we can't justify the cost of {X} component". You know what, that's what they call the cost of doing business. If you can't afford to produce a competitive car, or even the entire car, get out of the business and stop ruining your reputation by tricking people into buying something and delivering only on part.



  • 22.  Re: Finding models with Network Address not in a list

    Posted Sep 28, 2016 04:46 PM

    Hi Stuart,

     

    Currently I have the same issue where I get a list of IP addresses I need to get info from so this seems to be similar in terms of find out which ones on the list are in Spectrum and which are not.

     

    The way I do it is to dynamically create the XML file used in the search criteria. e.g.:

     

    REST 'POST' http://<SERVER>:<PORT>/spectrum/restful/models

     

    <rs:model-request throttlesize="1000"
    xmlns:rs="http://www.ca.com/spectrum/restful/schema/request"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.ca.com/spectrum/restful/schema/request ../../../xsd/Request.xsd">

    <rs:target-models>
    <rs:models-search>
    <rs:search-criteria
    xmlns="http://www.ca.com/spectrum/restful/schema/filter">

    <devices-only-search/>
    <filtered-models>
    <or>
    <equals>
    <attribute id="0x12d7f">
    <value>10.10.10.10</value>
    </attribute>
    </equals>
    <equals>
    <attribute id="0x12d7f">
    <value>11.11.11.11</value>
    </attribute>
    </equals>
    <equals>
    <attribute id="0x12d7f">
    <value>12.12.12.12</value>
    </attribute>
    </equals>
    </or>
    </filtered-models>
    </rs:search-criteria>
    </rs:models-search>
    </rs:target-models>
    <rs:requested-attribute id="0x12d7f"/>
    <rs:requested-attribute id="0x1006e"/>
    <rs:requested-attribute id="0x12de0"/>
    </rs:model-request>

     

    In the code above you just add a new

     

    <equals>
    <attribute id="0x12d7f">
    <value>10.10.10.10</value>
    </attribute>
    </equals>

     

    for each IP address. You have to watch out you don't add to many as this used to crash SpectroSERVER (I have scripts which split up big jobs into batches of maximum 500 logic operations to counter this crash!).

     

    So steps would be to add the top section up to the 'or' tag. Then read the CSV line by line, adding the above section. Then after you have all the IP's add the closing 'or' tag and include the rest adding any other attributes if you need them (in your case for a delete action you might not care about the attributes).

     

    So this will return all attributes for matching devices. You will then need to iterate through the original list and remove matched items so you are left with a list of devices that are not in Spectrum.

     

    You could simply perform a REST mass DELETE using the same method above (i.e. instead of listing the devices, you just delete all the devices.). I think in this case it will provide output of which model handles were deleted. (Always wanted to add a delete function for this but never had to delete so many in one go - just had to list the devices we find/don't find).

     

    Regarding your comment about slick ways to do things, I think you are right but I think it is difficult for Kiran and his team to add features for everything that everyone wants. If it's registered idea and everyone has this issue and it's a big one that many people have, then it will get resolved.

     

    For me Spectrum is a good product that does a lot of things very well. For the more 'advanced' or 'custom' things that you might want to do, you need to invest in the time (which is sadly what many people don't have) to learn things like REST, perl/python/bash/etc, VNM Shell, etc and then to build onto the already provided toolset.

     

    This is what I've been doing for some time, and from the looks of it, you seem to be doing a lot of this already. I'm sure though you have a lot of tools you need to maintain so I feel your pain there as we have so much I'd like to do but there needs to be a good case to invest in the time and that time needs to exist in the first place.

     

    While I understand your frustration I don't think the Spectrum dev team can fix all of these. There is so much Spectrum can do if you can invest in tinkering with it. I've been able to bear the fruits of some of that labour but then I've been working on my automation/scripts for some years and tweak them almost weekly.


    REST is underrated, really it is. It's kind of like when Darth Vader says 'You underestimate the power of the Dark Side'. CA don't 'pimp' the REST enough in my opinion, and it's not only REST but many features of Spectrum.

     

    Hope this helps you or someone else at least!

     

    Regards,


    Frank