SNMPv3 discovery fails Spectrum 10.4

[Manish Arora]

Hello

I am trying to discover Fortinet, Palo Alto and Aruba switches using SNMP v3 with options selected in SNMP profile for authentication with privacy, Wireshark displays error for malformed packets or decrypted packet not in order .

Tried MIB browser to check the SNMPv3 communication , it works for few devices but not able to discover such devices in Spectrum.

Any suggestions how to resolve it.

Thanks
Manish

[Jason Volpini]

Hi Manish

You could try running an SNMP walk on a few devices from the SpectroSERVER, while Wireshark is also running, and check the returned walk file for errors or issues. This would take Spectrum out of the equation to determine which direction the issue is coming from. Spectrum ships with sapwalk2 utility located in <SPECROOT>/bin directory. 

SNMPv3:

./sapwalk2 -i <ip_addr> -v v3 -u <snmpv3_username> -l <security_level> -xt <MD5/SHA> -xa <auth_passwrd> -xe <DES/3DES/AES128/AES192/AES256> -xp <priv_passwrd> -s 1.3.6.1 -o <output_file>.walk

Where, <security_level> is one of the following:

• nAnP (no Authentication no Privacy)
• AnP (Authentication no Privacy)
• AP (Authentication and Privacy)

If the SNMP walks also fail with errors, and running the walk utility is also showing malformed packets in Wireshark, then you want to check with network team or device owner for the cause. If the SNMP walks are ok, then would recommend to open a Spectrum Support case on the issue.

------------------------------
Senior Support Engineer
Broadcom
------------------------------

[Clark Kimball]

If you are using Windows from the CLI "DOS" CMD prompt, you should be able to SNMPv3 Walk.
However, if from Cygwin bash-login prompt, and you have any special characters in the Priv/Auth passwords, remember to escape them, ie.  \$ 
If from Linux, the same applies for Cygwin.
Also, make sure any Firewalls aren't detecting the SNMPv3 Walk as a non-read operation and dropping the packets.

[Manish Arora]

Rebooted the Server and tried again, issues found on the Network Device configuration. Palo Alto if deployed in HA the secondary device does not respond to request by default so configuration is required to make it respond to Spectrum SNMP requests. Able to discover the devices now.

Thanks everyone for the response.

[Glenn Shoemake]

I usually troubleshoot the problem in the following way:

1. Start a TCPDUmp (Linux) or Wireshark trace on the SpectroSERVER that the device is trying to be modeled on.
2. Try to discover the device and wait for it to fail
3. Dump the SNMPv3 cache into the VNM.OUT with the following command while connected to CLI (vnmsh)
./update action=0x10331 mh=<vnm_mh>
4. Clear the SNMPv3 cache by typing this command while connected to CLI
./update action=0x10333 mh=<vnm_mh>
5. Try to discover the device again, this should work now
6. Stop the TCPDump or Wireshark trace
7. Provide the VNM.OUT, TCPDump (or wireshark trace) and the SNMPv3 credentials so we can decrypt the SNMPv3 traces in a new support issue.

If you do not want to provide this due to confidentiality what I generally look for is why the original discovery was failing.  Normally the device will either not respond or respond with a REPORT and an Error code such as 1.3.6.1.6.3.15.1.1.2.0 (usmStatsNotInTimeWindows), also make note of the EngineBoot EngineID and EngineTime.

Compare these values to the values you receive after you clear the SNMPv3 cache to see if they are similar or completely different which may be caused by having mutliple devices with the same EngineID or the Agent providing a smaller EngineBoot to what Spectrum has.

If any further questions its best to open an issue with all of the info for further analysis.

Best regards,
Glenn