DX Unified Infrastructure Management

View Only

Connection spam from MongoDB probe

Shay Terreni posted Jul 17, 2024 04:47 AM

Hi,

Hoping this is the right place to ask about it...

We use these CA probes for many things, among them checking instance status of MongoDB instances (yes, also cluster, but this is about instance)...

Me being the production DBA, and not the monitoring specialist, I received a complaint that one specific instance of our clusters (apparently, but not yet confirmed by me, this happens in other clusters as well), keeps "becoming unavailable"... This, reported by the CA probe, for one single check, every once in a while.
I checked that instance's logs and to my annoyance I can see that specifically during the time of the "failed" checks, the probe opens a massive amount of (99+% of them) accepted connections (with the rest, < 1% , interrupted by the probe) ...
Massive spam of connections.
Every single one of them looks like this:

{"t":{"$date":"2024-07-17T08:26:49.362+02:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"<REMOVING_THIS_FOR_MASKING>","uuid":"fabc28f4-0957-4097-bf86-c9fce0d8ef2e","connectionId":12894848,"connectionCount":483}}

{"t":{"$date":"2024-07-17T08:26:49.362+02:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn12894848","msg":"client metadata","attr":{"remote":"<REMOVING_THIS_FOR_MASKING>","client":"conn12894848","doc":{"driver":{"name":"PyMongo","version":"4.0.1"},"os":{"type":"<REMOVING_THIS_FOR_MASKING>","name":"<REMOVING_THIS_FOR_MASKING>","architecture":"AMD64","version":"10.0.17763-SP0"},"platform":"CPython 3.9.0.final.0"}}}

{"t":{"$date":"2024-07-17T08:26:49.372+02:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn12894848","msg":"Connection ended","attr":{"remote":"<REMOVING_THIS_FOR_MASKING>","uuid":"fabc28f4-0957-4097-bf86-c9fce0d8ef2e","connectionId":12894848,"connectionCount":482}}

Which seems fine to me (maybe I'm wrong and someone will be able to correct me on that?) ...

Has anyone seen something like that happen?

Thanks,
Shay Z.T.