Errors when sending email via SMG from Exchange

[Tomasz Zielinski]

When Exchange (from Outlook) is sending emails via SMG i have errors in SMG "Abort message (message time out)". Problem was reported to Broadcom Support and after looking at packet capture they found that Exchange send data packet to SMG, SMG sends ACK to Exchange, SMG sends ACK with TCP widow update to Exchange, five minutes after most recent ACK no other packets have been received form Exchang and SMG time out the sesion. 
In Exchange transport service logs I have 

2023-05-18T20:18:41.277Z,SMG,08DB53CAAC00F8FF,21,192.168.45.35:49461,192.168.45.41:25,*,,"HandleError has encountered a suspicious connection reset from a remote, non-mailbox transport server."

From Exchange Queue 

[{LED=450 4.4.318 Connection was closed abruptly (SuspiciousRemoteServerError)} Exchange 2016

Emails sended via OWA are working fine

[alexander smg]

Certificate issues. ?

[Tomasz Zielinski]

After 4 atempts message is sent. To low max connection in Exchange connector?

[alexander smg]

If smg is closing the connection, then it’s smg. I ask again. Tls cert????

[Art_P]

I just said that it wasn't an action from SMG, that's not what "Abort message (message timed out)" means. There's not enough info to indicate a potential cert related issue, I mentioned that the statements about the packets lacked context. However it's not SMG that is closing the connection. 
If it were cert related, it wouldn't likely be directly related to the cert. It wasn't a cert failure, which would have specific responses, it was a timeout due to lack of network packet responses to the Messaging Gateway.

------------------------------
---------------------------------------------
Support Engineer
* Integrated Cyber Defense Exchange
* Messaging Gateway
* Packet Shaper
Symantec Enterprise Division
Broadcom Software
------------------------------

[Art_P]

Sounds like something might be between SMG and Exchange that should be looked at. the "Abort message (message timed out)" specifically means that the Messaging Gateway was communicating with the other server but the network connection was lost (not due to direct action from SMG). Your description of the packet capture lacked context, but in general you said that SMG sent an ACK (meaning it acknowledged Exchanges previous packet) and would generally then be waiting for Exchange's next packet, which never arrived - ending in time out.
Your Exchange logs seem to indicate that Exchange thought SMG stopped responding. Thus, most likely culprit is something in between.

------------------------------
---------------------------------------------
Support Engineer
* Integrated Cyber Defense Exchange
* Messaging Gateway
* Packet Shaper
Symantec Enterprise Division
Broadcom Software
------------------------------

[Tomasz Zielinski]

Description of pacet capture is from Case 33390143. Exchange and VM are in VLAN, same site maybe sometimes different host. But from OWA mails are working. From transport service logs i se that certificate negotiation using tls 1.2 is succeded 

[alexander smg]

The issue is new???
On May 24, 2023, at 10:18 AM, Tomasz Zielinski via Broadcom

[Tomasz Zielinski]

Nope, SMG is only for inbound mail. I want to enable outgoing mails via SMG and this hapens

[alexander smg]

Capture. And pic of errors

[alexander smg]

I also didn’t get your packet capture. I will need that as well please.

[Art_P]

It is absolutely not acceptable for you to ask for a packet capture, or any personal/business data, from someone else. Please refrain from such behavior.

------------------------------
---------------------------------------------
Support Engineer
* Integrated Cyber Defense Exchange
* Messaging Gateway
* Packet Shaper
Symantec Enterprise Division
Broadcom Software
------------------------------

[alexander smg]

I am helping

[Tomasz Zielinski]

Packet capture was sent to Broadcom Support and they replied with this

I think that problem is with Exchange receive connector. Email's that i've tracked were using connector with port 2525

[alexander smg]

Did you put the smg ip in its own connector to test.
On Jun 1, 2023, at 7:04 AM, Tomasz Zielinski via Broadcom

[Tomasz Zielinski]

Yes, when im testing i'm disabling Internet connector and leave only smg connector. When i'm sending emails with only subject name they are delivered. When email has small attachment, problem occurs

[alexander smg]

Show us both errors: exchange error and smg error please. Pics also help.
On Jun 7, 2023, at 7:45 AM, Tomasz Zielinski via Broadcom

[Tomasz Zielinski]

They are in first post. Yesterday I also changed log level to debug level but only error i see is "read: SSL_ERROR_WANT_READ"

[alexander smg]

Tls error
On Jun 7, 2023, at 8:02 AM, Tomasz Zielinski via Broadcom

[Tomasz Zielinski]

But why mails without attachment are delivered?

[Art_P]

I mentioned previously that it appears something is between the Messaging Gateway and your internal mail server. The things you've posted here also suggest that. The error from your exchange server is "HandleError has encountered a suspicious connection reset from a remote, non-mailbox transport server.", the error from Messaging Gateway is that it experienced a timeout waiting for an update from 192.168.45.35. Those are not the same results, the Messaging Gateway did not reset the connection, the 45.35 host did. So, when two different conversations appear to be happening, it tends to be something in the middle that's causing it. It could be a load balancer, a firewall, some other security process, or even potentially a bad cable or switch port.
At any rate, all that's been posted here is pointing to something other than the Messaging Gateway as the cause. If you have other evidence that the Messaging Gateway may be involved as a cause, feel free to post it or open a case and we will revise, but otherwise you should be investigating the network.

------------------------------
---------------------------------------------
Support Engineer
* Integrated Cyber Defense Exchange
* Messaging Gateway
* Packet Shaper
Symantec Enterprise Division
Broadcom Software
------------------------------

[alexander smg]

What is the email path of the flow.

[Jim Wallis]

Have you checked for IP address conflicts on your LAN? 

[Tomasz Zielinski]

There is no IP conflict. Exchange and SMG are in same VXLAN, MTU is 1500, firewall in Exchange is off. Should I run wireshark on Exchange? Alexander, I don't know what You meen by "What is the email path of the flow." It's standard flow from Outlook to Exchange and then via external connector to SMG

[alexander smg]

U need a packet capture on both ends