It would stand to reason that if you had enough VLAN broadcast traffic to saturate an ESXi host, the physical switch environment would also be quite loaded (since it receives all of the broadcast traffic in an environment that does not utilize something like VTP pruning). That being said, I have not encountered such a situation where broadcast traffic has negatively impacted ESXi hosts in an enterprise environment spanning several thousand VMs across a few hundred VLANs. There are also physical switch options, such as storm control, that can limit the exposure vulnerability to broadcasts on your trunked interfaces.
I don't think there are any hard and fast numbers that anyone could provide. Even "from experience" examples would be mostly unique to the experience and environment. If you grow to a size where there are a large volume of VMs talking to a large quantity of VLANs, you gain the ability to design around application groups based on VLAN. Considering that there is a finite number of VMs that can be placed on a host (both physically and for risk aversion), this becomes easier as an environment scales.