Hi Jasper,
Please check the settings for OAuth2 Audience Restriction (the check is by default not mandatory).
Also, because you mention only the internal hostname is allowed, check the settings for Host Pages on External Servers. By default this setting take over the hostname from ${gateway.cluster.hostname}, which in your case will be the internal hostname I guess. The #OTK Authorization Server Configuration policy will allow you to override this (of course setting the cluster.hostname can also do this, but you should check side effects as this is more generally used in cases).
See also: Token Exchange Grant and especially #OTK Validate Token Exchange Audience And Scope for the customization policy for validating audience and scope.
Only the OAuth Manager or DCR will allow client registration with private_key_jwt.
Remco