I suspect you are trying to this with new New-ViPermission cmdlet ?
Then I can confirm that the cmdlet doesn't seem to take vSwitches nor Portgroups.
It always comes back with the message:
New-VIPermission : Cannot bind parameter 'Entity'. Cannot convert the "PG1" value of type "VMware.VimAutomation.Client20.Host.VirtualPortGroupImpl" to type "VMware.VimAutomation.Types.InventoryItem"