That is indeed the solution.
The reason is that the Try/Catch/Finally construct works only for terminating errors.
Normally a Get-VM for a non-existing guest is a non-terminating error.
By adding the -ErrorAction Stop, the error became a terminating error.
See also
Get-Help about_Try_Catch_Finally
____________
Blog: LucD notes
Twitter: lucd22