Create your software resource for the SEP11 .msi, and configure it to run as you prefer. Exclude the .vbs from the software resource. Then create a task, 'SEP11 Pre-Install VBS Run Script Task,' which includes and runs your .vbs. For your software delivery, either create a Managed Software Delivery which follows the .vbs Task with the SEP11 software install, or create a Job which runs the .vbs Task followed by the SEP11 task.
Software Resources are not meant to run multiple commands (e.g. a VBS and an .MSI and a .reg and a .reg). You should stick to one cmmand per software resource.
Regarding the relative file path, let the GUI generate the command line for you -- by doing so, it will insert the correct path to the MSI. Even if you write it yourself, you now know you have the correct path to the MSI for that software resource.
Does this help?