Using Wisecomcapture is always the best option to use when registering a DLL. See this article for some background.
But, if there is a case when the DLL doesn't exist in the package and there is a necessity to register a file manually, this tip should prove handy.
To manually register a .DLL using regsvr32.exe in a .MSI package:
Step 1: Go to MSI Script in the WSI project. Add an Execute Program from Destination custom action after InstallFinalize in the Execute Immediate tab. The Execute Program From Destination dialog appears.
Step 2: Fill in the following information: Custom Action Name: <registerdll> Working Directory: Enter SystemFolder EXE and Command Line: Enter the full path to regsvr32.exe and the full path to .DLL. Example: [SystemFolder]regsvr32.exe [WindowsFolder]123.DLL
Step 3: Accept all other defaults and click OK.
Ensure that the custom action has a condition around it so that it only executes when the installation is getting placed on the computer:
If Not Installed Execute Program from Destination End Statement