In IIS, click on the website (e.g., default web site) to bring up its home page. Double-click the Authentication icon in the IIS section. Enable Anonymous, but disable all others.
In IIS, make sure the Siteminder web agent configuration wizard properly added the Siteminder DLL to both ISAPI Filters and Handler Mappings. Use View Ordered List to ensure the DLL is at the top of both lists.
In Siteminder's WebAgent.conf, (in <siteminderhome>\bin\iis) make sure EnableWebAgent is set to "YES". Make sure WebAgent.conf references a valid AgentConfigObject (ACO) on your policy server.
in Siteminder SmHost.conf (in <siteminderhome>\config), make sure if references a valid HostConfigObject (HCO) on your policy server.
In Windows Event Viewer, make sure Siteminder starts and did not throw any error messages.
On your policy server, you definitely need to configure a directory object that points to your MySQL user store.
On the policy serer, you will also need a policy domain for the application. The properties of the policy domain need to point to the directory object. The realm(s) of the policy domain need to reference the DefaulAgentName and/or AgentName(s) in the ACO.