I work in a team of 10 Domain Admins. My colleague and I are the "Virtual Infrastructure" admins. We have created an AD security group named such and added our admin accounts to that group. The "Virtual Infrastructure Admins" group has full administrative rights to the vSphere, we (the two admins) have the root password to the ESX hosts, and do not share that with the rest of the team.
The reasoning for restricting VI full admin access is that the VI is a very delicate environment. Highly tuned, and running smoothly. Any admin COULD make a catastrophic mistake and cripple hundreds of servers. The rest of our team understands and respects this. If they have a request for additional resources, or some sort of Virtual Hardware change, they complete a Change Control document and submit it to us for review and action.
All local admins to any of our guest VM's have "Virtual Machine User" rights, which allows them access to the console through the VI Client. Local admins can be Domain Admins, Vendor Admin Accounts, IS Internal Application Analysts, etc...
This is a political battle for you as well as an over-all system security issue. You need to stress that getting too many hands in the pot will screw up the soup, and that they should be documenting and proposing all changes BEFORE implementing.
Good Luck!