Can single instance of CA SPS act as a proxy server and federation gateway simultaneously ?
Yes it can, a single instance of CA Access Gateway can act as Proxy Server and also co-host federation functionality. However just make sure you design & tune it appropriately to handle traffic loads.
CA Access gateway hosts a variety of functionality + feature in addition to just being a Proxy. I generally do this in POC and Lower Environments (E.g. Dev); where I need to demo functionality e.g. Proxy, Federation, AuthAzWebServices, SessionAssurance, STS etc (I run all the features on the same instance of CA Access Gateway).
However in Production I may just segregrate based on traffic pattern e.g Federation Traffic goes to SPS1 and SPS2, whereas proxy traffic is handled via SPS3 and SPS4. Hence designing strategically is quite important after accounting for following points i.e. the features, future purposes, capacity planning etc.
Remember one can also host multiple INSTANCES of CA Access Gateway on the same box.