Application A creates file for consumption in application B. B in turn, send file back to Application A based on data in file and other components. Can DevTest mimic this process?
I would say yes, but all depends on whether you can give DevTest access to these files. Some thoughts...
I agree with you to call this "mimicking a process" and not that you need a virtual service to do this. Because application A is not calling a service of application B. This is more like a (simple) orchestration between applications, ie. a (simple) workflow between applications, and a DevTest testcase design is an implementation of a workflow. There are many OOTB steps, filters and assertions to help you realize this functionality. And any specific functionality you might be missing you can implement via scripting inside an "Execute Script" step.
So, I would build a test case that implements the functionality needed. And I would use CVS (Continuous Validation Services) as a scheduler to run this testcase every minute, or every 5 minutes, or ...
Only snag I can think of, DevTest coordinator/simulator runs as java processes, so can you give them access to the mainframe location where application A creates the files and where the testcase has to create the result file?