+1 sursu06 if you can, please provide the exception, it would be helpful.
Did I understand correctly that you are trying to open 10,000 files and each file is 249,000 bytes?
Although it is unlikely that these files are all open at the same time, you would consume a total of 2.5 GB of RAM without any of the overhead associated with DevTest if the files were open.
Does each file contain multiple messages or a single 249k message?
When you place the message on the queue, are you waiting for the response and performing validations or is this fire-and-forget?
What does your test case look like? Can you attach or at least share a picture of the model you are executing? Please remove any customer-specific information.
What technique are you using to load the file names?
Does your test need to execute sequentially (i.e., open file 1, send message(s), open file 2, send message(s), or can the test be executed in parallel? If the messages can be executed in parallel, is it possible to externalize the input source filenames into a CVS file rather than read the names from the file system?
When you say 'LISA heap' which of the server components are you referring to? (i.e., memory consumed by Workstation, Coordinator, or Simulator)
Do you specifically allocate memory to any of the servers via the VMOPTION directive (ex: -Xmx512m or -Xmx2048m)?