Service Virtualization

Expand all | Collapse all

TestCase ordering in Workstation

Jump to Best Answer
  • 1.  TestCase ordering in Workstation

    Posted 02-01-2019 06:48 AM

    Hi,

     

    We have 1000+ Automated Test cases in workstation who's naming convention starts with TC1, TC2,TC3....TC10...n

    These test cases needs to be added in suite and run according, but the ordering of test cases are not in correct way.

    PFB snap:

     

    However, it looks fine in our local directory.

    PFB snap:

     

    any suggestions would be helpfull.
    How we can arrange these Test cases in correct order



  • 2.  Re: TestCase ordering in Workstation

    Posted 02-01-2019 08:02 AM

    The testcases are ordered alfabetically, to my knowledge it is not possible to change to another ordering at execution time.

     

    With 1000+ testcases, it would work if your filenames for the first 999 contained leading zeroes. Eg. TC0001 Title Validation.tst

     

    If this is a one time conversion (meaning each new testcase will be compliant with this naming convention) then...

    Notepad++ has a feature to do regex search and replace and to do "block" copy and paste which makes it fairly straightforward to create a batch file with statements like:

     

    RENAME  "TC1 Title Validation.tst"                  "TC0001 Title Validation.tst"

    RENAME  "TC2 Forname Validation.tst"           "TC0002 Forname Validation.tst"

    RENAME  "TC3 Surname Validation.tst"           "TC0001 Surname Validation.tst"

     

    First of all, copy the content of your Tests directory into a backup so you can always recover.

     

    Start with a statement to capture a list of your current testnames: DIR /B > file1

    Load file1 in Notepad++

    Copy and paste the content of file1 into a new document which we call file2

     

    In file1:

    Transform each line from

    TC1 Title Validation.tst

    into

    RENAME "TC1 Title Validation.tst"

     

    By running a search and replace (all)

    Search: ^

    Replace: RENAME "

    and

    Search : $

    Replace: "

     

    In file2:

    Transform each line from

    TC1 Title Validation.tst

    into

    "TC0001 Title Validation.tst"

     

    By running a search and replace (all)

     

    Search: "TC([1-9]) "  

    Replace: "TC000${1} "

    (In Notepad++ the search and replace strings have NO QUOTES, I have put them here to make it clear there is one space at the end of this string)

    and

    Search: "TC([1-9][0-9]) "  

    Replace: "TC00${1} "

    and

    Search: "TC([1-9][0-9][0-9]) "  

    Replace: "TC0${1} "

    and

    Search: ^

    Replace: "

    and

    Search : $

    Replace: "

     

    Now you are ready to create the batch file by copy/past the content of file1 into a new file3

    Go to file2 and do a BLOCK COPY of all filenames (make sure you have the longest lines completely in the block)

    Go to file3, position yourself on the first line and make sure you position yourself in a column beyound the longest linein this file. Now paste the block. Save and run the batch file. (Obviously after doing a quick check of the statements)

     

     

    Good luck,

    Danny



  • 3.  Re: TestCase ordering in Workstation

    Posted 02-01-2019 08:57 AM

    Thanks Danny again, will work on it.
    meanwhile, could you please suggest what could be the perfect way of running these much on test cases in one go.
    as now we are running through suite, or staging it locally, but memory issue arises even after increasing memory in vmoption.

     

    i cannot stage and deploy to centralize machine due to some reason.

     

    any option locally.



  • 4.  Re: TestCase ordering in Workstation

    Posted 02-04-2019 02:59 AM

    Hi,

     

    You have 1000+ testcases to run, that's not a proof of concept situation, it's the real world, and similar to anything else in the real world I don't think there is one perfect way.

     

     

    I would use the old divide and conquer strategy, create10 suite definitions, 100 testcases per suite, use the File Regex Filter (TC0.*, TC1.*, ..., TC9.*). Then create a small batch file with 10 testrunner commands in sequence, each running a Suite.

     

    The divide and conquer strategy might also allow you to execute as much tests in parallel as possible. You mentioned that the testcases must run in order, so some testcases are prerequisites to running later testcases. But I would be surprised if you have 1000+ testcases and they all have to run in sequence. You could think about grouping those testcases with dependencies into one or more additional suites. Those suites would use a staging document defining one (1) instance so the testcases run in sequence. Other suites where the tescases can run in parallel can have many instances defined.

     

    Cheers,

    Danny  



  • 5.  Re: TestCase ordering in Workstation

    Posted 02-04-2019 11:27 AM

    Hi Danny,

     

    I have implemented both of your suggestion.

     

    Thank you for all your proactive support.

     

    lastly , will devtest allow me to run my batch files in parallel.??

    meaning can i run my 10 suites in parallel as all test cases will be hitting to same Rest API



  • 6.  Re: TestCase ordering in Workstation
    Best Answer

    Posted 02-04-2019 11:50 AM

    Theorethically, yes.

     

    Practically, it will depend on the sizing/resourcing of your DevTest server infrastructure. Only guessing but you mentioned you had resource issues when you had your 1000+ testcases in one suite, I would imagine that dividing your 1000+ testcases over 10 suites but then running them in parallel would have similar issues.

     

    In my opinion, assigning more instances (vUsers) to your staging document but running your suites in sequence also increases your throughput of test runs but you have more control.

     

    Cheers,

    Danny