Plex and Unit Testing

[Legacy User]

[left] [left] I am curious if anyone out there is doing any sort of formal unit testing in conjunction with their regular Plex development.   Traditionally, all of the unit-type (as opposed to acceptance, QA,  regression, etc.)  testing we do has been pretty informal.   We have experimented with building our own unit testing tools in Plex with some success, but more recently I have been experimenting with using NUnit.   Our environment happens to be WinC/BackOffice, but I'd be curious to hear what people are using in  any other  environments as well.[left]  [left] John[left] Data Specialists, Inc.[left]

Message Edited by JohnBell on 02-08-2008 04:34 PM

[Srini]

Hi John,
We started using the JUnit to test the webservices based infrastructure of the Plex code and later on moved to NUnit when we migrated to the C# based backend from the Java backend. We ended up with MBUnit as I like the extra functionally that MBUnit gives.

This being said there are some practical issues we faced when using the Unit testing framework as other developers (at least in our shop) do not have the working knowledge of writing the code in Java/C# which poses issues when writing these test cases and we end up with one coder doing this. As you can imagine this does pose issues with the over all development/testing of the application. If the developers who do not know how to write the code in native language (other than Plex) this would be at a disadvantage. The management does not have issues with the unit testing but they may not have the resources (both time and money) to bring other developers to learn new skill sets. At the end it will be up to the individual developer to see if they want to do learn a new tool/and language to do this.

This being said we saw lot of potential in this area. In our shop we are testing less than 1% of the application using MBUnit. But, This code is the center piece of the application and does has huge impact on the over application, so we are able to devote the resouces which helped us in the long run.

On the side note, When I was working on the JUnit I was able to write dynamic test cases using reflection, for the Plex functions. Hopefully some day I may post my experiences on my blog or in the plex wikipedia.

I did not know that we can test the WinNTC with NUnit, can you post your experiences and how you are doing it on Plex Wiki?

Regards
Srini

[Legacy User]

Srini,[left]  [left] Excellent feedback.   Thanks for your post.   I will post  a more complete article  on the Plex Wiki at some point.   Just have to find the time.   Glad to hear there is interest, that will get me motivated.   In the meantime, I have a couple of comment on issues you raise.[left]  [left] I too have run into issues with the fact that most of our internal resources are Plex developers and the fact that many will not  have the  language skills required to make direct  use of existing unit testing tools.   We have experimented with mitigating this issue by writing our unit tests in Plex and exposing them to the outside world via COM, in our case.   I then write the hook into the testing frameworks in C# using COM interop.   Of course, this means we are limited to unit testing the server side content for the time being.   I would imagine that a similar scenario would work even better writing unit tests in Plex and exposing them via .Net assemblies.   This way, the amount of work required in Java/C# to plug the tests into a unit testing framework becomes quite minimal.   This has worked for us relatively well on a very small scale, less that 1% code coverage at the moment, but I am encouraged by what I've seen thus far.[left]  [left] Of course, the route I'm taking does require that our build server have the Plex runtime installed in order to run the unit tests, but I'm okay with that.   My primary concern at the moment is the development overhead in requiring unit tests be written.   I'm basically asking developers to develop twice as much code as they used to.   In the long run I'm not sure if we will be able to sustain that.   At some point I will have gathered enough real experience to be able to determine whether or not the excercise is paying off.   I'm still waiting for the day when one of our unit tests catches an issue that would have made it to QA otherwise.[left]  [left] Regards,[left] John[left]

[Legacy User]

[left] Srini,[left]  [left] I have begun a Continuous Integration article on the Plex Wiki.   I'm still filling in details, but that will be the place to watch for my comments on introducing unit testing into various Plex environments.   There are spots in there for someone to comment about JUnit and MBUnit as well (hint hint :) ).   If you have time, I would appreciate any contributions you might have on any of the areas covered in the article.[left]  [left] Regards,[left] John Bell[left] Data Specialists, Inc.[left]

Message Edited by JohnBell on 02-20-2008 10:10 AM

[Anon Anon]

[left] John -[left]  [left] I'll throw another option out there. At ADC Austin, we use Worksoft Certify on all projects, unless our clients mandate otherwise. http://www.worksoft.com/index.cfm/page/Worksoft_Certify_Platform[left]  [left] Our business is focused around 2E migration/modernization, and general Plex consulting for a variety of clients - so we need a tool that will support all possible target platforms. 5250, C++, WebClient, Java, etc. Certify does this better than most (actually there are very few tools that do this), and it also supports scriptless testing. That means we can get QA folks up and running without a lot of training on scripts, programming, and all that sort of thing.[left]  [left] So it is probably not the tool for everyone, but it is worth a look.[left]  [left] Cheers,[left]

[Legacy User]

[left] John,[left]  [left] Thanks for the information.   I attended the Worksoft Certify session at the Plex/2E conference in Austin a couple of years ago.   At the time I had considered Certify more of a QA/acceptance testing option as opposed to a unit testing option, the difference being the point in development at which it comes into play.   I am currently looking at options for unit testing in an attempt to catch errors earlier on in the development cycle, before they get to QA.   Am I incorrect in thinking about Certify as more of a QA testing tool?   I am curious at what point in the development cycle you are introducing Certify.   Are your developers  using it to test on a granular, function by function basis or is it at a higher, feature by feature level?[left]  [left] I have begun an article on Continuous Integration on the Plex Wiki.   If you think Certify has a place in that space,  I would appreciate  any thoughts you might have incorporated into that article.   Time permitting of course :)[left]  [left] John Bell[left] Data Specialists, Inc.[left]

[Anon Anon]

John -[left]  [left] I would certainly argue that Certify is not just a QA or regression test tool. Yes, our developers use it extensively. Requirements are defined and test cases developed before the programs are actually developed in many cases, and then developers use it to validate their work before it gets to the QA folks. It does not do load testing / performance testing though - it is more of a functional approach. If load/performance is a requirement, then another tool should be used.[left]  [left] I am interested in seeing your wiki. Will check it out.[left]