Migrating the test cases from unmanageable to manageable

For a large project over the time it is normally found that the test cases grow to a large extent especially release after releases, however often with the tight schedule of ongoing project delivery, the test cases are not maintained properly. Finally one fine morning you discover that the test cases become unmanageable. It has really become cumbersome now and drains a lot of effort to train a new joiner especially when test scripts are not maintained with enough comments / when purpose of writing a test case has not been captured in the test cases / when the functionality the test cases is trying to test is not mentioned clearly. The test cases were just added and added and added during releases based on the requirements without any thought of maintaining it for future.....

Hence it becomes difficult now to identify and pick up some test cases from the test case lot for the purpose of testing/regressing functionality xyz for Release xxxx (say).

Sound familiar in your project too!

So managing the test cases becomes an important task to make a testing team more effective.

The problem seems to be more severe when you discover that the higher management will not be willing to deploy resources for this kind of activities which were not done correctly in past and you have no other choice but to take up this activity of cleaning the test cases (which itself is a major effort itself ) along with your critical project delivery without hampering it a bit. After all you can not jeopardize your ongoing project delivery at any cost. Can you :-)

Here I have cited few of the ways what you can try in your project once you are at similar condition and you want to take up the task of cleaning and properly managing the test cases for future then on.

1. Prioritize test cases

First of all it is required to identify the Test cases as priority 1, 2 and 3 if it is not already done.
It is not at all possible or not feasible to take up all the test cases at one go along with your regular project work and clean up. So the best way is to prioritize them and take them up in phase wise.

Suppose 1000 test cases are there. e.g. Identify 200 odd test cases as priority 1, 400 as priority 2, 400 as priority 3. Prioritizing is the first and foremost task.

Priority 1 test cases are test cases which test the very basic functionality/core functionality of the product.

e.g. for a mobile handset testing test cases should include
Whether you can make a call/ receive a call, send and receive message, save a number etc,
dial the number and call, pick up a number from the saved list and call etc etc.

2. Categorize test cases

Also categorize you test cases in different folder such as

Negative testing
Stress testing
Load Testing
Functional testing
Usability testing

If for a particular release customer is expecting only enhanced look and feel of the product, then you don't need to break your head and search test cases at Stress/load testing test cases at all...

3. Brainstorming and finalization of the standards/approach

Give one week of time to your team members. Ask them to come out with the ideas how to approach on cleaning and maintaining the test cases better for future.

You could also charge them up by linking this activity with rewards, e.g. the person who adds maximum value will get 1000/- voucher etc etc..[Do not make mistake of deciding whose contribution is best by yourself. Ask your team only to decide. Best option would be deciding by voting system.]

Once all the ideas are submitted, brainstorm on those ideas as a team. Book a conference for 3-4 hours and brainstorm exhaustively. Take up an idea and see the applicability, pros and cons etc and finalize the approach. With those finalized ideas set a guideline on what are the approaches you will take while cleaning/managing the test case for future.

Modify 2-3 test cases according to those guidelines. Now sent the guidelines and the modified 2-3 test cases for review. Review should be done from the junior most engineer to senior most manager (from inside or outside the team - whomever you think can add value). Most importantly give it to a new joiner and see whether he/she can understands on his/her own i.e. whether the test cases are self explanatory ( i.e. the test cases are elaborated enough and captured enough info and the new joiner can understands on his/her own).

Receive your review comments. I am sure you need to do a lot of follow up to get your review comments!

Once you receive the review comments, modify the guidelines accordingly. Now set the baseline for the guideline.

4. Set the guideline as Bible for your team now on. (Whenever a new test cases being added this bible should be followed).

5. Take up the whole effort as a project in phase wise

Take up the whole effort of migrating the test cases from unmanageable to manageable as a complete project.

Divide the project effort in 3 phases. You can take up priority 1 test cases in phase 1 and try to implement the standards/guidelines first. That is, Try putting a small investment and see how the investment is paying off.

6. Resource utilization (keeping in mind the ongoing release activities)

You can try this model of utilizing the resources.

You can deploy 1-2 full resources in this Test case management project depending upon the bandwidth and the criticality of your ongoing project delivery activities (basically what makes sense based on your ongoing project delivery condition).

Rest of the resources who are completely deployed on ongoing project delivery, can be given a long term deadline e.g. every two months 5 test cases needs to be modified /person. Thus later point of time you may find out that a significant effort can be absorbed along with the ongoing projects and also thus you can inculcate a sense of responsibility of maintaining the test cases properly by all the team members in your team.

7. Educating new joiners

Modify 1-2 test cases with max comments/info.
Explain the flow of the code to make them understand and record your explanation.
Give the recorded file to the new joiners and see whether they are facing any difficulties in understanding those on their own.

Thus you can save significant time of the existing team members who do not repeatedly explain the test cases every time a new member joins the team.

Also record how your test cases are segregated and how you are managing your test cases in the recorded file so that the new joiner need not ask for any kind of info from the existing team members.


8. Few ideas (subject to applicability to your project)

a. Capture as much info as necessary in the test script
who has written
what purpose it is done
CQ id
Requirement ID
functionality - what it is going to test etc etc

b. Create reusable modules / scripts - thus managing them would be easier and also writing new script would be easier

c. Version control might help; you can go back to previous version and can run if the necessity arises

d. Always keep an eye how you are developing the test script /test cases and how you can reuse in future (the already existing scripts)

e. Give proper access control for maintaining the test cases.
who will have read permission
who will have read/write permission
who can add new test cases etc etc.

f. Discard not relevant test cases / invalid test cases

g. Look back after each release and clean the system. Transfer not relevant test cases to an obsolete folder

h. Do a cost benefit analysis before diving into changing the whole suite.

i. Map Requirement with Test cases
You will be able to know for a particular requirement how many test cases are there at a glance. Also based on the requirement you can pick up the relevant test cases

j. Keep executed Test cases release wise in Quality Center - Test Lab Tab.

k. Collaborate with developers for minimum user interface change in order to minimize the rework effort of test case/ script change (of course without compromising with quality)

l. Last but not the least; Put a process in place for adding new test case. Say one person is in charge of reviewing the new test cases. i.e. the newly added test case needs to be reviewed and accepted by the person before it gets added into the Quality center which will ensure standards/guidelines are followed correctly till the process gets streamlined by all the team members.

Else who knows , you might end up doing the same kind of cleaning activity again in near future !