Tuesday, October 21, 2008

Blame Everything on Testing

Your project is executing according to the schedule; the code development is done and unit tested, the Documentum applications have been installed and smoke tested; and UAT is about to begin. The designated tester on the team has been testing and writing scripts on the Test system. There’s a week allocated for testing the system in test. Oh, by the way, there’s a migration of legacy content being tested as well.

Yessss, I’ll be able to take my planned vacation
The testing is going excellent in Test. What do you think could go wrong? Well, first there are LDAP users in Prod and not in Test. Second, there are slight undocumented differences in the ACL’s between the two environments. Third, the migration will not be fully tested until it is run in the Prod environment. Fourth, UAT Users were not fully engaged during the testing and script writing phase of the project. Fifth, basic performance tests were run on Test, but real use tests to soak the system have not been executed. Sixth, a key developer is rolled off the project to start a new project at another company. Finally, seventh the sponsor of the project left the company and a new more technical sponsor is critiquing the application.

Can I get a refund for my vacation?
LDAP Users are in Production: The user that are imported will need to be added to groups if they aren’t from LDAP. The user logins will need to be tested. Renaming users will need to be tested.

ACLs: It is extremely tempting to make changes in Dev, QA and then Prod for ACLs manually without deploying them through a docapp or scripts each time. This inevitably causes slight differences between the environments.

Migration: When the migration is finally run in prod more focus will be on the results. The real users will be searching and finding bugs. Rerunning the migration a few times to prod should be built into the schedule.

UAT: No matter how much you tell the UAT testers that they need to be engaged and thinking about testing while reviewing the test scripts, they’ll find a ton of issues with the application during UAT. Showing them demos and prototypes during the development of the application helps, but they still won’t be able to really test until they sit down and seriously try to figure things out.

Performance: The Test environment was not tested for performance and there’s not enough money in the budget for a separate performance env. You’re left with trial by fire and it’ll be challenging.

Key Developer Rolls Off: This happens a lot. The Project Manager schedules a developer to be available for a few days after Unit testing. The best documentation of code and customizations could exist, but if the developer is no longer on the project, the learning is huge for someone else to fix bugs quickly during UAT.

New Sponsor: In this time of layoffs, you’ll get a number of sponsors who are reassigned of layed off. This means that the new person will come in with preconceived notions as to what needs to be done. This new person will want to make difference and a quick win is to delay the project while reviewing what it is suppose to do. What business problems are being solved, let’s look at the bigger picture here, why is this over budget, can we consolidate more, etc?

The testing phase of projects will give you a few gray hairs. There are so many potential pot holes. How ever many days are scheduled for UAT, double it no matter what. If you want to go on vacation after, schedule it for next year.