The Special Challenges of
From the Winter 1996 issue of the Complexity Management Chronicles
Testing client server applications (c/s apps) intended for use in mission critical processing, such as payroll, presents special challenges. Based on our work with clients implementing these systems, this newsletter presents the major challenges so that you can be sure to address them in your testing.
We base all our testing on the following assumptions:
1.The architecture on which mission critical c/s apps is written has become extremely complex with multiple servers, LANs, WANs, operating systems, middleware, and third party software. As one client noted. "No application is an island anymore."
2.The complexity discussed above has caused the infrastructure to lack stability and robustness.
3.Most analyst/developers working in c/s lack critical experience in one aspect or another.
4.The organization form to support mission critical c/s apps has not matured.
More than ever, taking a "whole" product approach to testing, i.e. Total Quality Management (TQM), is imperative. What follows are major functional areas to include in your tests. We clearly do not have all the answers, but offer what solutions we can. The key is to surface the following functional areas in your test strategy and address them in your testing.
Performance: Be prepared for LAN/WAN performance problems. Do performance tests early and be sure to test the app in its ultimate destination. Do not assume that the techies have thought through performance issues across a network or that all LANs are the same. Recently, it took three minutes for our client to retrieve one transaction, even though they wanted an eight second retrieval! (See the 10/23/95 issue of Information Week, pg. 48, which compares LANs, or call us for a copy.)
Recovery/Synchronization: Identify as many failure and out of sync situations as possible, for example, LAN/Wan fails, server fails, disk fails, software changes on one server only. For any given failure, the speed at which your application can recover depends upon the specific backups, or redundancy, built into it. However, the more backups or redundancy built in, the more the system costs to build and maintain. Your job is to help the business strike an acceptable balance between recovery speed and cost.
Audit & Control: What reporting, monitoring, logging, notification, and control activities are in place to ensure that every transaction, and every hardware problem, is accounted for?
Vendors: Does the vendor's client-side software peacefully coexist on the desktop? Certain applications cause GPFs and can cause other apps running on the desktop to fail in strange ways. For more detail, call us. Is the vendor over-promoting? A third party vendor we know, who sells a bug reporting system, promoted their product as distributed. Once installed, our client discovered the product wasn't distributed and had to work around it. Later , in exasperation, our client switched vendors. How well documented are the error messages? See the book "Crossing the Chasm" by Geoffrey Moore, (HarperCollins Publishers) which discusses pioneering companies and product limitations.
Software Control: How will the client side software be controlled? How will installations be controlled? How will the software be kept synchronized across different servers? To help you with these questions, we have a great article which discusses the market leaders in network management software - contact our office for a copy.
Good luck with your c/s testing - the complexities make mainframe testing look simple in comparison!
©Complexity Management 1996
Complexity Management Chronicles, a newsletter for software quality assurance professionals, is published in print form four times a year. Send your name and snail-mail address to the e-mail address below if you would like to be on the mailing list - at no cost to USA mailing addresses.