Apr 03 2008
Your Software Is Only as Good as the Testing You Do
In the process of designing and managing the development of several web-based applications, your correspondent has come to learn a critical lesson in software development (that many others in the software industry likely already know):
Your software is only as good as the testing you perform.
Your correspondent worked on the team that recently released an online chat tool called University WebChat. Powered by jQuery, University WebChat enables college and university admissions recruiters to easily host web chats for talking with prospective students.
The Bug
The application was working great during beta testing, except when more than 20 chatters joined a room. Once we hit north of 20 chatters, odd things started to happen: chatters got booted from the room, some couldn’t join the room at all, others just got an endless connection waiting when trying to load the room URL.
But none of these issues happened consistently, only consistently intermittently. We couldn’t regularly reproduce the issue, so it was impossible to fix. We focused on other bugs.
A few weeks went by and we still had no luck in finding the cause of the issue. Then we decided to really test out the application by hosting a high-profile chat where we expected more than 75 chatters.
What a difference a little pressure makes. We were forced to take a different approach to debugging the issue. So we ran a more complex test on the bug, and dug a bit into how Apache works (it’s a LAMP application). Eventually we found the source of the bug, fixed the issue and felt confident that the application would support 80 chatters in one room.
Test, Debug, Refine and Repeat
If we hadn’t been forced to test the application harder then the true source of the bug would never have been identified, the software wouldn’t scale as well as we needed it too and (worst of all) customers would have had problems with our software.
Your software won’t get better on its own – you have to force it to become better with great developers, useful features and most of all hard testing. Over and over again. From different angles and vectors. From different computers, networks, browsers, operating systems, locations, monitors and processors.
This isn’t a new revelation, granted. But an important one.
Missing features axiom #7: Your software is only as good as the testing you perform.
What about Drupal? Huge community and thousands of contributing developers and many users testing and refining the CMS.