Is Continuous Delivery still a hostage to (manual) testing?

In traditional software development, you spend weeks, even months, developing code. Then comes the dreaded “integration cycle” and finally the “testing” or “QA” phase. We all know the story. At first, nothing works. But what’s even more interesting is that nobody even expects it to work! It takes days and sometimes weeks before a stable, working build is available and QA can start testing.

Agile, with all its promises of delivering us from slow and error prone SDLC didn’t get us very far. An enterprise development lead described his team’s process as “Scrum Waterfall”. I have also heard “Water Scrumfall” and “Scrumfall”. The point he was making is that irrespective of how fast Development moves with Agile and/or CI , the mandatory testing phase drags the whole process back to the slow lane.

From what I have observed, there are 3 primary reasons why despite all the automation tools, we have been unable to accelerate testing:

  1. Lack of environments – several self service environment management tools have been available for a while but unfortunately they have done little to alleviate the real pain of environments – ability to closely mimic Production with all its configurations AND services (and this is an important but missing piece of the puzzle)
  2. Delivery of “untestable” code – Developers may be running 2 week Sprints to deliver components that make no sense unless other “driver” components are delivered. For example, delivering db schema (not data) that will be used later
  3. Reliance on “Manual Testing” during development – since the code changes every day (several times a day), current automated tools are ineffective unless presentation layer is locked down right in the beginning (and this almost never happens)

While we can improve on all three,  “Reliance on Manual Testing during development” is what any forward looking IT organization should be addressing right away.

Imagine a system where requirements are turned to business process, which in turn generates test cases automatically and then loads test data that can be generated and managed automatically too. Such a process can bring down the end to end feature/app testing cycle from several days/weeks to minutes!

This is not as far fetched as it sounds.Using current TDM tools (and there are several in the market) and integrating them with already available testing tools like Selenium and QTP, some customers have already realized the true potential of Continuous Delivery. They have (more or less) eliminated a separate test phase making every change deployable.  Even if you want to keep the UAT a separate phase, it is still super fast – lasting no more than a few days rather than months it used to take earlier!

So what do testers do? They spend their time on more productive tasks like exploratory testing.

The key takeaway here is that unless we automate testing end to end – including test case creation and update, test data generation, test execution and defect filing, CD will remain hostage to manual test cycles.  Changing the mindset from record-replay method of test development to more of a model-based system will go far in addressing this challenge.

Leave a Reply

Your email address will not be published. Required fields are marked *