If we look at the changes that technology brings us today, and with what organizations have to deal with in their development teams, we witness: big-data, cloud and mobility – which are the main factors for the complexity in many systems.

Those technologies, in combination to DevOps approach, which is catching up and evolving to include production in the scope of the basic development life cycle process, introduce many challenges to the testing world.

[Published at Testing Experience Magazine, October 2013]

What are we facing? In which areas should we prepare?

Testers no longer are faced only with technical challenges, but have to answer and bring solutions to: extremely high complex algorithms, data which is scattered throughout the world, legal issues of data in regards to the privacy act (especially when using cloud technology and distributed databases), and the fact that our phones (especially android versions) are facing high challenges in regards to security and privacy.

In light of the above challenges, I believe testing has to change. We shall have to get closer to requirements, in order for us to add more value to the business, introducing greater and more robust test design, integrated with risk based testing approaches, to leverage production. We shall have to introduce new tools to test things in the cloud, combining big data elements, and legal elements. Elasticity and performance and many other things (e.g. security) have to be addressed in the testing life cycle, methods, processes and tools, including automation approach and automation tools and platforms.

We shall have to use more and more modeling techniques, to be able to introduce robust model based testing into our automation, and to be able to do it fast, and efficient, as Agile approaches are getting more and more in demand, and common – starting to be implemented in corporates – on top of the technologies mentioned.

A pattern we are likely to see in multiple products, is that a big data cluster will be connected to the cloud, and end users will retrieve data from it on their laptops, tablets and mobile devices in a secure way, which will give us good performance, and reliable information. The information will arrive on time (high speed), and shall be location-based in many cases, or at least relevant to our personal profile (as individuals).

Using big data, is already introducing complex algorithms and new seek engines to DBs (like graphical engines), meaning a challenge to test (e.g. how can we verify that the data we got, coming from many places in the cloud scattered DBs, is the right one for the question/query we have requested). Securing this data over the air for mobile devices/tablets will demand new approaches/tools/mechanisms and higher speed from cellular operators, to be able to open the market for such application vendors.

In order to prepare for such challenges, we should be innovating new testing platforms (for automation), new tools (for cloud and big data investigation), and new processes/methods for developing test design using risk based testing as an integrated part of our life cycle.

As testers, we accumulate a lot of data on testing, on defects, on impacts, and many other important and critical items of information. I believe we should start looking at BI (business intelligence) tools for the testing data, so we shall be able to analyze it faster, and with more accuracy. It should be more immediately available for us in that way, and will enable us to investigate trends over projects, over time, and cross organization. BI tools are implemented today in many other fields, and their use is for the same scenarios I have just mentioned. Thinking of how to integrate BI tools into the help of testers and test managers might be found very useful in years to come, and enable us to cope not only with the technology complexity from development side, but also to handle the enormous amount of test data which is now being collected, and will be coming in higher amounts, when trying to solve the testing challenges introduced in this article.

The cloud and other technologies, also brings huge amounts of data, that we cannot (or will be not) able to simulate in our labs. If you combine that to the DevOps approach which is starting to catch up in the world that means that testing in production is a trend which will be forced on us, in order to cope. That will introduce tools and mechanisms that have to be enhanced and developed, together with code developed inside the production code to cope with that, identifying those data elements as test elements in production. We already have such patterns and mechanisms in production for the credit card companies, as they have developed such mechanisms for vendors wanting to integrate their systems to the credit card system, thus having to test things in real life scenarios. Today you can introduce a test-like credit card into the credit card company’s production system, and do all activities on that card, to test it.

Summary

Complex technologies and new development approaches that are being enhanced and introduced to the market today, will impact the testing methods, tools, processes, platforms, and skills required - a lot.

We should carefully pay attention to those trends, and be proactive in our training, in innovating new tools, platforms and processes to support them. We should discuss this w/w in forums and discussion groups, involving the development engineers to give us feedback, being closer to the business side, to learn what are our customers’ requirements and solutions that are needed, in order for us to develop and enhance those areas, and be ready on time to give an answer to those challenges from the testing and quality side.