Test Automation Strategy for IoT Applications
Software QS TAG 2017 Conference Report, Frankfurt
Got to hear Felix Christy @ the Software QS TAG conference (one of the biggest conferences for testing and quality assurance in Europe, organized by imbus, holding 500 people in 2017), and sharing here some lessons learned.
The message was the strategy, process and critical aspects which we should take into consideration when approaching IoT application testing. What I liked a lot about the approach Felix presented, was that the whole time testing was involving the business, asking questions, figuring out what is important for them, and implementing those aspects into the test strategy. I believe too many times I have witnessed that testing teams are not that close to the business, thus not focusing enough on the right/important things of the applications, and adding less value they might have been.
The presentation starts explaining the test strategy used - finding key factors (the business wanted and the application is targeting), collecting test data, developing test cases, designing and developing test suites.
After which, Felix explained the business compelling event - transporting goods in a truck, for long distances, where altitude, temperature and heat (of the container) were a big factor for keeping the goods in a preserved situation.
Walking us through the key factors, he identified challenges of the truck and transportation - outside temperature affects the truck container temperature, altitude affects that as well, vehicle heat as well. solving that via collecting monthly historical data on temperatures w/w, recording min-max temperatures, and adding location based sampling - mostly using existing internet collected existing data! impressive.
After that data collection, they could have answered a more complex questions, like: "measure the change in temperature of the container for a truck, traveling from Berlin to Lisbon in the month of January with an average speed of 60 Kph", and continuing to design test cases.
The analysis of business needs, with the data collected, allowed to design test cases for a few critical needed topics: cross country transport test suite, high altitude transport test suite, multiple transport comparison test suite and temperature season test suite.
The results were encouraging and proving the business case. the team managed to use extensive test data (geo based, weather based), simulate real world scenarios (extending their test suites further more), and get things done!
Lessons learned are that the mentioned process is structured right (for their challenge, but my experience is that it would fit others very nicely), the strategy is correct - investigating sensors, collecting real data as much as possible, and being able to extend the business requirements for testing in near real measures, and above all - being closer to the business while defining the testing challenges, and involving them while implementing test design and running tests was proven to be very much beneficial.