Author : Apptest.ai

Quality assurance is all about making sure the product works as designed and planned. Just like any decision-making, QA inevitably evolved around two approaches: top-down and bottom-up. For instance, in case of shopping, the top-down approach is materialized through a search: the customers first decide what to buy and then searches for what he/she needs such as “jacket” or “socks.” The bottom-up approach, on the other hand, involves being exposed to many choices first, and be inspired to know better about what he/she likes and dislikes, and the final decision goes to the best one. Again for a shopping experience, he/she will have a chance to browse various options like a bomber jacket, a suit jacket, a raincoat, and purchase the one that will serve the need the best.

In software testing world, the top-down approach is often called as scenario testing or functional testing, where the testers know exactly what to test and run as many scenarios that he/she can think of and manage the process as effectively as possible. It has become the most widely adopted approach for testers today. The second approach, which is bottom-up, is called exploratory testing and is a lesser-known practice. Many attempts were made to actualize the bottom-up approach, and a minor success has been achieved through random or monkey testing. The challenge has been due to the lack of a method to present a combination of many choices as effectively as possible.

If we were to imagine a perfect world where the two approaches have no barriers or technical difficulties in implementing, both QA methods will arrive at the same outcome, and let me explain why, but starting from our shopping example first. For a shopper to be able to purchase what he needs, with the top-down approach, the search should be clearly defined with many details as possible. For instance, it’d be “Hugo Boss Men’s Active Series Herringbone Classic-Fit Suit Separate Jacket which is less than $1,000 but more expensive than $500 with blue, or navy color of size 40R that is most suitable for winter and lightweight at the same time which is currently on sale”. It’d require laying out precisely what you want to “command” to be able to arrive at the option as quickly as possible. On the other hand, with the bottom-up approach, the shopper has to be presented with tons of choices until he’s ultimately happy to make a purchase decision. It can take over thousands of options for him to compare and contrast until one outstanding option resonates with him that would lead to a final purchase. One requires infinitely many words that will require many thoughts and should be stated as precisely as possible. The other requires infinitely many examples for one perfect choice. Then, how does this aspect translate to the QA world? Either infinitely many scenarios have to be baked into every testing to ensure 100% quality; or, all the screens and pages, I mean 100% of them with every possible sequence that can lead to each screen, should be navigated to ensure 100% quality.

It looks like both suffer from either lack of the ability to map out all the possible scenarios, or the lack of the ability to step into all the combinations of user journeys.

Although, it’s worthwhile to point out that the top-down is the approach that has gained tremendous momentum and seems to be winning. Adoption for the bottom-up approach has never been significant; and many QA roles, as well as many software tools, thrive on being able to reliably test all the scenarios as fast and effectively as possible. How can we explain the discrepancy in the adoption?

It’s simple. The human brain is wired to seek rewards and instant gratifications. We ask questions, we share what we desire, we google, we get what we want. Even if we can’t get to what we want in the end, we know that what we expressed that we wish to is not available. How we have exposed to myriads of options are rather gradual, because we can’t scan all of them at once, but we need time to learn to finally come to pick the one that matches near to what we wanted in the first place. In other words, we don’t have a processing power to scan them all in a short amount of time. That said, not all of us know exactly what we want 100% of the time, so that’s why it’s a combination of top-down and bottom-up approaches altogether that makes us come to a final decision. Notwithstanding the limitation of a bottom-up approach, there are recommendation engines that so many of the e-commerce sites or movie content sites present choices that expedite the process for us to reach decisions.

Oh, wait. It sounds like we humans already deligated what we cannot achieve to the area of machine learning. Now, turning our attention back to the QA world, what if a bot exists and navigates as many user scenarios as possible and checks the quality as it navigates? That’s what exploratory testing meets machine learning, where it complements what human cannot process simultaneously. Artificial intelligence is deployed to automatically log in, prioritize what the next step such as click, back, filling-in, scroll, and search. It’s a tedious part of the mundane QA tasks to make sure all the user paths are error-proof, but essential in improving the satisfaction score of the users. The prior form of random testing or monkey testing now has a richer context than ever before.

It’s not only bug discovery that exploratory testing with AI can deliver. All in all, there are also bonus to the business impact, which can be highlighted as:

• 50% cost saving 
• 400% increase in release frequency
• 300% increase in testing capacity
• 10% increase in customer retention
• 10% increase in conversion rate

In the next article, we’ll take a detailed look at why that is, and you’d be the judge if a mere exploratory testing approach now has a full-fledged wing thanks to artificial intelligence.