Author : Apptest.ai

As AI dominates the news headlines these days, it is worth pointing out the differences between AI capabilities. One of the earliest uses of a five-level classification system for AI came from the “self-driving car” industry. This way of distinguishing capabilities also serves as a road map for knowing how much more there is to achieve with AI.

In 2014, the Society of Automotive Engineers first published their “SAE J3106 Levels of Driving Automation”. There are six levels in total, but level zero is no automation.

Features at Levels 1 and 2 are now widely available, including on the author’s 2019 Rav 4 Hybrid: Lane centering and Adaptive Cruise Control at the same time. If I am momentarily distracted, the car keeps itself in the lane and slows down to remain at a safe distance behind the car ahead of me. But is levels 1 and 2, the driver still has to be ready and able to take the wheel. This is why Tesla’s famed “auto pilot”, with divers sleeping or playing games while driving, is not yet certified above level 2. Accidents have sadly proven that this automation is not yet ready to be called level 3 or 4. Human intervention is still needed.

In the wider world of AI, we can also apply this framework. Language generation has long been in the process of automation, from the “Autocorrect” feature on your smartphone to the recent explosion of Chat GPT (Level 4).

“OpenAI’s GPT-3 (Generative Pretrained Transformer 3), which powers this chatbot, is considered to be at Level 4 of AI automation, Autonomous AI Automation. GPT-3 is a language generation AI model that can generate human-like text based on input data and generate responses to questions and prompts. It operates independently and can make decisions based on its training data and algorithms. However, it still operates within the constraints and limitations set by its training data and algorithms, and it is not capable of truly autonomous decision making beyond its defined scope.” – Chat GPT Chatbot, February 6, 2023

Similarly, in test automation, there have been many advances over the years to the amount of automation possible. Apptest.ai is proud to be the first (and so far, the only) test automation provider at Level 4 – fully autonomous. Human interaction is not needed to ensure a successful, useful and well recorded test.

Level 1: Rule-Based Automation

This is the most basic level of automation, where pre-defined rules and algorithms are used to automate simple and repetitive tasks. These systems are limited in their functionality and decision-making capabilities. The simplest automations, like saving a screenshot of an error to a specific file, are Level 1 automations in mobile app testing.

Level 2: Self-Learning Automation

This level of automation involves systems that are able to learn from data and improve over time, but they still require human input to make decisions. For example, a system that learns to recognize patterns in test failures and can recommend solutions from a list of predefined options is a Level 2 automation.

Level 3: Limited AI Automation

This level of automation refers to systems that are capable of making decisions based on pre-defined parameters and guidelines, but they are still limited in their ability to operate outside of these boundaries. Most other test automation tools are at Level 3. They are able to execute tests randomly, but cannot determine which functions should be tested based on progress so far or the typical use of the app.

Level 4: Autonomous AI Automation

At this level, AI systems are able to operate independently and make decisions on their own. Apptest.ai is fully automated, there is no need for any human interaction, test scripting,  or other effort. The test uses machine learning and natural language processing to determine the best pathways for testing. 

Level 5: Augmented AI Automation

This is the highest level of AI automation, where AI systems are able to collaborate with humans and make decisions together. Considered the “holy grail” of all automation projects, this would be akin to the driverless car with no steering wheel and chatbots talking about whatever interests them.

As AI technology grows in sophistication and expands across more use cases, we are excited to see more examples of Level 4 automation. The strict definitions may vary slightly, but having the framework really helps technologists and users understand what the “AI” is capable of.