Nowadays test automation is a must, yet lots of companies are struggling with automation. Test automation is especially difficult during short sprints. According to a recent Capgemini case study, only 22% of the test cases are automated during sprints. The cause is that test automation starts when the implementation is ready and test automation is a time-consuming task.
We also faced the problem of slow test automation. Finally, we realized that we should do test automation in two steps:
1. First, we should do the implementation-independent part of test automation that is test design. We use a new method called action-state testing. Here abstract test steps are created resulting in a model. From the model abstract tests are generated that can be executed by the tester, but not a machine. This should happen before the implementation.
2. Creating the test cases based on the action-state model. The model guides the testers to create executable test cases in a very short time.
Action-state testing is introduced in the book Paradigm Shift in Software Testing will be published soon. This is a step-by-step test design technique where abstract test steps are created one by one from a given state, supported by an algorithm. Instead of making a whole model, it is made gradually from a starting point that is the initial state. In our agile world, making a whole model is usually not possible as the features are selected and implemented gradually. Action-state testing conforms with this concept. In action-state testing, we start from an initial state and investigate which actions can be done. Then we select an action such as ‘add a coke to the cart’ and investigate the system’s response and the arriving state when this action has been finished. The response can be ‘two cokes in the cart’, the state can be ‘total cost is below to pay’. According to this our first abstract test step is: (‘add coke to the cart’, ‘two cokes in the cart’, ‘total cost is below to pay’). Then we add another step from the initial state or from the state ‘total cost is below to pay’.
Action-state testing is an implementation-independent test first method. You can do it at the beginning of each sprint. This is the larger part of the whole test design + automation process, thus the test cases can be automated in-sprint.
In an abstract step, the user actions are the inputs, responses are the outputs to be validated and the states are the cornerstones showing the subsequent steps. The only requirement for such an abstract test step is that the tester be able to execute the step and validate the result. Our step: (‘add coke to the cart’, ‘two cokes in the cart’, ‘total cost is below to pay’) is an appropriate step as the tester can add a coke, validate the cart to contains two cokes independently from any implementation (if not, then this is an issue to be fixed).
In action-state testing, we build a model by adding abstract test steps. We always start from a state and investigate the possible actions from here. By doing the action, the systems responses can be validated. After the system responses, it arrives at a state that can be the same as we started. Creating abstract test steps, we also build a state transition graph. This graph helps you to know which are the missing actions if any. If we start from an initial state
INITIAL STATE state 0
and the first step is
first action => first response STATE state 1
then in the background, we get the following state transition diagram:
You can display the current graph after adding a new step. The graph can be used to realize any missing action or state. For example, if there is an action from state 1 that arrives also state 1, you immediately realize that this action is missing. Adding the following step:
second action => second response STATE state 1
the graph contains an additional edge labeled by ‘second action’:
You should add new steps until the state transition graph contains all the available actions and test states. This is the first phase of action-state testing. Note that the first phase contains all the basic tests that cannot be ignored in any case. This is the minimum requirement for testing any application.
When the first phase is ready, you can start the second. In this phase, Harmony offers the necessary steps to satisfy the build-in test selection criterion. Currently, one test selection criterion is in Harmony that is the all-transition-pair criterion that requires that each adjacent transition pair be covered.
Harmony marks the steps that may be extended by child steps. For example, the well-known ATM authentication feature has the following model after the first phase:
The marked steps may have child steps. You should investigate each offered step and either accept or reject it. When all the steps for which Harmony offered child steps are ready (✓), you are ready. This means that the generated test cases satisfy the all-transition pairs test selection criterion. As a consequence, the tests will detect lots of tricky bugs.
It’s very important that making the first step is a must, but making the second step is very fast. It takes much less time than fixing the tricky bugs some months later. Completing the second phase doesn’t require any step creation or test execution automation only checking the feasibility. Therefore, we strongly suggest doing it.