Exploratory Testing Framework

Introduction

Some of the common complaints about exploratory testing are that it is it needs subject matter expertise, it is not structured and it cannot be measured. But the fact is that exploratory testing is self-managed and self-structured. Exploratory testing is not a technique but an approach of exploring software without following documented steps.

Skills needed for exploratory testing

Exploratory testing can be more of skill based testing than a role based testing. Some of skills like questioning, observation, critical & lateral thinking will help an exploratory tester explore better. The ability to think beyond just a one sentence requirement and related test steps is part of the inherent skill which is a core need for exploratory testers. A well-executed exploratory testing rely on skill of the person. The tester uses his skills to approach the problem and controls the process. The tester continuously implements the feedback received from the previous test. So we can say that, exploratory testing is highly interactive and is an approach that allows in creating test designs when new information is discovered.

Structured exploratory testing

Some of the best practices during exploratory testing are to define test charters, test design heuristics and time boxing. These techniques will put some general guidelines around the testing being done and it gives a structure to define its success.

Test charters

Test charters give exploratory testing sessions a mission without being authoritarian.

Test charters outline:

• What areas of the system needs to be concentrated which is the mission scope
• When is the application ready to be used
• Responsibilities of the team doing the exploratory testing
• Duration of testing sessions
• The date and times of the testing
• The test environment being used to do the test
• Test data needed for testing

As the exploratory testing session progresses, charter should be updated to document observations or issues the team come across which can be used for further testing.

Test design heuristics

In today’s IT world where the functional complexities are always increasing and requirements always changing which makes launch applications to market even more challenging. It becomes imperative to think scientifically, use questions, explain the behavior, and based on the experiences gained from testing user predictions to create a test strategy. Scientific approach and using heuristics give the structure to exploratory testing.

The purpose of heuristics is to remind testers of what to think about when they are creating tests. Following are some pointers:

• Resources, constraints, and other elements in the project that may support or roadblock testing. Sometimes a tester must challenge constraints, and sometimes accept them.
• Software functionality is complex. All the corner cases to test should be covered. Cover all of it that matters, not just the parts that are easy to see.
• Quality criteria needs to be adopted that enables to determine if the application has problems.
• Test techniques are heuristics for creating tests. All techniques involve some sort of analysis of project environment, functional elements, and quality criteria.
• Observed quality is the result of testing. One can never know the “actual” quality of a software product, but through the application of a variety of tests, one can make an informed assessment of it.

Time boxing

Exploratory test session can be limited to a set timescale, a practice also known as time boxing. This will be a focused test effort of fixed duration. This can help concentrate on the project’s specific goals and scope, rather than drift into unfocused exploration.

Measuring efficiency

The efficiency of the exploratory testing can be measured by the time spent on the following:

• Test design and execution (T)
• Defect investigation & reporting (D)
• Configuration (C)

Analyze on the basis of the following criteria:

• More of ‘T’ indicates great code but might also mean poor defect identifying skills.
• More of ‘D’ might mean poor code quality but also suggest inefficient test reporting.
• More of ‘C’ reflects configuration and testability issues but it can also mean system is not ready to test yet.

The above method of measure learned from other practitioners is very helpful in calculating the efficiency.

Value of exploratory testing

The value of exploratory testing over other testing methods is

• Identifies complex defects in a system earlier
• Facilitates experimentation, discovery and learning
• Creates engagement by enabling people use their minds
• Provides user-oriented feedback to developers and business analysts

A special thank you to Moin Syed for writing this article.

If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.