Test Planning requires careful consideration and it takes time to do it right in order to ensure proper test coverage. Test organizations that don’t perform proper test planning often rush to test execution with little to no regard to proper test coverage. The result often has devastating results and has significant long term impacts. Lets review some issue which will occur as a result of poor planning.
Insufficient Test Coverage: Rushing to test execution without proper test planning will result in insufficient test coverage
Too many Test Cases: Lack of planning will result in testers creating large amounts of test cases and duplication of testing.
Large Regression Suites: Testers will take the test cases and move all of them into a regression suite and it will result in requiring extra time to run the test cases which have been duplicated
Redundant Automation Scripts: If the regression suites above are automated, it will result in a maintenance nightmare for the automation team which will require additional time and money to resolve.
Proper test coverage requires significant test planning time. The more time spent in test planning will result in more efficient and reduced test execution time. In general if you spend 3 to 4 times more in planning that test execution, you will have a meaningful result during test execution. With the emphasis on Agile and Waterfall methodologies becoming more efficient, it is more important now than ever for testers to really think how something should be tested and have great test cases. I prefer my teams have less test cases that are more efficient versus having many test cases that result in duplicate validation. Testers often start out by thinking the more test cases that are created and executed the better testers they are versus their peers. This is simply not true. Efficient test coverage helps everyone and it reduces the amount of time that is spent testing. With teams becoming more product focused, it is important to test the product you are working on and have other teams validate their products.
I hope this information has been helpful in understanding how to have efficient test coverage.
The product model is rapidly gaining traction within the software development lifecycle. Companies continue to look at options to improve the software development delivery process. I am seeing more and more organizations move in this direction. The challenge involved is determining how software testing will play a role in this process.
The product model leverages the agile methodology to help develop a product or series of products. Typically the teams are 6-8 people and they develop solutions in short iterations (Sprints) and deploy small code changes into production on a frequent basis (2-4 weeks). The team typically will have business analysts, development, and testing. The numbers will vary based upon the needs within the product team. I see the product model working well in companies that are small to mid-sized and have a relatively lean IT organization. In addition, I believe those companies that have a few products will be able to perform the best. If you have a lot of products and those products interface with each other, you will have significant challenges with integration and moving data from one system to another. This will require extreme collaboration to ensure interfaces and data are provided when they are needed, otherwise the release going to into production could potentially get delayed.
The product model team is typically driven by a product manager or product owner. That individual is responsible for providing direction and helps to remove obstacles. It is important that the team works well together so they can increase their velocity in terms of software products. If there is one or two team members that aren’t pulling their weight, it could affect the entire team. Either those team members improve or they will need to be replaced with those who have the proper skills. The team also must be self-sufficient and they need to do everything possible not to be dependent on other organizations either inside or outside the organization. Certain pieces such as infrastructure setup might be outside the team but it requires constant collaboration to make things happen. If there are multiple products, those teams will usually roll up to a program manager.
In terms of software testing within the product model, it will involve functional, automation, and performance. Some software testing organizations may differ in terms of test automation and performance activities. I have setup a shared service where the functional testers will reach out to automation and performance resources who can build and execute automation and performance tests. I have also seen examples where the automation and performance activities are accomplished by the testers within the team. As long as those activities get done, it should work well either way.
Software testing has been rapidly growing over the past several years and here are some software testing trends to be aware of in 2017.
More Open Source Tools-As more pressure is placed on reducing expenses, more software testing teams will look at alternative options in reducing software testing licensing. Adoption of open source tools will continue as Agile and DevOps practices evolve. This software testing trend will continue for the next several years.
Test Automation Growth-Most software testing organizations already have some test automation practice in place, however, there will be a focused effort to increase the amount of automation that is in place. Companies will continue to focus test automation in the areas of smoke testing and regression testing. Where companies can, they will use automation tools such as Selenium and Appium.
Performance Engineering-There will be additional attention in the area of performance engineering due to additional data demands and production level failures related to system performance. Companies will begin to require performance testing to be completed prior to production implementation.
Digital Transformation-More attention will be place on customer satisfaction and this will require a digital transformation with the customer workflow. This will require extensive amounts of software testing to ensure that the customer has a positive experience. This transformation will go across multiple systems and require security in terms of the customer data. In most cases there will be a mobile component to the digital transformation.
Big Data-Explosion of demand for data due to IoT devices and additional data in order to make better business decisions will require new technology to support the additional capacity. Software testers will need to be able to learn more about Big Data and will need to verify that the data is correct before it is consumed. Platforms such as Hadoop will need to be learned. This will require a solid test data management practice.
Cloud-Continue cloud adoption will grow as more traditional companies such as insurance and finance begin to slowly transform their business to the cloud. This transformation will continue to evolve for the next several years. Software testers will need to understand more about public, private and hybrid cloud solutions. Software testing companies will also look for software testing platforms to help them accelerate software testing.
Agile/DevOps-Most companies will have Agile as the default software development lifecycle. As Agile continues to take shape, companies will begin to focus on building DevOps practices. This will also include Continuous Development and Continuous Integration. The DevOps approach will help to build collaboration across teams and help increase the amount of automation used in developing and deploying software. In addition, most companies will start to integrate Security as a part of their DevOps practice.
I hope this list helps. It will be interesting to see where software testing heads as we look at 2017 and beyond.
It is important that your QA organization has the necessary items to be successful. Perhaps the most important element is the organization is able to protect the production environment. Here are a few items that are required.
Ensure you have Executive level support. This is the most important items. Without having your executive to support the QA organization when there is a quality concern, delivery will always win.
Reduce the testing cycle time. Another important item, is to make sure your organization is doing what it can to reduce the amount of time spent during test execution. The best thing that can be done is to focus automating your regression suite.
Shift Left. Focus more time getting involved up front and identifying issues with requirements and other documentation.
Focus on Planning. Make sure the QA organization is covering all the bases and spending sufficient time planning.
If you are in QA, I will personally guarantee you will have to justify why testing is needed. It should be a simple answer. Humans make mistakes therefore developers = humans and developers = defects. I don’t blame developers at all because I am a tester and I make mistakes. However, I find that developers don’t get asked to justify their existence. In my humble opinion, developers should have to justify their existence and they should be evaluated in the same way testing departments are. All teams including developers should provide metrics that justify their existence.
QA is usually one of the few organizations that capture metrics and this can often backfire on QA teams. With automation for example, we capture ROI to justify the spend required for the budget to create automation. Test automation is essentially development so why is that required. Anyone who knows the least bit about automating anything can make a solid business case that automation is a no brainer. However, we are forced to justify the need to spend money on developing code.
With testing misses making more mainstream news, such as the Obamacare website, hopefully IT executives will understand the need for testing and shift gears to ask other teams to justify their departments instead.
Test Planning requires careful consideration and it takes time to do it right in order to ensure proper test coverage. Test organizations that don’t perform proper test planning often rush to test execution with little to no regard to proper test coverage. The result often has devastating results and has significant long term impacts. Lets review some issue which will occur as a result of poor planning.
