What is the QA Revolution?
So you are probably curious about the name of this website and want to learn more about it. I have been building this website for many years. My passion is software testing and I really enjoy learning as software testing is changing every year. I enjoy sharing my knowledge and passion with other people. I am hoping this website will be helpful for you and that you will learn something when you stop for a visit.
Many years ago, while I was completing my MBA, I began to learn the importance of marketing and how it could help transform the current testing organization that I was leading. I met with the team and discussed the idea and wanted to learn more about why we were struggling and what we needed to do to become a critical pillar within the IT organization. We talked about some of the mistakes we had made in missing some defects, but I also encouraged them that we needed to build a brand and market it outside our team. Everybody embraced it, and we came up with a team name and called ourselves The QA Revolution. I wanted the team to be able to hold their heads up high and be proud of the work that they had done. We built a quality brand and literally started QA Revolution! We created posters, created a logo, had signature lines for our email and developed websites that would help to build our brand. This was a tremendous hit, and because of that, suddenly being on the QA team was cool, and others wanted to come join the team. And believe it or not, they did. I am talking developers and engineers..the cool kids wanted to join the cooler organization.
I tell the story in order to build a point, that building a brand is important. It helps to define who you are and where you are going. It establishes a vision with a mission. These days people are looking for something to believe in that they can fully embrace. Those memories will live on forever, and I am looking forward to be able to reproduce that again and again.
Why is Quality Important?
Years ago I worked for a CIO who believed in Quality over Delivery. I was thrilled to hear that. It doesn’t often happen where a CIO is focused on Quality first. That approach and mindset went really far and everyone took quality very seriously. If there were issues, the products were not deployed into production until they were fixed. This was so refreshing that it made everyone work that much harder to deliver a Quality production into production on time. I more recently worked for a company that didn’t put as much emphasis on quality, and there were production issues across all teams, mainly because the focus was on delivery first.
Build a Quality Engineering Roadmap
It is important for the leader of the Quality Engineering team to create a Quality Engineering Roadmap. That is usually one of the first things that I do after performing an assessment of where things are in the organization. Building a roadmap is important because it sets the direction for the organization, and it is a great conversation starter across the organization. Ensuring software quality within an organization takes time, and it is important to be able track that progress. Once you create the initial roadmap, there is one thing for sure, that roadmap will evolve and change. It is important to keep that quality roadmap updated so that you will be able to see the progress. I recommend a roadmap that is no longer than 1 year. That is usually the longest time you can project out and be somewhat accurate.
The Agile methodology has turned traditional software testing organizations on its head. We have real world experience transforming multiple industries and successfully implementing best agile testing practices. Agile done well can have a tremendous impact. The agile product methodology is the one that I have successfully used at one of my most recent positions. It is important to leverage best practices when using agile. Developing software in short agile sprints with high quality production delivery can put your organization far ahead of your competition. There are many learning opportunities in reaching agile nirvana. It is a process and more of a marathon than a sprint. Some challenges I have learned along the way is there are agile mistakes along the way. It is fine to make mistakes since no organization is perfect. The important thing is to learn from your mistakes. From a testing perspective, things like end to end testing are often forgotten because they take too long. The result of that is poor quality, which is something that your business organization cannot afford.
DevOps has blurred the lines between development and testing. It has accelerated the pace of automated code deployments and given opportunities to integrate automated testing into the process. DevOps is simply the processes of collaboration between development and the operations team. While the goal is to make processes more efficient automating everything may not be the best answer. It has to be determined within each organization. The ultimate goal is to help your business win. If you have built a killer DevOps team, but don’t set your business up for success, you will have missed the goal entirely. I believe the DevOps practices are here to stay, but I have used these same techniques for many years before the word was invented. DevOps enables testers to spend more time testing and less time waiting for test environments to be setup. If you are looking for more information on DevOps, I have published an article that highly recommends reading The Phoenix Project and The DevOps Handbook which are both written by the DevOps Rockstar Gene Kim. I don’t like to read but I really enjoyed these two books. It is critical in invest time in learning DevOps. There are DevOps challenges that will happen as you standup the organization, so you need to be realistic and allocate enough time and resources to do it properly. Once you reach a steady state, then you can focus on continuous process improvements which will really get you to the next level and provide some significant DevOps benefits. DevOps is really very complementary enabling software testing transformation.
Test Automation has gone mainstream and all software testing organizations must do it to keep up with business demand. It requires extreme technical expertise and multiple toolsets are needed. Test automation is my passion and I have built multiple organizations and driven automation up to 95% automated. I believe all software testing organizations need a strong test automation practice. I have used both Selenium and Unified Functional Test to help deliver strong value in automating pretty much everything from mainframe to mid-range to web front ends. There needs to be a strong focus on automating what makes sense, and what will provide a good return on your investment, since it can be very expensive to maintain. Creation of an effective test automation framework is critical so that it can handle most of the workload. The beauty of an automation framework is that you can leverage it, instead of building thousands of scripts that will have to be individually coded.
Over the years, software testing has evolved. It has grown from some fantastic software testers to quality engineers. Manual testing has evolved for sure over the last 20 years into something that is more sophisticated. Sure, there will be manual testing forever, because everything does not need to be automated, but the days of having only manual testers are long gone. In order to keep up with the demand, you have to have highly skilled individuals with development type skills. Notice I said development type skills, not developers, because I firmly believe that a great developer may not transform into a great tester. The key is a mindset. Are you curious? Do you ask a lot of questions? Are you analytical? Do you like numbers? Can you see typos and other errors better than anyone else? Will you ask the difficult questions and challenge the status quo? These are some of the questions I ask when I talk to potential candidates. While having a stacked resume with skills, education, and certifications are truly impressive, they will not answer the questions above. A quality engineer needs to be able to seek, probe and find those edge test cases where nobody else dares to venture. Quality Engineering has somewhat merged with DevOps, so those lines are starting to cross. Those that can think, code, and solve complex problems are the quality engineers who will have long careers in this field. Are you one of those? Are you looking to build your career into a quality engineering expert?
In order to ensure quality is achieved for a given release, it is important to be able to track and measure the productivity of testing. Without an accurate measurement, it will be impossible to adequately answer that question. In fact, metrics is usually one of the first areas which I will start. You will first need to identify what types of metrics will be captured, I recommend starting simple. For example, you will want to track the number of planned test cases versus the number of actual test cases. You will also want to capture the number of defects and understand which defects and impeding progress of the test case execution. Once you have those 10-12 metrics in mind, you can start capturing those items for a few releases, and then begin to expand those metrics. You will want to evolve them and be able to drill down to any given metric. For example, you will want to start to measure how many test cases an individual is executing to see of there are individual opportunities for improvement. With metrics, you will want to be transparent across the organization. I usually publish a daily, weekly, monthly, and quarterly metrics report and share that across all the key stakeholders. This helps to keep them informed and allows critical decisions such as go/no go to be made. In addition, I have created a web based dashboard that will allow anyone in the organization to be able to see real time testing progress on any given project.
Performance engineering is one of the most important practices that should be a mandatory component of software testing. Unfortunately, it is one of the least followed. Perhaps it is done in some form or fashion in a lot of companies, but it lacks consistency and is often an afterthought and only completed at the very end when everything has been completed (which rarely happens). I am shocked at the amount of major companies that do not put enough budget and resources to get this critical task completed. I compare performance engineering to purchasing a life insurance policy. It isn’t necessarily required, but if it is not done, then there can literally be catastrophic issues that will occur. Often, code is put into production without running performance tests or adequately sizing the environment, which will usually result in having to back out the code due to degraded performance issues. Companies also are unwilling to size the lower test environments, so it is really hard to get a true test without a small environment which usually lacks a full instance of production data.
There is often a lack of understanding by those that manage the performance engineering team. This understanding usually results in lack of emphasis and lack of direction or vision. The performance team members will do the best they can, but they will often get discouraged and look for other places to work if they feel their work isn’t adding value. Agile performance testing throws another element in, and the performance team literally gets lost in that methodology. I always encourage my performance engineering team to stay engaged and test early and often so that they will have the opportunity to find issues which bring value to the team.
End to End Testing
In one of my previous roles, I led a test team that performed End to End software testing. Prior to that role, I had teams that performed the function of End to End, but I honestly never gave it much thought. The application stack that we were testing had hundreds of applications tied together. It was quite a mess. Changes in one application often may have an impact in downstream systems, especially when data is involved. In order to perform the end to end testing effectively, a broad understanding was needed of all the systems which were integrated. For our situation, we had ordering systems, provisioning systems, and billing systems that all had to work together. For example, without ordering a particular service for a customer, provisioning, nor billing would occur. End to End testing requires a significant amount of information and planning ahead of time, so that when it comes to performing the tests, things go as smoothly as possible. While the timeline in Agile will be different that using the Waterfall methodology, the basic principles remain the same. Usually End to End test cases are much longer and require more time to test, so while the number of test cases are smaller, it will take more time to perform those activities. In addition, if you are automating those End to End test cases, those will be more challenging to automate, primarily due to the data dependencies across applications.
These days it takes a solid understanding of software testing in order to understand what it takes to run a high performing QA organization. With so much pressure to deliver a high quality product to our business partners quickly, you need to be able to apply risked based techniques in order to be successful. This requires the ability to understand the technical landscape, understand the various flavors of software development lifecycles, know the capability of the development teams and deeply know the QA team and educate executives when it makes sense to deploy releases and when it doesn’t. In the ever evolving world of technology, one thing will remain constant and that is the need for software testing.
I am a huge fan of tools and automation and I encourage my teams to build automation to gain efficiencies. I am also a realist and understand that everything cannot be automated and effective risk based techniques must be applied to maximize productivity. Planning is one of the most important components of any testing which is done. The more time you spend upfront with planning, the more effective your testing will become and the less stress you will place on your teams when they are focused on testing during the final push to production implementation.
As a QA Leader, I have made many mistakes. I have led individuals and entire teams down the wrong path. I have also made the wrong choices, and that has impacted results in production. Those lessons were difficult and painful, but they were extremely valuable. Those mistakes enabled my teams and I to not make the same mistakes twice, and to be more successful as we evolve.
One of the most important components of software testing is to ensure your teams are properly trained. This QA training is critical to the overall success of the team and the quality deliverables they produce. I highly encourage allocating a certain percentage of time and budget for training. This training includes: testing techniques, software testing certifications, software testing tools, and software development lifecycles.
This training can include in person, online training, or self-paced training. Everyone learns differently, so it is important that you understand how your team members learn. Some might prefer self-paced, while others might require in person. Of course you won’t be able to please everyone, but if you can balance that if you are able to, the team will certainly appreciate it.
It is also important to be all inclusive with training on your team. You want to give everyone the same opportunity to learn. Some will gravitate towards training and embrace it, while others might need to be encouraged to participate. Training can also be a bit intimidating because there is a real fear of failure. For example, if you want to train functional resources on how to write and execute performance test scripts to someone who doesn’t have experience writing code, that can be a daunting experience. It is important when determining on what training is needed, that you understand the audience an their current skills.
Finally, training can be extremely rewarding and provide a high boost of confidence for those who are participating. If you want your team to become ISTQB certified, the team will need to study and pass an exam. That thought of having to pass instills some fear, but there is a great reward when the team is able to pass those exams.
I have built many years of knowledge on The QA Revolution website. I encourage you to look around and learn more. I am constantly adding new content that is relevant to the rapidly changing technology landscape that we currently live in.