It is extremely important to develop a strategic plan when creating test automation scripts. With the push towards Agile and DevOps the mindset has become to automate as much testing as possible. While that idea makes sense, it is important to determine how much test automation is needed. Part of the problem is that those who are managing Agile or DevOps teams have a really different mindset, and they really don’t understand basic testing principles. Instead of taking the time to understand, they force their understanding and push test engineering in the wrong direction.
Test Automation Planning
Test automation is certainly critical for Agile and DevOps. There are things that can and should be automated but before that is done, a few questions must be asked:
- How many times are these test automation scripts going to be executed?
- Is the code stable or is it going to change a lot over the next several sprints?
- Has the test been executed manually and has it passed successfully?
- How much time will it take to maintain the test scripts?
- Do you have the right resources to build and update the test scripts?
- What type of tools are you going to use?
- What type of framework are you going to build?
- How much does the test automation software cost?
These questions will help lay the foundation of your test automation planning. The next step will be to identify the number of test cases that need to be automated.
Testing That Should Not Be Automated
Here is a list of test cases that should not be automated:
- Test cases that will only be executed a few times
- Test cases that require human touch, for example review of a customer bill
- Test cases that require a third party billing system such as a payment gateway that doesn’t have a test environment
- Test cases were the code is unstable and is constantly changing
Test Case Quality
Test automation scripts help with providing test coverage across applications. This allows faster deployments because it doesn’t require manual testing. However, if you create hundreds of test cases and those never identify any defects, is it really worth maintaining and running those over and over again? Probably not. The test cases that are automated should be able to capture defects. Therefore, it is important that you have some really sharp manual testers who can think through the testing scenarios and identify problem areas that need attention. Without that system knowledge, you will not capture defects and it will not be a good use of your resources.
Return on Investment
It is really important to understand how much cost is involved in creating automated tests. If you create thousands of them, how much is it going to take to make changes on those when the GUI or back end systems change? If it is your department or team, as a manager or employee, you really need to be able to answer this challenging question. At every organization, I make sure that I calculate the ROI. There are a few ways to do this. First, you roughly need to know what is the cost of a member of your team or the average rate per contractor or employee. Most organizations will have that already. Second, you will need to determine how many scripts are going to be automated and how many times are they going to be executed. Third, you need to know how much time does it take to execute those test cases manually. Fourth, once the test cases are automated, you need to know how much time does each one take to execute. These items will at least give you a basic ROI. Most ROI numbers are somewhere between 6-12 months. Anything over a year, probably isn’t worth it. There are other variations that are more complex, but this will at least get you started. Knowing this information is certainly powerful, and it will help justify the expense for having automation engineers.
I cannot stress the importance of doing these basic steps. With Agile and DevOps, everything tends to be rushed. Perform this basic analysis first and spend a little time up front to yield large returns in the long run.
If you are a manual software tester, you have probably read or been informed by your boss that manual testing is slowly dying and that you need to upgrade your skills and become an automation engineer. I don’t think this can be further from the truth. While it is true that there is value in upgrading your skills and learning test automation, manual testing will always be a valuable skillset.
Why Manual Testing Isn’t Dead
- Manual testing can uncover some significant defects through exploratory or adhoc testing
- Test cases are typically built by the manual tester which requires analysis and a deep understanding of the application being tested
- If test cases are automated, they will typically first be created by the manual tester and then automated
- There are some test cases can’t be automated. Test cases that require analysis of bills for example shouldn’t be automated
- Some test cases shouldn’t be automated because it doesn’t provide a good return on investment
- A manual tester sees trends and typically knows where the defects are found
- When schedules are compressed, the manual tester brings value in prioritizing what is tested
Recently with things such as machine learning and AI tends to surface the question again. While machine learning and AI can be valuable additions, I don’t see the value of the manual tester decreasing anytime soon. Learn why manual testing isn’t dead. Many software testing organizations have some manual testing rockstars and they wouldn’t trade those valuable resources for anything.
If you are looking to learn more information about DevOps it will take you a while to figure out what are the top 2 DevOps books on the internet today. I am not a huge fan of reading, so when I read, I really like to enjoy the books I read. In addition, you want to make sure that you are getting the right information. There are so many thoughts around DevOps, and there is quite honestly a lot of misinterpretation on the subject, whether it is at a conference, on a blog, or written in books. You want to make sure you can understand DevOps best practices so that you can effectively implement those either as a consultant or an employee for the organization you are working for. As you probably already know DevOps is one of the key cornerstones of organizations that are looking to transform their development and operations processes. Anyone you talk with today is going to mention DevOps over and over again as a part of their IT transformation strategy. You want to make sure you understand and can actively contribute to the organizational objectives. Simply put, DevOps is DevOps is a software development methodology that combines software development with information technology operations. The goal of DevOps is to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.
Top 2 DevOps Books
Several years ago, I was at a conference, and I heard about a particular book that sounded really interesting. It isn’t your typically technology book because it is written as a fictional novel. Is is called The Phoenix Project. It really is a fantastic book and quick, easy read. The book helps you to identify with key challenges and issues that are facing most corporations from both a business and a technology perspective. Here is a high level summary of the book.
Bill, an IT manager at Parts Unlimited, has been tasked with taking on a project critical to the future of the business, code named Phoenix Project. But the project is massively over budget and behind schedule. The CEO demands Bill must fix the mess in ninety days or else Bill’s entire department will be outsourced.
With the help of a prospective board member and his mysterious philosophy of The Three Ways, Bill starts to see that IT work has more in common with a manufacturing plant work than he ever imagined. With the clock ticking, Bill must organize work flow streamline interdepartmental communications, and effectively serve the other business functions at Parts Unlimited.
In a fast-paced and entertaining style, three luminaries of the DevOps movement deliver a story that anyone who works in IT will recognize. Readers will not only learn how to improve their own IT organizations, they’ll never view IT the same way again.
It really helps to be able to understand the challenges that Bill faces, and the steps he and others take in order to overcome the daily struggles. The organization is constantly putting out fires, so they are unable to get ahead of these issues and develop a long term strategy. I have personally been in several organizations, and part of the reason why I really enjoyed the book was that I could identify with the challenges that the characters in the book were facing. The Phoenix Project authors (Gene Kim, Kevin Behr, George Spafford) got together and wrote the book leveraging the ideas and concepts from a prior book call The Goal, which I also recommend.
The second book which I highly recommend is The DevOps Handbook. This book is more of what you would typically expect from a technology book. It helps you to gain a much better understanding of what DevOps is, and what steps you can take to implement DevOps in your organization. The authors (Gene Kim, Jez Humble, Patrick Debois, John Willis) are all industry heavyweights and really have perfected the art of DevOps and are helping others to do so. Here is a high level summary of the book.
More than ever, the effective management of technology is critical for business competitiveness. For decades, technology leaders have struggled to balance agility, reliability, and security. The consequences of failure have never been greater―whether it’s the healthcare.gov debacle, cardholder data breaches, or missing the boat with Big Data in the cloud.
And yet, high performers using DevOps principles, such as Google, Amazon, Facebook, Etsy, and Netflix, are routinely and reliably deploying code into production hundreds, or even thousands, of times per day.
Following in the footsteps of The Phoenix Project, The DevOps Handbook shows leaders how to replicate these incredible outcomes, by showing how to integrate Product Management, Development, QA, IT Operations, and Information Security to elevate your company and win in the marketplace.
I think a combination of both of these books will help you gain a tremendous understanding of DevOps. After the initial reads, I keep both of these books handy, and I have read them at least a few times. In addition to that, I have recommended them multiple times to everyone who is interested in learning more about this software development practice.
The most important thing you can do your your career is to Build Your LinkedIn Network. It is getting harder to find your next opportunity. I have been focusing on building my LinkedIn network and there are some helpful hints that I have learned along the way. Here they are:
- Connect with people at your current job. Many years before LinkedIn, you had to keep up your previous co-workers through email or phone. LinkedIn has made it a lot easier. I make sure that I periodically add coworkers. It helps to keep things current. It is really important to add those individuals who you work with and trust because they know you the best.
- Connect with others who are industry experts. It is important to connect with others who are influential within your industry. They will help you learn more about your profession, and they might be able to help you land your next dream job.
- Limit your connections within LinkedIn. Currently there is a restriction of connecting with 30K members. Once you hit that limit, you will not be able to add any additional people within your network. This is somewhat limiting and many within the LinkedIn network are requesting this restriction is lifted. Once you have hit 30K, people will need to follow you. I made the mistake early to follow everyone and then I saw the LinkedIn limit. I am more cautious now, so I make sure that those who I connect with are relevant. I currently have over 24K connections.
- Follow LinkedIn Influential People: There are several people such as Oleg, Brigette, and Cory are really helping others in any way they possibly can. I have seen many instances, where their influence has helped to get people jobs. It is amazing to see the positive power and the influence that those are providing. I have followed then and plan to stay connected with them and I know I will learn a lot from them.
Hopefully this information has been helpful and will help you build your LinkedIn network. If you are looking for additional career tips please read 5 Steps to Increase Your Career Opportunities.