There are several situations where test automation doesn’t make good financial sense. Let’s face it, test automation is great, and I am the greatest champion of test automation. If done effectively, it will result in tremendous Test Automation ROI and significantly increase the ability for the QA team to test a wide assortment of tests. The key is you have to be smart. Just because you can automate it, doesn’t mean that you should.
QA has become highly technical. The technical capabilities of teams today are far more advanced than they were 5 years ago. This has tremendous advantages but it can also be extremely dangerous. The same resources who can work circles around their predecessors are also the ones that are laser focused on proving their technical acumen and aim to prove their worth to their bosses and peers. One thing they forget: Test Automation ROI.
Test Automation requires resources and those resources cost money. All work should be closely monitored and justified. It has to make good financial sense and it should provide a high Test Automation ROI within a short period of time. The challenge is that when technology meets business the technical does not always translate to justification of automation.
I have been using automation since I started my testing career 20+ years ago. I have learned a lot and I have both automated and had teams that automated things that other previous teams had miserably failed. I constantly heard statements like : “you can’t automate that”, “we have already tried that”, “I can execute it faster than a script”, “this is a waste of time”. I have proved all them wrong.
I have seen a few situations where test automation simply doesn’t make sense. I have made these mistakes and I have learned.
- New Applications: It doesn’t make sense to automate brand new applications right from the start. With new applications, requirements are constantly changing and the UI goes through many iterations. Not only is this difficult to test from a manual perspective, it becomes impossible to keep up with the automated scripts. The best thing to do, is to give it time and come back once things have stabilized. You will be able to automate scripts a lot faster, and it will result in less frustration and a higher ROI.
- Frequent UI changes: If you are creating test automation scripts and the UI is constantly in flux, it will result in constant script updates and it often results in things becoming unmanageable. Every code build will result in automation scripts which are broken, and those will have to be updated. My best advice is to be patient and wait.
- Data Model updates: If your application is going through a lot of data model changes, it will result in UI changes which will have an impact on your scripts. These data model changes are often done early on, so if you give it sometime to settle, you can step in and automate those components.
- Stable Code: One of the most important factors is to have stable code. If the code isn’t stable, and there are a lot of defects and frequent code builds, this will often result in a high degree of test automation failures. It is crucial to let the code settle, and in some situations have less code delivery, in order to maximize test automation coverage. More builds don’t necessarily mean better applications.