I am a firm believer that end to end testing is critical in Agile. I have personally experienced countless instances of where production issues could have been avoided if a few end to end tests would have been executed. Most of my professional career has been devoted to software testing and end to end best practices so I have a high degree of understanding of the testing process. Testing is a critical piece of the agile process but it is often misunderstood.
Here are some items that will help you determine if end to end testing is needed:
Dependencies: For the code change that is being made, is it isolated to a specific product or does the change impact another product or service? If the answer is no, then no end to end testing is needed. If the answer is yes, then end to end testing is required. This is the #1 factor when determining if end to end testing is required and it is often the primary reason why issues occur in production. It is important that the agile teams communicate both internally and externally. Often an agile team will make a change and not believe there is an impact to another product or service but the impact occurs because certain misunderstandings or incorrect assumptions are made.
Critical Business Workflows: If a code change is made within a critical business workflow, then it is always a good idea to perform end to end testing to ensure no adverse impacts occur. Often running a few end to end tests will prevent issues from occurring in production.
Billing/Payments: It is critical to run end to end tests anytime there is a change to billing or payments. Billing and payments can impact all businesses and it is important to spend the extra time performing end to end tests. I have personally experienced many issues with companies due to billing and payment related issues.
Customer Impacting: If the change has the potential to impact the customer, then end to end testing should be completed.
This end to end testing framework is not exhaustive, but it should cover the critical areas where performance testing is required. Always if you think there is a remote potential it is a good idea to run end to end testing. Most companies have test automation that will cover the critical business scenarios so it is always a good idea to run those automated regression tests to ensure you are covered. Many in agile believe that end to end tests are a waste of time and not needed but I have experienced too many issues. I also am a firm believer that if then agile team who has made the change has access to either an upstream or downstream system, then they should be able to perform the testing so they are not dependent on another team to perform the validation. I believe that as long as you are running end to end tests that cover the code change, you don’t need to run hundreds of tests to ensure that things are working. Run enough end to end tests but don’t run more that is needed.