by Syed Moin | Nov 19, 2015 | DevOps
Introduction
Lately there is a huge traction towards DevOps in the IT world. Many companies have adopted agile processes in pursuit of faster releases. So the logical next step to this is DevOps. So now every other organization wants to jump onto the DevOps band wagon. If we study the DevOps landscape, testing is a critical part of the handoff between development and IT operations. So testing should be continually improved for the success of agile teams using DevOps practices.
Continuous testing in DevOps
Every one of us who understand DevOps are familiar with Continuous Integration and Continuous Delivery. But the most important bridge that keeps these two entities to work effectively and which makes the concept of DevOps successful is ‘Continuous Testing’. It is not just automation as most people think. Continuous Testing goes beyond automation and encompasses all practices — including tooling and cultural change. It is mechanism of continuous validation that keeps the DevOps engine moving forward. It provides continuous feedback to the project teams to guide them towards successful software implementation.
Continuous testing in the context of DevOps should take a test-first approach, which will help measure test coverage to check that software features work as expected. It will enable to find if changes break existing test suites. And also help the project teams to focus on what they need to deliver.
As part of continuous testing, test teams can also benefit with ‘Exploratory Testing’ by simultaneously understanding the system, designing the tests and executing the tests. This will add value to the test automation by actively exploring the system without a script to learn how the system behaves in unanticipated ways.
Continuous test improvements
Test improvements are a critical part of continuous testing. They drive the continuous effort to optimize test performance of the software. This should and will provide improvements through automation and reuse of best practices. The continuous test improvement can set thresholds for each test phase that determine whether failures are significant enough to revert the changes or it is necessary to stop the DevOps cycles long enough to fix the problems. Test first driven development, behavior driven development concepts allow test processes to improve in a DevOps Model. Some of the key areas that can be addressed as part of continuous test improvements are code analysis and test optimization & virtualization.
While test automation acts as a driver for detecting software defects, the concept of continuous testing highlights process improvements that can prevent future defects from occurring. Test improvements enhance real-time assessments and continuous measurements to refine the development process so that business expectations are continuously met.
Conclusion
Continuous testing is a great complement to the DevOps movement. Organizations cannot be successful with DevOps if they cannot improve the existing processes. They should know how and what to improve. Continuous testing which acts as an enabler to test improvements can be a good medium through which success is achieved. What is more important is that the ideas of test improvement should be embedded into the DevOps culture. As I said in my earlier posts, quality is built into the entire system with DevOps. When implemented the right way, the concept of continuous testing in DevOps will be more than a method of enforcing test automation tools or best practices.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Nov 9, 2015 | Agile, Blog, DevOps, Software Testing
There are many DevOps benefits. Here are a few:
- Improved quality of software deployments. Developing and testing more frequently produces a much improved quality product.
- Frequent software releases. Having more frequent software releases gives the business what they need to be successful.
- Faster business response. Times are changing rapidly and that requires businesses to adjust quickly to their competitors.
- More Agile development. Since there has been a big push towards Agile, there needs to be a greater amount of collaboration between teams and requires continuous delivery of code into production.
- Improved visibility into IT. IT is notorious for not keeping the business informed on what is going on within IT. DevOps helps to improve this communication and provides an additional layer of transparency that adds value to any organization.
- Greater collaboration. Perhaps one of the best DevOps benefits is the amount of collaboration that Agile brings to the table. Product owners, developers and testers are working closer together and producing much better results than ever before.
As DevOps evolves there will be more benefits in terms of efficiency that will occur. High quality implementations along with more frequent code deployments will allow companies to provide more for their customers. It will also allow companies to try things that may not work and give them flexibility to grow and expand.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Syed Moin | Nov 9, 2015 | Agile, Blog, DevOps, Software Testing
Introduction
Aristotle said, “Quality is not an act, it is habit. What we can interpret is that quality cannot be a response to a certain situation or something that is faked or postured. Rather quality should be part of a routine; the acts we do on a continuous basis and which should be a reflection of the essential nature of work. This is how today’s IT world is looking at the QA practice.
In a traditional IT world, testing is often seen as separate from the development. They were two distinct entities separated by ideological walls. Thanks to the new trends in IT world, these walls are fast disappearing. One of the major driver for this transformation is “DevOps”.
DevOps
So what is DevOps? It is not a methodology or a suite of tools as many people think. DevOps is a culture that enables seamless coordination among the different stakeholders in the software chain. DevOps brings together the two critical silos of software process namely, development & operations to integrate within the same software cycle.
DevOps drives all the IT stakeholders to collaborate to deliver value faster. For this it collates the practices of continuous integration, continuous testing, continuous deployment and continuous monitoring.
DevTestOps
As we know DevOps aims to break the barriers between Dev and Ops. It also demands breaking the barriers between Dev, Test and Ops. Testing is a critical part of the handoff between development and IT operations. For DevOps to be successful it is mandatory that testing should be fully integrated into the software development and delivery.
With this mindset, everything changes for a traditional testing organization. Testing operations undergo a major changes in a DevOps culture. This mandates a shift in tester’s skillset, methods, tools, timing and approach. DevOps advocates starting test early and constantly.
Some tips for successful DevOps testing:
- Involve development team during test design.
- Identify the critical test cases and prepare the test suite to test the functionality of the required build.
- All the environments required for testing need to be standardized and deployments have to be automated.
- Adopted test-first approaches such as test-driven development (TDD) and behavior-driven development (BDD).
- Keep the test execution cycles short and quick.
- Adopt regression test automation to run across various environments.
- Implement code analysis and coverage tools to ensure 100% code coverage.
- Critical bugs found need to be reported, fixed and passed through the same chain of events before the code is deployed on to the production environment.
- Encourage parallel execution of tests to help reduce time for go live which is the heart of a successful DevOps implementation.
Testing maturity is a key differentiator for the success of DevOps. Even if organizations can automate their integrations, builds and delivery processes but they still struggle to manage test orchestration and automation. Testing brains play a critical role to achieve this with their expertise in test design, test automation and test case development with DevOps. Irrespective of what DevOps processes, models and tools organizations use, testing is a vital part of the overall DevOps process — not only to ensure code changes work as expected and integrate well — but to ensure the requirement changes do break the functionality.
DevOps Testing Tools
In order to realize the desired business goals of DevOps, it is essential to have right tools to maintain the quality associated with the delivery chain. This calls for accurate and comprehensive testing. So it becomes very important to understand the existing testing process and identify the right testing tools.
Some recommendations of testing tools are below:
- Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including Java, C#, Groovy, Perl, PHP, Python and Ruby.
- XL TestView is a test management and analysis tool that allows you to define and execute tests across your full spectrum of test tool suite. It analyzes test results across multiple test tools and track release metrics and quality trends over time. It helps aggregate and visualize results to provide true insight into quality.
- Cucumber is a testing tool that computer programmers use for testing. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Cucumber is written in the Ruby programming language. Cucumber projects are available for other platforms beyond Ruby.
- FitNesse is a web server, a wiki, and an automated testing tool. It is based on Ward Cunningham’s Framework for Integrated Test. FitNesse is designed to support acceptance testing rather than unit testing in that it facilitates detailed readable description of system function.
Conclusion
As it is said, “Nothing is permanent but change”. Waterfall model gave way to iterative model which in turn was replaced by agile as the preferred choice for software development. DevOps is the future. DevOps isn’t an individual effort, it’s a core value of IT organization. The only reason that DevOps works is because quality is built into the entire system. So it’s time we embrace and adopt this change.
A special thank you to Moin Syed for writing this article.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Syed Moin | Nov 4, 2015 | Agile, Blog, DevOps, Manual Testing, Software Testing
Introduction
Some of the common complaints about exploratory testing are that it is it needs subject matter expertise, it is not structured and it cannot be measured. But the fact is that exploratory testing is self-managed and self-structured. Exploratory testing is not a technique but an approach of exploring software without following documented steps.
Skills needed for exploratory testing
Exploratory testing can be more of skill based testing than a role based testing. Some of skills like questioning, observation, critical & lateral thinking will help an exploratory tester explore better. The ability to think beyond just a one sentence requirement and related test steps is part of the inherent skill which is a core need for exploratory testers. A well-executed exploratory testing rely on skill of the person. The tester uses his skills to approach the problem and controls the process. The tester continuously implements the feedback received from the previous test. So we can say that, exploratory testing is highly interactive and is an approach that allows in creating test designs when new information is discovered.
Structured exploratory testing
Some of the best practices during exploratory testing are to define test charters, test design heuristics and time boxing. These techniques will put some general guidelines around the testing being done and it gives a structure to define its success.
Test charters
Test charters give exploratory testing sessions a mission without being authoritarian.
Test charters outline:
- What areas of the system needs to be concentrated which is the mission scope
- When is the application ready to be used
- Responsibilities of the team doing the exploratory testing
- Duration of testing sessions
- The date and times of the testing
- The test environment being used to do the test
- Test data needed for testing
As the exploratory testing session progresses, charter should be updated to document observations or issues the team come across which can be used for further testing.
Test design heuristics
In today’s IT world where the functional complexities are always increasing and requirements always changing which makes launch applications to market even more challenging. It becomes imperative to think scientifically, use questions, explain the behavior, and based on the experiences gained from testing user predictions to create a test strategy. Scientific approach and using heuristics give the structure to exploratory testing.
The purpose of heuristics is to remind testers of what to think about when they are creating tests. Following are some pointers:
- Resources, constraints, and other elements in the project that may support or roadblock testing. Sometimes a tester must challenge constraints, and sometimes accept them.
- Software functionality is complex. All the corner cases to test should be covered. Cover all of it that matters, not just the parts that are easy to see.
- Quality criteria needs to be adopted that enables to determine if the application has problems.
- Test techniques are heuristics for creating tests. All techniques involve some sort of analysis of project environment, functional elements, and quality criteria.
- Observed quality is the result of testing. One can never know the “actual” quality of a software product, but through the application of a variety of tests, one can make an informed assessment of it.
Time boxing
Exploratory test session can be limited to a set timescale, a practice also known as time boxing. This will be a focused test effort of fixed duration. This can help concentrate on the project’s specific goals and scope, rather than drift into unfocused exploration.
Measuring efficiency
The efficiency of the exploratory testing can be measured by the time spent on the following:
- Test design and execution (T)
- Defect investigation & reporting (D)
- Configuration (C)
Analyze on the basis of the following criteria:
- More of ‘T’ indicates great code but might also mean poor defect identifying skills.
- More of ‘D’ might mean poor code quality but also suggest inefficient test reporting.
- More of ‘C’ reflects configuration and testability issues but it can also mean system is not ready to test yet.
The above method of measure learned from other practitioners is very helpful in calculating the efficiency.
Value of exploratory testing
The value of exploratory testing over other testing methods is
- Identifies complex defects in a system earlier
- Facilitates experimentation, discovery and learning
- Creates engagement by enabling people use their minds
- Provides user-oriented feedback to developers and business analysts
A special thank you to Moin Syed for writing this article.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Oct 21, 2015 | Agile, DevOps
Technology will revolutionize the things people and businesses use in everyday life so much that the very foundation of the software testing industry will significantly change. The future state of testing will become disrupted. Digital transformation is rapidly underway. The transformation goes to the heart of what you make and serve to your customers. It redefines some of your core competencies and what you are in business to do. It could even end up placing you in a new industry—possibly one that never existed before. This requires rapid adoption and embracing of technologies that are either underway or not developed. It requires individuals to embrace the unknown and gain knowledge.
Disruptive testing will challenge all individuals in the software testing industry. There will be some that will embrace the challenge, but most will struggle. Those who are able to move forward will be the testing leaders of the future. It will require Quality Engineers who are able to pick up the technology and come up with the best ways to identify problems. The days of manual testers banging away on keyboards are quickly disappearing. Disruptive testing has to include significant collaboration across all boundaries including development and business partners. Test metrics will evolve into test analytics and require an additional level of interpretation and understanding of what the information means.
These are exciting times and rapid changes are ahead! Seize the opportunity and become the best you can be.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Oct 16, 2015 | Agile, Automation, DevOps
More and more organizations are incorporating Quality Engineering into the SDLC. The organizations that are able to embrace this approach are able to significantly improve the quality of the applications that are delivered into production. It is necessary for the leadership within IT to buy into the concept for adoption to take place. Quality engineering helps to identify significant gaps that impede implementation of a top notch product. Quality is not a one time process that is created and implemented. It takes years and significant trial and error to get organizations where they need to be.
Moving your organization from QA Analysts to QA Engineers will accelerate your time to market. Automation used to be a luxury item in most QA organizations but in the DevOps world it is a necessity. Leveraging automated scripts for both development and qa will allow focused attention on the new features that are being built.
Quality Engineering is a mindset shift focusing on Business Assurance. Organizations need to ensure that the technology is high quality so that the company can make money. If you are able to build it right the first time, you can focus on what is coming next and not be so concerned about fixing what is broken in production.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
