by Ron Wilson | Nov 9, 2015 | Agile, Blog, DevOps, Software Testing
There are many DevOps benefits. Here are a few:
- Improved quality of software deployments. Developing and testing more frequently produces a much improved quality product.
- Frequent software releases. Having more frequent software releases gives the business what they need to be successful.
- Faster business response. Times are changing rapidly and that requires businesses to adjust quickly to their competitors.
- More Agile development. Since there has been a big push towards Agile, there needs to be a greater amount of collaboration between teams and requires continuous delivery of code into production.
- Improved visibility into IT. IT is notorious for not keeping the business informed on what is going on within IT. DevOps helps to improve this communication and provides an additional layer of transparency that adds value to any organization.
- Greater collaboration. Perhaps one of the best DevOps benefits is the amount of collaboration that Agile brings to the table. Product owners, developers and testers are working closer together and producing much better results than ever before.
As DevOps evolves there will be more benefits in terms of efficiency that will occur. High quality implementations along with more frequent code deployments will allow companies to provide more for their customers. It will also allow companies to try things that may not work and give them flexibility to grow and expand.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Syed Moin | Nov 9, 2015 | Agile, Blog, DevOps, Software Testing
Introduction
Aristotle said, “Quality is not an act, it is habit. What we can interpret is that quality cannot be a response to a certain situation or something that is faked or postured. Rather quality should be part of a routine; the acts we do on a continuous basis and which should be a reflection of the essential nature of work. This is how today’s IT world is looking at the QA practice.
In a traditional IT world, testing is often seen as separate from the development. They were two distinct entities separated by ideological walls. Thanks to the new trends in IT world, these walls are fast disappearing. One of the major driver for this transformation is “DevOps”.
DevOps
So what is DevOps? It is not a methodology or a suite of tools as many people think. DevOps is a culture that enables seamless coordination among the different stakeholders in the software chain. DevOps brings together the two critical silos of software process namely, development & operations to integrate within the same software cycle.
DevOps drives all the IT stakeholders to collaborate to deliver value faster. For this it collates the practices of continuous integration, continuous testing, continuous deployment and continuous monitoring.
DevTestOps
As we know DevOps aims to break the barriers between Dev and Ops. It also demands breaking the barriers between Dev, Test and Ops. Testing is a critical part of the handoff between development and IT operations. For DevOps to be successful it is mandatory that testing should be fully integrated into the software development and delivery.
With this mindset, everything changes for a traditional testing organization. Testing operations undergo a major changes in a DevOps culture. This mandates a shift in tester’s skillset, methods, tools, timing and approach. DevOps advocates starting test early and constantly.
Some tips for successful DevOps testing:
- Involve development team during test design.
- Identify the critical test cases and prepare the test suite to test the functionality of the required build.
- All the environments required for testing need to be standardized and deployments have to be automated.
- Adopted test-first approaches such as test-driven development (TDD) and behavior-driven development (BDD).
- Keep the test execution cycles short and quick.
- Adopt regression test automation to run across various environments.
- Implement code analysis and coverage tools to ensure 100% code coverage.
- Critical bugs found need to be reported, fixed and passed through the same chain of events before the code is deployed on to the production environment.
- Encourage parallel execution of tests to help reduce time for go live which is the heart of a successful DevOps implementation.
Testing maturity is a key differentiator for the success of DevOps. Even if organizations can automate their integrations, builds and delivery processes but they still struggle to manage test orchestration and automation. Testing brains play a critical role to achieve this with their expertise in test design, test automation and test case development with DevOps. Irrespective of what DevOps processes, models and tools organizations use, testing is a vital part of the overall DevOps process — not only to ensure code changes work as expected and integrate well — but to ensure the requirement changes do break the functionality.
DevOps Testing Tools
In order to realize the desired business goals of DevOps, it is essential to have right tools to maintain the quality associated with the delivery chain. This calls for accurate and comprehensive testing. So it becomes very important to understand the existing testing process and identify the right testing tools.
Some recommendations of testing tools are below:
- Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including Java, C#, Groovy, Perl, PHP, Python and Ruby.
- XL TestView is a test management and analysis tool that allows you to define and execute tests across your full spectrum of test tool suite. It analyzes test results across multiple test tools and track release metrics and quality trends over time. It helps aggregate and visualize results to provide true insight into quality.
- Cucumber is a testing tool that computer programmers use for testing. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Cucumber is written in the Ruby programming language. Cucumber projects are available for other platforms beyond Ruby.
- FitNesse is a web server, a wiki, and an automated testing tool. It is based on Ward Cunningham’s Framework for Integrated Test. FitNesse is designed to support acceptance testing rather than unit testing in that it facilitates detailed readable description of system function.
Conclusion
As it is said, “Nothing is permanent but change”. Waterfall model gave way to iterative model which in turn was replaced by agile as the preferred choice for software development. DevOps is the future. DevOps isn’t an individual effort, it’s a core value of IT organization. The only reason that DevOps works is because quality is built into the entire system. So it’s time we embrace and adopt this change.
A special thank you to Moin Syed for writing this article.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Syed Moin | Nov 4, 2015 | Agile, Blog, DevOps, Manual Testing, Software Testing
Introduction
Some of the common complaints about exploratory testing are that it is it needs subject matter expertise, it is not structured and it cannot be measured. But the fact is that exploratory testing is self-managed and self-structured. Exploratory testing is not a technique but an approach of exploring software without following documented steps.
Skills needed for exploratory testing
Exploratory testing can be more of skill based testing than a role based testing. Some of skills like questioning, observation, critical & lateral thinking will help an exploratory tester explore better. The ability to think beyond just a one sentence requirement and related test steps is part of the inherent skill which is a core need for exploratory testers. A well-executed exploratory testing rely on skill of the person. The tester uses his skills to approach the problem and controls the process. The tester continuously implements the feedback received from the previous test. So we can say that, exploratory testing is highly interactive and is an approach that allows in creating test designs when new information is discovered.
Structured exploratory testing
Some of the best practices during exploratory testing are to define test charters, test design heuristics and time boxing. These techniques will put some general guidelines around the testing being done and it gives a structure to define its success.
Test charters
Test charters give exploratory testing sessions a mission without being authoritarian.
Test charters outline:
- What areas of the system needs to be concentrated which is the mission scope
- When is the application ready to be used
- Responsibilities of the team doing the exploratory testing
- Duration of testing sessions
- The date and times of the testing
- The test environment being used to do the test
- Test data needed for testing
As the exploratory testing session progresses, charter should be updated to document observations or issues the team come across which can be used for further testing.
Test design heuristics
In today’s IT world where the functional complexities are always increasing and requirements always changing which makes launch applications to market even more challenging. It becomes imperative to think scientifically, use questions, explain the behavior, and based on the experiences gained from testing user predictions to create a test strategy. Scientific approach and using heuristics give the structure to exploratory testing.
The purpose of heuristics is to remind testers of what to think about when they are creating tests. Following are some pointers:
- Resources, constraints, and other elements in the project that may support or roadblock testing. Sometimes a tester must challenge constraints, and sometimes accept them.
- Software functionality is complex. All the corner cases to test should be covered. Cover all of it that matters, not just the parts that are easy to see.
- Quality criteria needs to be adopted that enables to determine if the application has problems.
- Test techniques are heuristics for creating tests. All techniques involve some sort of analysis of project environment, functional elements, and quality criteria.
- Observed quality is the result of testing. One can never know the “actual” quality of a software product, but through the application of a variety of tests, one can make an informed assessment of it.
Time boxing
Exploratory test session can be limited to a set timescale, a practice also known as time boxing. This will be a focused test effort of fixed duration. This can help concentrate on the project’s specific goals and scope, rather than drift into unfocused exploration.
Measuring efficiency
The efficiency of the exploratory testing can be measured by the time spent on the following:
- Test design and execution (T)
- Defect investigation & reporting (D)
- Configuration (C)
Analyze on the basis of the following criteria:
- More of ‘T’ indicates great code but might also mean poor defect identifying skills.
- More of ‘D’ might mean poor code quality but also suggest inefficient test reporting.
- More of ‘C’ reflects configuration and testability issues but it can also mean system is not ready to test yet.
The above method of measure learned from other practitioners is very helpful in calculating the efficiency.
Value of exploratory testing
The value of exploratory testing over other testing methods is
- Identifies complex defects in a system earlier
- Facilitates experimentation, discovery and learning
- Creates engagement by enabling people use their minds
- Provides user-oriented feedback to developers and business analysts
A special thank you to Moin Syed for writing this article.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Oct 21, 2015 | Agile, DevOps
Technology will revolutionize the things people and businesses use in everyday life so much that the very foundation of the software testing industry will significantly change. The future state of testing will become disrupted. Digital transformation is rapidly underway. The transformation goes to the heart of what you make and serve to your customers. It redefines some of your core competencies and what you are in business to do. It could even end up placing you in a new industry—possibly one that never existed before. This requires rapid adoption and embracing of technologies that are either underway or not developed. It requires individuals to embrace the unknown and gain knowledge.
Disruptive testing will challenge all individuals in the software testing industry. There will be some that will embrace the challenge, but most will struggle. Those who are able to move forward will be the testing leaders of the future. It will require Quality Engineers who are able to pick up the technology and come up with the best ways to identify problems. The days of manual testers banging away on keyboards are quickly disappearing. Disruptive testing has to include significant collaboration across all boundaries including development and business partners. Test metrics will evolve into test analytics and require an additional level of interpretation and understanding of what the information means.
These are exciting times and rapid changes are ahead! Seize the opportunity and become the best you can be.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Oct 16, 2015 | Agile, Automation, DevOps
More and more organizations are incorporating Quality Engineering into the SDLC. The organizations that are able to embrace this approach are able to significantly improve the quality of the applications that are delivered into production. It is necessary for the leadership within IT to buy into the concept for adoption to take place. Quality engineering helps to identify significant gaps that impede implementation of a top notch product. Quality is not a one time process that is created and implemented. It takes years and significant trial and error to get organizations where they need to be.
Moving your organization from QA Analysts to QA Engineers will accelerate your time to market. Automation used to be a luxury item in most QA organizations but in the DevOps world it is a necessity. Leveraging automated scripts for both development and qa will allow focused attention on the new features that are being built.
Quality Engineering is a mindset shift focusing on Business Assurance. Organizations need to ensure that the technology is high quality so that the company can make money. If you are able to build it right the first time, you can focus on what is coming next and not be so concerned about fixing what is broken in production.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
by Ron Wilson | Oct 13, 2015 | DevOps, Software Testing, Test Environments
Software testing has evolved and performed a more critical role in the software development lifecycle. As you can see from the picture, software testing has grown increasingly more important since 1990. When I started testing in the late 1990’s testing groups were established and black box testing was the standard. Typically testing was done around the large projects and done within a waterfall model. Automation and performance testing tools were created and regression suites were automated to speed up the testing timelines. Metrics evolved during that timeline as well to give the managers the ability to make decision based upon information.
The industry then moved to centralize testing teams and create consistent standards across multiple projects. Through software testing evolution, teams became experts in multiple applications in order to gain additional efficiency through a single organization. With business and development teams embracing Agile, testing organizations had to evolve as well. Testing organizations moved from a centralized team to distributed teams in order to operate more efficiently. It will take several years for companies to gain efficiency and determine what additional skills and tools will be needed. Testing teams will need to move more towards the left in order to find defects faster and gain domain business knowledge with the applications that are being tested.
The future in software testing evolution will require testers to evolve into QA Engineers. This will require more advanced skills around automation and development programming. It will be really interesting to see what the future holds.
If you would like more information on Agile, DevOps or Software Testing, please visit my Software Testing Blog or my Software Testing YouTube Channel.
