Choosing the Right Test Strategy for Effective Testing

Key Concepts:

This presentation provides practical ideas about choosing the right software testing strategy.
The test strategy defines how the product will be tested. We discuss the project factors involved in selecting suitable test strategy for different software environments.

The test strategy helps us in answering the following questions:
  • How do we find critical problems in the product quickly?
  • What areas and functionality of the product are tested?
  • What techniques and tools are used for creating and executing tests?
  • What types of tests will be performed and when?
A good test strategy has the following characteristics:
  • Diversified, so that we include variety of different test techniques
  • Risk-focused, so that more time is spent in testing critical areas of the product
  • Product-specific, takes into account the user requirements of quality for a given product
  • Practical and pragmatic, maximizes efficiency of the available resources by cultivating them
  • Simple or complex testing based on the maturity of the product

Learning Objectives:

  • General and specific testing strategies and methodology useful for testing any software product
  • Choosing and customizing the right test strategy for a given product.
  • How to maximize people and machine resources to fulfill the chosen test strategy.
  • Understanding of the testing strategy used in a complex software product.
  • Understanding of real-life QE process and best practices.

Success Story:

I have applied these test strategies for more than 15 years in several projects successfully and delivered high quality software with effectiveness and efficiency. At Diebold technologies, currently I am applying these strategies to test one of the largest ATM  cloud networks in the world.

Cloud testing deep dive
Testing with respect to a cloud will employ both traditional and new age methods to perform testing. Testing and cloud are equally large domains in their own right. Let’s learn how to view the two together in the below sections.
1) Cloud testing forms:
Cloud testing can be broadly divided into four different categories based on what they aim to do :
  • Testing of the whole cloud: The cloud is viewed as a whole entity based on its features and testing is carried out based on that.
  • Testing within a cloud : This is the testing carried out inside of the cloud by checking each of its internal features
  • Testing across clouds: Based on specifications, here the testing is carried out on the different types of cloud like public, private and hybrid clouds.
  • SaaS testing in cloud: Functional and nonfunctional testing is performed based on requirements.
2) Cloud testing environments:
There are 3 types of cloud testing environments:
  • Private or public environment where the quality of applications deployed in them need to be validated.
  • A hybrid environment, where the quality of applications deployed in them need to be validated.
  • A test environment which is cloud based, where the quality of applications deployed in the cloud need to be validated.
3) Types of testing performed in cloud:
Testing in a cloud has to not only ensure that the functional requirements are met, but a strong emphasis needs to be laid on nonfunctional testing as well. Let’s take a look at the different kinds of testing that are performed.
Functional testing:
Goes without saying, that functional testing has to be performed to make sure that offering provides the services that the user is paying for. Functional tests ensure that the business requirements are being met. Some of the functional tests are described below:
  • System Verification Testing: This makes sure that the various modules function correctly with one another, thus making sure that their behavior is as expected.
  • Acceptance Testing: Here the cloud based solution is handed over to the users to make sure it meets their expectations.
  • Interoperability Testing: Any application must have the flexibility to work without any issues not only in different platforms, but also must work seamlessly when moving from cloud infrastructure to another.
Non-Functional testing:
Non-functional tests mainly focus on the web application based tests ensuring that they meet the desired requirements. Here are few forms of nonfunctional tests discussed below :
  • Availability Testing: The cloud supervisor/ vendor has to make sure that the cloud is available round the clock. As there could be many mission critical activities going on, the administrator has to make sure that there is no adverse impact to the consumers
  • Multi Tenancy Testing: Here, multiple users use a cloud offering. Testing must be performed to ensure that there is sufficient security and access control of data when multiple users are using a single instance.
  • Performance Testing: Verification of the response time needs to be done to ensure that everything is intact even when there is a large amount of requests to be satisfied. The network latency is also one of the critical factors to evaluate performance. Also workload balancing needs to be done when there is a reduction in load, by decommissioning resources. Thus load and stress testing are done in the cloud offering to make sure applications are performing optimally with increase/decrease in load and stress.
  • Security testing: Since with the cloud everything is available anytime, it’s essential to make sure that all user sensitive information has no unauthorized access and the privacy of users remains intact. When maintaining the applications in cloud, user data integrity must also be verified.
  • Disaster recovery Testing: As already stated in availability testing, the cloud has to be available at all times and if there are any kind of failures like network outages, breakdown due to extreme load, system failures, etc, measure how fast the failure is indicated and any data loss during this period.
  • Scalability Testing: Test to make sure that offering has the capability to provide scale up or scale down facilities as per the need.

Cloud testing tools:

In this section, some of the different tools used in various kinds testing performed in a cloud are mentioned. The details of the tools are out of the scope of this article.
Tools for Performance Testing in the Cloud:
Many of the tools are basically used for performance, load, stress testing. Some of these tools below can also be used for functional tests:
  • SOASTA CloudTest
  • LoadStorm
  • CloudTestGo
  • AppPerfect.
  • Jmeter
  • Cloudslueth
  • CloudTestGo
  • AppPerfect
Cloud Security Testing Tools:
There are also some security related tools which are commonly used:
  • Nessus
  • Wireshark
  • Nmap
You can refer the internet for details on various tools and how they can be used in a cloud infrastructure. One such link is found here.

Cloud Testing Challenges

As exciting as it cloud sounds, all is not hunky-dory here. There are some challenges with relying and using cloud as an infrastructure as well. Let’s take a look at some of the primary concerns using the cloud.
Challenge #1. With everything available on demand to any user, security is a primary issue for the businesses as currently there is still a lot of discussion and research going on in the industry to set up security standards. User privacy protection, security standards on cloud, security of applications running within the cloud, security testing techniques are some of the primary issues that need to be addressed in the cloud infrastructure.
Challenge #2. Another big challenge is the performance of an application in a cloud: specifically in private clouds. It will be shared across many users and hence could lead to delays. Also in case of some maintenance or outage related activities, the bandwidth may seem insufficient.
Challenge #3. Sometimes for testing purposes, we require certain configurations: with respect to servers, storage or networking which may not be supported by the cloud provider. This sometimes makes it difficult to emulate customer environments.
Challenge #4. Another commonly faced challenge is with respect to integration testing whereby the testers test the network, database, servers, etc. In such situations the tester will not have control on the underlying environment. Secondly, the challenge is doubled when there has to be an interaction between these components because the tester will have to anticipate risks like crashes, network breakdown or servers going kaput.

Conclusion

Cloud computing has today become one of those “big bangs” in the industry. Most organizations are now leaning to adopting the cloud because of its flexibility, scalability and reduced costs.
Using the cloud for testing is immensely helping organizations to acquire the required tools, software licenses, infrastructures at a very low cost without having to set it up themselves and then worry about its maximum utilization.
========================================================================

Comments

Post a Comment

Popular posts from this blog

How to calculate maximum number of concurrent users using littile law formula

When load testing an application, the first set of tests should focus on measuring the maximum throughput. This is especially true of multi-user, interactive applications like web applications. The maximum throughput is best measured by running a few emulated users with zero think time. This means that each emulated user sends a request, receives a response and immediately loops back to send the next request. Although this is artificial, it is the best way to quickly determine the maximum performance of the server infrastructure. Littile Law: Once you have that throughput (say X), we can use  Little’s Law  to estimate the number of real simultaneous users that the application can support. In simple terms, Little’s Law states that : N = X  / λ where N is the number of concurrent users,  λ is the average arrival rate and X is the throughput. Note that the arrival rate is the inverse of the inter-arrival time i.e. the time between requests. To under...
Read more

How to develop a security test strategy

What do you need to write a security test plan? The folowing documents comprise the list of what I would expect as inputs to the creation of the individual security test plans. This is a good point to go and review your overall security delivery plan. Does it include these documents as deliverables? Does the supplier have any of these as standard off-the-shelf products? : System security requirements System threat model System risk model Secure coding standards Technical threat modelling Infrastructure secure configuration guides Security defect metrics What is in a security test plan? The security test strategy should as a minimum specify the following points as the core content for each of the security test plans: What are the inputs to the test plan? When in the process does this test occur? Who specifies the test plan contents? Who performs the test? How many instances of the test are expected? What are the required outputs of the test? What are the success...
Read more

Singleton Design Pattern using Page Factory

Image
n this tutorial, We will learn about the  Framework Design Pattern using singleton class  in Selenium. In the Previous tutorial we have seen, How can we run the Test using Page Factory. Now question is, Will we repeat the Browser invoke and Logger code again and again in each and every Test. No, To resolve this issue we will use the Singleton Pattern. We will design the Single  Test Base  class.  First, Let's see what is  Singleton Pattern .  With the Singleton design pattern you can: Ensure that only one instance of a class is created Provide a global point of access to the object In  TestBase  class, we will Design the Singleton Pattern, We will write the command code here and all Test Classes will inherit this  TestBase  class.  n previous Example, we have seen we were invoking  WebDriver   instance  in each @Test. Here, We will Implement the WebDriver initializat...
Read more