Performance Testing Introduction
Performance Testing is a Non-functional testing.
Performance testing is an engineering process in which the behavior of the software, hardware or network is observed by putting a load of virtual users on the system. Performance parameters like Response Time, Transactions/Second and Throughput etc. are determined in general cases.
That was a high-level definition, to explain it better let me take an example, for instance, there is a web application (assume an eCommerce website). On an eCommerce website a user expect a product to be loaded as fast as possible, even faster than an eye blink, Images should be displayed quickly, browsing of the products should be fast.
In order to achieve that, performance testing is required.
There are different types of performance testing, each type has its own rules. Depending on the goal of the performance testing, what needs to be captured, which area of the application needs to be focused a type of performance test is chosen. Below is the meaning of each type of testing, which tells when and why to do it:
Load Testing: Load testing is usually done while performing a performance test. This is done to observe the response time, throughput, hits/second and other performance parameters under a reasonable load to understand the behavior of the system. This test, in general, is of a shorter duration like 1 or 2 hours. This test is generally executed multiple times. Every time a tweak is made by the developer, the performance tester execute this test to check the performance gain.
Scalability Testing: is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability. Performance, scalability, and reliability testing are usually grouped together by software quality analysts.
Volume Testing: Volume testing is done for a duration of 1 to 2 hours depending on the need. Where the software is subjected to a huge volume of data. It is also referred as Flood Testing.
Spike Testing: is done to test the sudden increase in the number of virtual users or load generated by the users to a very large amount and observing the behavior of the system. The goal is to determine whether the performance of the system will suffer/fail or it will be able to handle the dramatic changes in the load.
Stress Testing: This testing is done to determine the system’s robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently in case the current load goes well above the expected maximum.
Endurance Testing: To determine if the system can sustain a continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks, i.e. applying a significant load on a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. This type of Performance testing is also known as Soak Testing.
Baseline Testing: It is also referred as Benchmark testing. Performance testing starts with performance baseline testing, performed by the test engineer for comparing the performance of a new server or an unknown server to a known standard of reference like existing measurements or a software.
Smoke Testing: A smoke test is a test run under very low load that merely shows that the application works as expected.If smoke is generated, the test fails and no further testing is necessary until the simplest test passes successfully.
There are many different phases in performance testing. The phases shown here are high-level which are always touched during any type of performance testing. This could change depending on the application under test.
Requirement Gathering: During the requirement gathering phase performance tester/engineer asks questions that will help in understanding the purpose of the application, technologies used to develop the application, concurrent load with which application needs to be load tested and purpose of the load testing, etc.
Workload Modeling: Identification of the application workflows that needs to be tested and the concurrent user load on each of the workflow. There is always a confusion in identifying the person who can suggest the workflows to be tested. The workflow identification should be done by the Business Analyst of the application and performance tester should be engaged as a consultant, but this responsibility lies with the Business Analyst. While the responsibility to give the final shape to workload model lies with the performance tester/engineer.
Scripting: In this phase a performance tester/engineer creates the performance scripts using tools like JMeter. Which includes recording of the workflow, putting parametrization, correlation of dynamic values, put loops for recurring events, assertions points and wait timers to make it a real time scenario.
Test Environment Setup: Creating execution plan in the performance testing tools which includes the setting up of the performance scripts to execute, configuring the runtime settings like pacing, ramp up time, timeout time, deployment of the server side monitoring tools, application and database profiling tools.
Load Test Execution: In this phase, the performance tester/engineer executes the test plan to put the load on the application and monitors the load test results like response time, errors, hits/second, etc.
Analysis & Reporting: After the load test execution, it’s time to analyze the raw results and find out the bottlenecks in the application. A performance report is created by the performance tester/engineer which is then shared with all the stakeholders to take necessary action.
- We are living in the world where irrespective of the collection and quality of the products user will lose interest if the speed of the website or application is slow. According to surveys done by Akamai and Gomez.com, nearly half the web users expect a site to load in 2 seconds or less, and tend to abandon a site that does not load within 3 seconds. 79% of web shoppers who have trouble with a website performance say they won’t return to the site to buy again and around 44% of them would tell a friend if they had a poor experience shopping online. This means you’re not just losing conversions from visitors currently on your site, but that loss is magnified to their friends and colleagues as well. The end result –lots of potential sales down the drain because of just a few seconds difference.
- There are chances that during heavy load the hardware or server may crash which will cause the user to lose trust in your site. This makes an impression on the user that the website owner is not serious about their business. This impression will last too long and because of which users avoid falling on such websites second time.
- JMeter: is the most widely used tool in performance testing. Not just because it’s free, but also due to the fact that it is equipped with all the necessary features. If it doesn’t have any particular feature, then you can easily build that feature as it’s open source as well. As JMeter is free, open source and widely used it’s quick to learn and interesting to use.
- Loadrunner: can test almost any type of application which makes it unique, but the cost of doing performance test using Loadrunner is too high, only big companies with high revenues can afford this tool. Which restricts the use of this tool in small and mid-level organizations.
- Microsoft VSTS or Microsoft Visual Studio Team System: is a performance testing tool from Microsoft. This tool is best suited for .Net applications. We find performance testers using VSTS for the performance testing of .Net applications because by using VSTS the scripting time of .Net applications reduces to some extent.
NOTE: There are other tools as well in the market, but the tools listed here are the once those are commonly known and are generally used for performance testing.