Monday, 30 January 2012

Performance Testing - Non Functional testing requirements

Before starting the Performance testing, performance test engineer should collect non-functional requirement and Below is comprehensive list of performance Testing Requirements.

  •  Total user base : How many users will have access to this application? 
    • Customer will provide answer, how many users are having an access to the application.
    • If the application is existing one, Customer will get the data from previous years to understand how many users are using the application. There are third party tools like Web trends to analyze the traffic of existing used application
    • If the application is new and not yet in live, Business analyst and Application owners will be able to provide total user base.
  • Concurrent User Load :What is the number of total users who will access the application together at any given point in time?
    • According to industry standard, it is 10% of total users will be the concurrent load but this formula will not applicable for all the domains.
    • Third party tools like web trends will be useful
  • Business Processes ( Performance Scenarios):  
    • What are the business processes (Workflows) to be performance tested with concurrent users? Objective here is to find few business processes which can cover up to 80% of the application code"

    • Load Distribution :What is the user/Load distribution on each business processes  listed 
    • What is the time duration taken for each business process to complete end to end(Workflow)? 
    • How often are the users doing each business process (Workflow) in an hour, a day, etc? to get this data for existing application
  • Service Level Agreement (Performance - Service Level Agreement):
    • What are the acceptable response times for each transaction (page) in all the business flows that needs to be performance tested? Mention SLA for against each of the transaction (page) identified
    • 1. All pages within the portal shall load in 3 seconds or less.
    • 2. Report should load in 5 sec
    • 3. Yearly Report should load in 20 
    • 4. SecgetAccountHistory API’s roundtrip response time should be < 250 msec

Sunday, 22 January 2012

Core Activities of Performance Tester using Loadrunner

Core Activities of Performance Tester using Loadrunner

In performance testing there are 4 phases:
  • Planning
  • Scripting
  • Execution
  • Analysis and Reporting
Activity 1. Identify the Test Environment : 

Identify the physical test environment and the production environment as well as the tools and resources available to the test team. The physical environment includes hardware, software, and network configurations. Also gather sizing difference between production server and load test environment. Having a thorough understanding of the entire test environment at the outset enables more efficient test design and planning and helps you identify testing challenges early in the project.

Activity 2. Performance Acceptance Criteria :
Identify the response time, throughput, and resource utilization goals and constraints. In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern. Additionally, identify project success criteria that may not be captured by those goals and constraints.

Activity 3. Plan and Design Tests :
Identify key scenarios, determine user mix among representative scenarios and how to simulate that variability, define test data, and establish metrics to be collected. Consolidate this information into one or more models of system usage to be implemented, executed, and analyzed.

 Activity 4. Configure the Test Environment :
Prepare the test environment, tools, and resources necessary to execute each strategy as features and components become available for test. Ensure that the test environment is instrumented for resource monitoring as necessary.

Activity 5. Implement the Test Design :
Develop scripts using Loadrunner Vugen componenet

Activity 6. Execute the Test :
Run and monitor your tests. Validate the tests, test data, and results collection. Execute validated tests for analysis while monitoring the test and the test environment. use Loadrunner Controller or Performance center to execute load tests and Monitor servers.

Activity 7. Analyze Results, Report, and Retest :
Consolidate and share results data. Use Loadrunner Analysis to Analyze the data both individually and as a cross-functional team. Reprioritize the remaining tests and re-execute them as needed. When all of the metric values are within accepted limits, none of the set thresholds have been violated, and all of the desired information has been collected, you have finished testing that particular scenario on that particular configuration.