Load and Performance Testing

Load and performance tests can help expose and identify causes of poor performance. These tests provide insight for how a site will perform during peak traffic spikes.

Load vs Performance Testing

Before you start, it's important to understand the difference between load and performance testing and know when to use each.

Performance Testing

Performance testing is the process in which you measure an application's response time to proactively expose bottlenecks. In addition to regularly referring to your New Relic reports, you should consider regularly executing performance tests as part of routine maintenance to ensure performance isn't being degraded by code or configuration changes. You should run these test before any load testing. If your application is not performing well, then you can be assured that the load test will not go well.  

The scope of performance tests should be limited to the application itself on a development environment (Dev or Multidev) without caching. This will give you an honest look into your application and show exactly how uncached requests will perform. You can bypass cache by setting the no-cache HTTP headers in responses.

Pre-launch Load Testing

Load tests provide critical insight into how a site will perform when released, including peak traffic spikes. Load tests ensure that your website launches smoothly and can sustain high traffic.

Using proprietary tools developed by the Pantheon Onboarding team, we generate traffic to your Live site, simulating complex user interactions if needed. This is the best way to expose and identify potential performance bottlenecks before you launch your site.

Newly purchased Elite plans include a pre-launch load test. Contact us for more information.

If your site is already live, then you should run load tests on the Test environment. For Live environments with multiple application containers, keep in mind that the Test environment has two application containers. Run a proportionate amount of traffic based on the number of Live environment app containers you have. If you have four app containers in Live, then test with half of your anticipated peak traffic. You can see the number of app containers using Pantheon's free New Relic® Performance Monitoring offering.

Preparing for Tests

The procedure for executing a load test and a performance test are similar:

1. Enable New Relic® Performance Monitoring within the Site Dashboard on Pantheon to ensure you have clear reporting to monitor response times.

   • Set your apdex threshold according to your business rules (.5 is the default). Be careful not to set this too high, otherwise you will not get as many transaction traces in New Relic.
   • If you have particular transactions that you want to ensure are traced, set them up as key transactions.

2. Select a load testing tool:

   • SaaS Solutions
     • Blazemeter
     • K6.io
   • Open Source tools
     • Jmeter
     • Locust

   Ultimately, it doesn't matter what tool(s) you use as long as you test your site against the anticipated traffic patterns of the site in terms of overall volume and the proportion of anonymous versus authenticated traffic. Note that load testing for anonymous visits is considerably easier than testing authenticated workflows, which will require more investment of time and skills.  

3. Determine how much load to apply.

   • Performance Tests: Smaller loads should suffice, as you should be able to see transactional bottlenecks with 10-20 concurrent users.
   • Load Tests: Determine how many concurrent users per second the site is expected to serve based on historical analytics for the site. Identify the peak hourly sessions and average session duration, then do some math: Concurrent Users = ( hourly_sessions x average_duration ) / 3600

Running the Tests

If this is a performance test, be sure to run the test on a development environment (Dev or Multidev) without caching. Run load tests on the Live environment before launching the site. If the site is already launched, use the Test environment instead.

Note the start time for the test. As the test executes, it's a good idea to keep a close eye on log files. Make note of any errors and warnings that pop up during the test so that you can fix them.

Once the test is running, execute common tasks done by editors and administrators and note the time. Example tasks may include:

• Clear the cache
• Clear the edge cache (if this is a load test, performance tests should not be cached)
• Run cron
• Run any scripts that could be triggered while users are on the site
• Flush Redis cache (if Redis is running)

Determine the Number of Concurrent Users in Google Analytics

Pantheon offers Pre-launch Load Testing to newly purchased Elite plans. For the Professional Services team to proceed with executing the load test, a load test questionnaire form needs to be filled out with required information. The required information includes Peak Page Views, Peak Sessions, Peak Users and Average Session Duration which can be retrieved from your site's Google Analytics data.

The following procedure guides you on how to retrieve load test information from your site's Google Analytics data. Ensure you have been invited or have access to the correct account as well as the correct views and properties.

1. Click the All Web Site Data dropdown menu in the top left corner of the page.

2. Select the correct Analytics Account, Properties & Apps and Views of the site.

3. Click Audience > Overview.

4. Use the dropdown menu to set the date range filter to a specific year in the top right corner.

5. Open the Overview tab > select Sessions.

6. Click Day in the Overview tab to set the range to “Day“ > Take note of the day with the highest sessions.

7. Set the date filter to the day with the highest sessions.

8. Set the range to “Hourly“ and find the highest peak.

   The following information is displayed below the main graph:

   • Users: Users who have initiated at least one session during the date range.
   • New Users: The number of first-time users within the selected date range.
   • Sessions: Total number of sessions within the date range.
   • Number of Sessions per User: The average number of sessions per user.
   • Pageviews: The total number of pages viewed. Repeated views of a single page are counted.
   • Pages / Session: The average number of pages viewed during a session; it is also known as the Average Page Depth. Repeat views of a single page are counted.
   • Avg. Session Duration: The average length of a session.
   • Bounce Rate: The percentage of single-page sessions in which there was no interaction with the page.

9. Click the peak to display the date, time, and number of sessions.

   The highest hourly peak denotes the peak sessions.

10. Open the Overview tab > Click Users.

11. Set another metric filter to “Pageviews“.

12. Use your mouse to hover over the highest peak.

    The Peak Users and Peak Pageviews should now display.

Assess Results

1. Examine the New Relic data in the Overview tab.

   This will give you an average response time for the duration of the test. Times above 750ms are good indicators of performance optimization opportunities.

2. Review the Transactions tab in New Relic and sort by Slowest average response time.

3. Click on the slowest transaction to pull up the transaction trace > Review to find the performance bottleneck.

4. Review the Error analytics tab in New Relic.

   PHP errors often indicate huge performance bottlenecks.

5. Correct any errors.

Calculating Load Capacity After Launch

After launch, you can establish a baseline that X response time will let you handle Y traffic. If X degrades in Dev/Test, that will impact how much traffic Live can handle.

More Resources

• Load Testing Drupal and WordPress with BlazeMeter