Discover issues with performance testing

Try this approach to discover issues before they even come into play in production. Would you like to know how? Check this out!

Image describing the blog title: Discover issues before your app goes to production with performance testing!
  1. The cache is not working correctly.
  2. The REST API is too slow for most of the requests.
  3. The database uses 100% of the CPU during peak hours, but it seems dead even in some basic circumstances.
  4. Some users receive data from other users, which could denote race conditions (or perhaps even worse things).
An almost pyramid format showing three approaches to test you application.

Gathering functional requirements

Before creating your performance testing script, it’s a good idea to start answering how many requests your application should handle (in the case of a REST API), given its business requirements. To illustrate, imagine that your application is an ad-hoc one specially built to deal with incoming requests from your partners, and it should be capable of working with 3,500 requests per minute. In addition to that, your app should answer each request until the limit of 500 ms. There we go! We have something to start our script 😄. By the way, I’m roughly speaking here. I’m just giving a fictional example. Sometimes you might not even have this information 😬, which is why some measurements are required to retrieve that first. You can check out many cases from the setting performance goal on the Wikipedia article about the related topic.

Apdex: Measure user satisfaction

According to the APDEX official website:

JMeter

There are many tools out there like Gatling, Locust, and K6, but here I’ll explain just a bit of JMeter. It is totally visual and pretty easy to follow and understand 🤗; it also includes a module that generates dashboards that gives us an APDEX result, which is what we want. By default, I have it on my system; you can use the same idea or simply download it from the official website and run it afterward.

Sample project and creating a script

We’ll use the REST API Version of the Tic Tac Toe game that I have written using C#! Now let’s see the performance test script. Ah! It’s worth mention that I won’t explain in detail each configuration step. JMeter has an excellent assistant feature. You just add a resource and click on the help icon. It will redirect you to the manual related to the same resource you added. It’s awesome 😛. Thus, if you don’t understand a given component, either go to the help page or comment below that I will happily help you 🤟.

The current configuration of the thread group used by the performance testing.
  • 775 users (threads). Each user will execute 10 requests (see the green dropper icon symbolizing an HTTP request), which means we will have 7750 samples.
  • All the users will make their first request in 60 seconds. After this ramp-up, the test will be fully up to speed.
  • Each user can live up to 120 seconds.
  • apdex_satisfied_threshold: Sets the satisfaction threshold for the APDEX calculation in ms. By default is 500.
  • apdex_tolerated_threshold: Sets the tolerance threshold for the APDEX calculation in ms. By default is 1500.
A table containing the APDEX result.

Simple workflow to start with

I believe half a loaf is better than none. So starting from this idea, with the help of containers, given our performance testing script is done and well tested, we can create a compose service to execute the test plan so that the result is available in some format. This output can be used to assert that the result matches our expectations. Look at this simple C# code that we can use to verify the result:

public class PerformanceTestingResultTest
{
private JObject _generalStatistics;
private JObject _applicationPerformanceIndex;
[SkippableFact(DisplayName = "Should have no error in all requests")]
public void TestScenario1()
{
// Arrange
PreparePropertiesAndSkipTestIfNeeded();
// Assert
_generalStatistics.Count.Should().Be(11);
foreach (var token in _generalStatistics)
{
var errorCount = token.Value["errorCount"].As<int>();
errorCount.Should().Be(0);
}
}
[SkippableFact(DisplayName = "Should APDEX result be greater than 0.9 to all evaluated requests")]
public void TestScenario2()
{
// Arrange
PreparePropertiesAndSkipTestIfNeeded();
var rawOverallResult = _applicationPerformanceIndex["overall"]["data"];
var rawRequestsResults = _applicationPerformanceIndex["items"];
// Assert
rawOverallResult.Should().NotBeNullOrEmpty();
rawRequestsResults.Count().Should().Be(10);
foreach (var token in rawRequestsResults)
{
var apdexResultForGivenRequest = token["data"][0].Value<double>();
apdexResultForGivenRequest.Should().BeGreaterThan(0.9);
}
}
private void PreparePropertiesAndSkipTestIfNeeded()
{
// Skip test if required
var value = Environment.GetEnvironmentVariable("EVALUATE_PERFORMANCE_TESTING");
bool.TryParse(value, out var shouldNotSkipTest);
Skip.If(shouldNotSkipTest is false, "It was informed to be skipped");
// General statistics
_generalStatistics = FileHandler.ReadFileAsDictionary("tests-jmeter/statistics.json");
// APDEX sadly is not in only one file, we should extract it
var lineWhereApdexResultIs = @"createTable($(""#apdexTable"")";
var regexPatternToCaptureResult = @", ({"".+]}), ";
var dashBoardFile = FileHandler.EnumerableFromFile("tests-jmeter/content/js/dashboard.js");
foreach (var line in dashBoardFile)
{
if (line.Contains(lineWhereApdexResultIs))
{
var match = Regex.Match(line, regexPatternToCaptureResult);
var rawApdexResult = match.Groups[1].Value;
_applicationPerformanceIndex = JObject.Parse(rawApdexResult);
}
}
}
}

Conclusion

When I started my career as a developer, I wish I’d had this advice right from the start, so that’s why I created this post. JMeter can help you learn even more programming topics such as concurrency, parallelism, threading, coroutines, and many others. In addition, it opens a new broad set of tools that are at our disposal. I hope it can help you in any way and, as always, feel welcome to post any question you have.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store