Today I decided it was past time to reorganize the slides in the performance testing section of the course. I added a number of new diagrams and charts to help organize the materials and then I went back and started to review some of the definitions that I was using. First up was the question; what is performance testing and how is it related to load testing.
I must confess that I derived a number of my own definitions for a number of terms simply because I wasn’t satisfied with the definitions that were out there. I couldn’t really fault with many of the definitions. It was just that they were either, not useful in that they didn’t suggest a course of action when they should have. Or, the definitions were not in alignment with my experiences. That said, I do want to try to conform even if I find it difficult to do so on occasion. So, I went out the web to try to see if I could normalize my definitions of load and performance testing to what others were using.
While it’s clear that there is a difference between load and performance testing, that difference isn’t readily apparent until you define the two. According to Wikipedia, performance testing is “testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system such as scalability, reliability and resource usage.”
Quite a mouthful indeed and we can quibble about the quality of a definition that uses the words it’s trying to define as part of the definition (is that like autophagic?). What I can say is with all the words the definition uses, there are still a few missing that are needed if this is to be used to define a load test. For consistency lets return to Wikipedia to see what it says about load testing. It’s definition for load testing is; “the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions”. The definitions sound very similar but now the missing words have surfaced. The load testing definition constrains the phrase “particular workload” found in performance testing to “normal and anticipated peak load conditions”, what ever that means.
The differences in the definitions make the relationship between load, stress, endurance and performance testing while subtile, is apparent. The former terms are slightly different specializations of the later. In fact, the difference between them is sometimes so slight that it’s not surprising that there are so many different definitions out there. But the difference is important as each type of performance testing serves a different purpose and requires different treatment.