Performance & Stability Scoring
Overview
When evaluating providers, you often want to balance raw performance (for example, speed) with consistency (low variability). A provider with the fastest average latency but frequent spikes might not be the best choice for latency-sensitive applications.
Provider Intelligence offers a Performance & Stability score that combines both components in an approach that ensures that providers with higher scores in both areas are advantaged over providers with only one high score.
The combined Performance & Stability score does not appear on the results page; it feeds directly into the overall score when you choose Performance & Stability as your results calculation mode.
Comparing Averages
A simple average (arithmetic mean) can artificially inflate combined scores when one component is high and the other is low. We use a kind of average (harmonic mean) that advantages higher component scores and penalizes providers where one component is high and the other low.
Why Harmonic Mean Is Better
You want providers that are both fast and consistent.
A provider that's very fast but unpredictable isn't a good choice.
A provider that's very consistent but slow isn't a good choice either.
The harmonic mean ensures you prioritize providers that are strong in both areas.
See Specify Results Calculation to understand when to choose Performance & Stability scoring vs. Performance only scoring in your overall score calculation.
Harmonic Mean Calculation and Example
Understanding the harmonic mean is best understood in relation to the arithmetic mean.
Arithmetic Mean: An arithmetic mean for 2 numbers (X and Y) is their sum divided by 2.
For example, if the performance score is 95 and the stability score is 50, the simple average is (95 + 50)/2 = 72.5.
Harmonic Mean: A harmonic mean multiplies X by Y and then by 2, before dividing by their sum. In this way, any difference between the two values pulls the final score more towards the lower of the two, penalizing the difference.
In the example above, the harmonic mean for 95 and 50 is instead 95 * 50 * 2/(95 + 50) = 65.
While the simple average treats both scores equally and can be pulled up by a high score, the harmonic mean is more sensitive to the lower score. This means if one score is significantly lower, the harmonic mean is closer to that lower value, effectively reducing the combined score to reflect the weaker aspect.
Comparison Example
Provider 1
90
90
90
90
Provider 2
95
50
72.5
65
Provider 3
70
70
70
70
Interpretation: Provider 2 has excellent raw performance (95) but poor stability (50). The simple average (72.5) suggests Provider 2 is better than Provider 3 (70), but this ignores the instability. The harmonic mean (65) corrects this by penalizing Provider 2 for inconsistency.
Last updated