An “order of magnitude” gain in efficiency, whether its a business process or computer program, is something to strive for, but two orders of magnitude, despite sounding cool, is one too many.
Assume you have a perfectly linear process — say, a computer program processing data –Â whereby you can add additional processing nodes for parallel processing.Â If 1 run of your program takes 1 minute and you’ve got 100 iterations, you can reasonably expect to wait for 100 minutes.
100 units of work x 1 minute per unit = 100 minutes elapsed time
But since your program can scale linearly, you can add an additional program and cut the time in half!
(100 units x 1 minute) / 2 processors = 50 minutes elapsed time
Similarly, you can scale up to 4 processors and reduce elapsed time to 25 minutes.Â This is perfect linear scaling and with your big math brain, you figure out that you can get a 10X gain by scaling up to 10 processors!
So far, so good.Â 10x is an order of magnitude and represents a 90% decrease in elapsed time.
(100 units x 1 minutes) / 10 processors = 10 minutes elapsed time
10 minutes is 10% of the original 100 minute elapsed time. 10x gain!
I think the second order of magnitude is a waste of time.Â That’s right, it’s not worth going for another 10x gain.
Why?Â It costs too much!
Assume that a server costs $1000 and your process will consume the entire processing capacity of a server.Â Scaling up to 10 servers costs $10,000.Â You reduced processing time by 90% for $10k.
Math is not on your side for the second order of magnitude.Â Taking your elapsed time from 10 minutes to 1 minute is another order of magnitude, but it also is 90% of your cost!
You have a perfectly linearly scalable process, right?Â So, reducing your 100 minute elapsed time requires 100 servers at $1000 each.Â That’s $100,000!Â Meanwhile, already achieved a 90% reduction for $10,000.
90% of the gain is achieved by 10% of the investment.Â The remaining 10% of the gain requires 90% of the investment!
Pareto was right.Â The 80/20 rule applies, but in our case its 90/10.
The chart below shows two orders of magnitude.Â You can’t help but notice the point of diminishing returns.Â It doesn’t seem worthwhile to go for that second order of magnitude.