An “order of magnitude” gain in efficiency, whether its a business process or computer program, is something to strive for, but two orders of magnitude, despite sounding cool, is one too many.

Why?

Assume you have a perfectly linear process — say, a computer program processing data –  whereby you can add additional processing nodes for parallel processing.  If 1 run of your program takes 1 minute and you’ve got 100 iterations, you can reasonably expect to wait for 100 minutes.

100 units of work x 1 minute per unit = 100 minutes elapsed time

But since your program can scale linearly, you can add an additional program and cut the time in half!

(100 units x 1 minute) / 2 processors = 50 minutes elapsed time

Similarly, you can scale up to 4 processors and reduce elapsed time to 25 minutes.  This is perfect linear scaling and with your big math brain, you figure out that you can get a 10X gain by scaling up to 10 processors!

So far, so good.  10x is an order of magnitude and represents a 90% decrease in elapsed time.

(100 units x 1 minutes) / 10 processors = 10 minutes elapsed time
10 minutes is 10% of the original 100 minute elapsed time. 10x gain!

I think the second order of magnitude is a waste of time.  That’s right, it’s not worth going for another 10x gain.

Why?  It costs too much!

Assume that a server costs $1000 and your process will consume the entire processing capacity of a server.  Scaling up to 10 servers costs $10,000.  You reduced processing time by 90% for $10k.

Math is not on your side for the second order of magnitude.  Taking your elapsed time from 10 minutes to 1 minute is another order of magnitude, but it also is 90% of your cost!

You have a perfectly linearly scalable process, right?  So, reducing your 100 minute elapsed time requires 100 servers at $1000 each.  That’s $100,000!  Meanwhile, already achieved a 90% reduction for $10,000.

90% of the gain is achieved by 10% of the investment.  The remaining 10% of the gain requires 90% of the investment!

Pareto was right.  The 80/20 rule applies, but in our case its 90/10.

The chart below shows two orders of magnitude.  You can’t help but notice the point of diminishing returns.  It doesn’t seem worthwhile to go for that second order of magnitude.