Two “orders of magnitude” is one too many

An “order of magnitude” gain in efficiency, whether its a business process or computer program, is something to strive for, but two orders of magnitude, despite sounding cool, is one too many.

Why?

Assume you have a perfectly linear process — say, a computer program processing data –  whereby you can add additional processing nodes for parallel processing.  If 1 run of your program takes 1 minute and you’ve got 100 iterations, you can reasonably expect to wait for 100 minutes.

100 units of work x 1 minute per unit = 100 minutes elapsed time

But since your program can scale linearly, you can add an additional program and cut the time in half!

(100 units x 1 minute) / 2 processors = 50 minutes elapsed time

Similarly, you can scale up to 4 processors and reduce elapsed time to 25 minutes.  This is perfect linear scaling and with your big math brain, you figure out that you can get a 10X gain by scaling up to 10 processors!

So far, so good.  10x is an order of magnitude and represents a 90% decrease in elapsed time.

(100 units x 1 minutes) / 10 processors = 10 minutes elapsed time

10 minutes is 10% of the original 100 minute elapsed time. 10x gain!

I think the second order of magnitude is a waste of time.  That’s right, it’s not worth going for another 10x gain.

Why?  It costs too much!

Assume that a server costs $1000 and your process will consume the entire processing capacity of a server.  Scaling up to 10 servers costs $10,000.  You reduced processing time by 90% for $10k.

Math is not on your side for the second order of magnitude.  Taking your elapsed time from 10 minutes to 1 minute is another order of magnitude, but it also is 90% of your cost!

You have a perfectly linearly scalable process, right?  So, reducing your 100 minute elapsed time requires 100 servers at $1000 each.  That’s $100,000!  Meanwhile, already achieved a 90% reduction for $10,000.

90% of the gain is achieved by 10% of the investment.  The remaining 10% of the gain requires 90% of the investment!

Pareto was right.  The 80/20 rule applies, but in our case its 90/10.

The chart below shows two orders of magnitude.  You can’t help but notice the point of diminishing returns.  It doesn’t seem worthwhile to go for that second order of magnitude.

100x.jpg

Leave a comment

You must be logged in to post a comment.