Archive for November, 2008
23rd Nov 2008
Two “orders of magnitude” is one too many
An “order of magnitude” gain in efficiency, whether its a business process or computer program, is something to strive for, but two orders of magnitude, despite sounding cool, is one too many.
Why?
Assume you have a perfectly linear process — say, a computer program processing data – whereby you can add additional processing nodes for parallel processing. If 1 run of your program takes 1 minute and you’ve got 100 iterations, you can reasonably expect to wait for 100 minutes.
100 units of work x 1 minute per unit = 100 minutes elapsed time
But since your program can scale linearly, you can add an additional program and cut the time in half!
(100 units x 1 minute) / 2 processors = 50 minutes elapsed time
Similarly, you can scale up to 4 processors and reduce elapsed time to 25 minutes. This is perfect linear scaling and with your big math brain, you figure out that you can get a 10X gain by scaling up to 10 processors!
So far, so good. 10x is an order of magnitude and represents a 90% decrease in elapsed time.
(100 units x 1 minutes) / 10 processors = 10 minutes elapsed time
10 minutes is 10% of the original 100 minute elapsed time. 10x gain!
I think the second order of magnitude is a waste of time. That’s right, it’s not worth going for another 10x gain.
Why? It costs too much!
Assume that a server costs $1000 and your process will consume the entire processing capacity of a server. Scaling up to 10 servers costs $10,000. You reduced processing time by 90% for $10k.
Math is not on your side for the second order of magnitude. Taking your elapsed time from 10 minutes to 1 minute is another order of magnitude, but it also is 90% of your cost!
You have a perfectly linearly scalable process, right? So, reducing your 100 minute elapsed time requires 100 servers at $1000 each. That’s $100,000! Meanwhile, already achieved a 90% reduction for $10,000.
90% of the gain is achieved by 10% of the investment. The remaining 10% of the gain requires 90% of the investment!
Pareto was right. The 80/20 rule applies, but in our case its 90/10.
The chart below shows two orders of magnitude. You can’t help but notice the point of diminishing returns. It doesn’t seem worthwhile to go for that second order of magnitude.
An “order of magnitude” gain in efficiency, whether its a business process or computer program, is something to strive for, but two orders of magnitude, despite sounding cool, is one too many.
Why?
Assume you have a perfectly linear process — say, a computer program processing data – whereby you can add additional processing nodes for parallel processing. If 1 run of your program takes 1 minute and you’ve got 100 iterations, you can reasonably expect to wait for 100 minutes.
100 units of work x 1 minute per unit = 100 minutes elapsed time
But since your program can scale linearly, you can add an additional program and cut the time in half!
(100 units x 1 minute) / 2 processors = 50 minutes elapsed time
Similarly, you can scale up to 4 processors and reduce elapsed time to 25 minutes. This is perfect linear scaling and with your big math brain, you figure out that you can get a 10X gain by scaling up to 10 processors!
So far, so good. 10x is an order of magnitude and represents a 90% decrease in elapsed time.
(100 units x 1 minutes) / 10 processors = 10 minutes elapsed time
10 minutes is 10% of the original 100 minute elapsed time. 10x gain!
I think the second order of magnitude is a waste of time. That’s right, it’s not worth going for another 10x gain.
Why? It costs too much!
Assume that a server costs $1000 and your process will consume the entire processing capacity of a server. Scaling up to 10 servers costs $10,000. You reduced processing time by 90% for $10k.
Math is not on your side for the second order of magnitude. Taking your elapsed time from 10 minutes to 1 minute is another order of magnitude, but it also is 90% of your cost!
You have a perfectly linearly scalable process, right? So, reducing your 100 minute elapsed time requires 100 servers at $1000 each. That’s $100,000! Meanwhile, already achieved a 90% reduction for $10,000.
90% of the gain is achieved by 10% of the investment. The remaining 10% of the gain requires 90% of the investment!
Pareto was right. The 80/20 rule applies, but in our case its 90/10.
The chart below shows two orders of magnitude. You can’t help but notice the point of diminishing returns. It doesn’t seem worthwhile to go for that second order of magnitude.
Posted by Mark Turansky under 
Business, Technology 
1 Comment »
14th Nov 2008
Best technical definition ever
“ORA-12505: TNS: listener does not currently know of SID given in connect descriptor”
What does that error mean? The site below defines the error eloquently:
“ORA-12505: TNS: listener does not currently know of SID given in connect descriptor”
What does that error mean? The site below defines the error eloquently:
Posted by Mark Turansky under Engineering 1 Comment »
12th Nov 2008
Being an oversexed man in a whorehouse
What can you learn from the consumer confidence index? Plenty, if you’re a fan of Warren Buffet!
Let’s look at this graph for a moment and focus solely on the points where consumer confidence slipped below 70: 1974, 1980, 1982, 1990/1, 2008.
1974
The first big dip on the consumer confidence graph above is October 1974 when the stock market tanked 40% in a single year (why does that sound familiar?). How did Uncle Warren feel at that time? “Like an oversexed guy in a harem.” The quote is from a Fortune article from 1974.
Note that consumer confidence cratered in 1974, touching below 60. Buffet’s feeling was “Now is the time to invest and get rich.”
And what happened after the crash? Up 30% in one year. Up 60% in two years.
1980
Another big dip in consumer confidence in 1980. Lower than 1974. If you were waiting for clear bargains in the stock market, how would you have faired? Had you bought in the panic, you’d have gained nearly 30% in a couple of months.
1982
Consumer confidence gained after the 1980 dip, only to retreat again in the recession of 1982. From nearly any point in 1982 (DOW around 850) to nearly any point in 1983 (DOW around 1250), you’d have gained 47%.
1990
Another hit to consumer confidence in 1990-1991. Bill Clinton campaigned with the theme “It’s the economy, stupid!” From the panic to the recovery by Clinton’s inauguration day (a little over a year), you’d have gained33%.
NOTABLE CRASH - 1987
Black Monday (again in October!) saw a 22% decline in prices. Your shares would have recovered in two years, but buying in the panic yielded 16% in one year and 50% in two years.
RECESSION 2002
We all know the market hit record highs in recent years, but simply focusing on the panic (again, October!) yielded 25% in a year and lots more if you held for another 5.
TODAY!
It’s October again and the market is down 40% year over year. The world is ending! Panic ensues! Everyone get out of the market before all the banks melt!
Consumer confidence is as bad as it was in 1974, when that decade saw oil shocks and a renewed focus on ending dependence on foreign oil. Sounds familiar. History repeats itself.
But “it’s different this time!” Maybe. Yes, we’ve lived beyond our means as a nation for twenty years. Yes, the housing bubble is causing a global financial mess that requires us to recapitalize our banks, but the Asian Tigers did it in the late 90s and Sweden did it in the early 1990s. It costs money and it’s painful, but we’ll recover. The dollar, incidentally, has surged during this global crisis. Traders are moving towards a currency and economy they are confident will recover.
We might be good and truly screwed or this might be another deep panic by the masses. It might be a good time to be an oversexed man in a whorehouse.
What can you learn from the consumer confidence index? Plenty, if you’re a fan of Warren Buffet!
Let’s look at this graph for a moment and focus solely on the points where consumer confidence slipped below 70: 1974, 1980, 1982, 1990/1, 2008.
1974
The first big dip on the consumer confidence graph above is October 1974 when the stock market tanked 40% in a single year (why does that sound familiar?). How did Uncle Warren feel at that time? “Like an oversexed guy in a harem.” The quote is from a Fortune article from 1974.
Note that consumer confidence cratered in 1974, touching below 60. Buffet’s feeling was “Now is the time to invest and get rich.”
And what happened after the crash? Up 30% in one year. Up 60% in two years.
1980
Another big dip in consumer confidence in 1980. Lower than 1974. If you were waiting for clear bargains in the stock market, how would you have faired? Had you bought in the panic, you’d have gained nearly 30% in a couple of months.
1982
Consumer confidence gained after the 1980 dip, only to retreat again in the recession of 1982. From nearly any point in 1982 (DOW around 850) to nearly any point in 1983 (DOW around 1250), you’d have gained 47%.
1990
Another hit to consumer confidence in 1990-1991. Bill Clinton campaigned with the theme “It’s the economy, stupid!” From the panic to the recovery by Clinton’s inauguration day (a little over a year), you’d have gained33%.
NOTABLE CRASH - 1987
Black Monday (again in October!) saw a 22% decline in prices. Your shares would have recovered in two years, but buying in the panic yielded 16% in one year and 50% in two years.
RECESSION 2002
We all know the market hit record highs in recent years, but simply focusing on the panic (again, October!) yielded 25% in a year and lots more if you held for another 5.
TODAY!
It’s October again and the market is down 40% year over year. The world is ending! Panic ensues! Everyone get out of the market before all the banks melt!
Consumer confidence is as bad as it was in 1974, when that decade saw oil shocks and a renewed focus on ending dependence on foreign oil. Sounds familiar. History repeats itself.
But “it’s different this time!” Maybe. Yes, we’ve lived beyond our means as a nation for twenty years. Yes, the housing bubble is causing a global financial mess that requires us to recapitalize our banks, but the Asian Tigers did it in the late 90s and Sweden did it in the early 1990s. It costs money and it’s painful, but we’ll recover. The dollar, incidentally, has surged during this global crisis. Traders are moving towards a currency and economy they are confident will recover.
We might be good and truly screwed or this might be another deep panic by the masses. It might be a good time to be an oversexed man in a whorehouse.
Posted by Mark Turansky under Business No Comments »
