Sunday, February 22, 2009

Excel Evils

I'm a big fan of using the best tool for the job. The old adage that says when you only have a hammer everything looks like a nail explains why so many people use Excel for tasks beyond its capabilities. This article provides some facts for what I've been seeing anecdotally for years.
  • "Spreadsheet errors are probably more prevalent than most users realise. A 2005 paper by Jonathan P. Caulkins, Carnegie Mellon University, refers to earlier studies which suggest that the (simple) average proportion of spreadsheets with errors was 46%. A further analysis of this literature by Panko suggested that even those numbers might understate the problem, since not all audits find all errors. Out of 8 studies auditing a total of 421 spreadsheets using better methodologies, only one study had an error rate below 21%. According to Panko, a weighted average of published spreadsheet audits since 1995 suggests that 94% of spreadsheets had errors. Even if these stunning rates overstate the problem in practice, it is hard to avoid the conclusion that errors in spreadsheets are common. "
The question is, what can replace Excel? I've been writing one-off pieces of software to replace Excel for years. The problem is that Excel has become so ingrained in many users workflow that even if I provide all of the options they used in Excel they still want the ability to export the data back to Excel.

It is time to move past Excel (and thus spreadsheets) and into tools that treat data as data and calculations as calculations and not confusingly mix the two as is often the case with Excel. Enterprise mashups are going in the right direction, but still don't give users the same flexibility as Excel. So what do you think will eventually replace Excel as the defacto analyst tool in the enterprise?

The Lazy Programmer: Introduction


1. averse or disinclined to work, activity, or exertion; indolent.
2. causing idleness or indolence: a hot, lazy afternoon.
3. slow-moving; sluggish: a lazy stream.
4. (of a livestock brand) placed on its side instead of upright.

Nowhere in the definition above does it mention anything about a disinclination to think. In my mind "The Lazy Programmer" is not a derogatory term. In fact the best programmers are lazy. They would rather think before acting in order to prevent more work later.

Those programmers who are not lazy often code themselves into corners. They are too busy writing code to stop and think about what they are doing. They are the same programmers who get married to one programming language, one tool, or one methodology. They are the same ones who will copy and paste the same piece of code all over a program when a class or library should have been refactored out.

The Lazy Programmer will be an ongoing section where I will pick one software development topic and explain what it means to be lazy.

Hello World

The typical starter blog post. I went through many names before finding "The Pensive Programmer." It's quite annoying the number of abandoned blogs out there that are occupying many good names.

So what is this blog about?

First, I plan to write about my ideas on anything and everything that has to do with the creation of software. I hope that my ideas can spur on further discussions. I have only been writing software professionally for a little over 10 years and know enough to know that there is a lot I do not know.

Second, this blog will be a place where I write about interesting problems that come up at work and how I solved them. Hopefully these answers can be helpful to others and through discussion I can find even better solutions to the problems I come across. I come across various blogs all of the time that provide me with solutions to problems and it is time I give back. Knowledge wants to be shared.