Monday 12 January 2009

Do I need to write software?

[caption id="" align="alignleft" width="240" caption="Photo by Jan Tik"]Photo by Jan Tik[/caption]

In a previous post, we discussed what our project objectives were and how collect them.  We've also considered how to choose a programming language , as well as how to write pseudocode and build a prototype.  But we've missed something.  We've made an implicit assumption and we should pause to consider it.

Do I need a computer program?

This might seem like a strange question for us to ask in a blog about programming, but it fits our aim: to help people to do more good science, more quickly.  Here, this means deciding whether your objectives can be met most quickly by writing some software or by doing the work by hand. 

There is an old computing adage that says, "If you perform an action more than once, write a program to do it." If you are going to perform lots of identical (or nearly identical) actions then a program is probably a good idea because a computer will almost certainly complete the actions more quickly than you would.  And of course, once the program is written it takes you negligible effort to run your program.  You can set it going and turn your attention to something else.    This will also reduce the number of random one-off errors, as the computer will slavishly perform the actions you give it. This means that the consistency of your results will be higher than if you'd done the work by hand.  (of course it will also mean that any errors are performed on all the inputs.  But a program can be changed and re-run to fix this.)

However, it takes a certain amount of time to develop a program in the first place. Could this time be spent processing the data by hand, or partially by hand? You must decide whether the upfront costs in development and maintenance time is worth spending especially if the problem to be solved, or data to be processed, requires a lot of decision making rather than a lot of repetitive actions. 

Even if you have a problem that requires a lot of data processing that must be done by computer, do you need to write a new program to do it? Has somebody else already written a program that you can use, or several programs that you can chain with a little extra ‘glue' code?  You may be able to save all the time of development if someone has already done it for you!

On this basis it is well worth searching to see if this is indeed the case.  The Internet has made this searching process far easier but ferreting out useful programs can be a hit or miss proposition. The Open Source movement has also made it far easier to get programs with their source code, which opens up the possibility of changing an existing program to get exactly what you want. Beware though, because beyond a certain point it becomes more effort to modify existing code that it would be to write your own from scratch.  The reason for this is that as program size increases, the time required to fully understand a program rapidly outstrips the time taken to implement another program with similar objectives.  A reasonable rule-of-thumb (based on the empirical and anecdotal evidence of developers over the last few decades) is that if you need to change more than about 20-25% of a program then you are better off writing a new one.  However, if you can achieve your objectives with less (or no) changes, then modifying/using existing code may well be a viable solution.

A key consideration here is to understand what tasks computer are (and are not) suiting to doing, because of the implications for how long it will take you to build the necessary software.  Computers are very good at performing logical and mathematical operations; they can do this reliably and much, much faster than doing them manually.  This even extends to lots of ways of solving problems that don't necessarily appear to be mathematical:  Searching, sorting and processing are all bread-and-butter problems for computers.

There are also lots of problems that computers aren't (currently) very good at. They are mainly problems that require a lot of decision-making based on partial, or contradictory, data.  Many abilities that humans take for granted are hard for a computer to mimic: face-recognition (and pattern matching in general), learning and understanding of human speech/language, to name a few.  While these tasks can be performed by a computer, they are far from trivial to implement.   In general if the problem you need to solve requires a lot of decision-making based on rules and knowledge that aren't very clear then writing a program will probably be very difficult.

You can write a program to perform almost any task (some may be far harder than others as, as covered previously) but to be useful the cost of producing the program must be covered by the time saved using it. If a task that is performed repeatedly, especially if it is a mathematical or logical task, then the cost of producing and maintaining the program will be recovered quite quickly. Conversely, even if a program is run only once, if the work it does is beyond the scope of a person to perform in the time it takes to develop and then run the program then it is still a worthwhile investment of time.

So consider this, this next time you're planning a project to get something done.  The question is easy to pose, so asking yourself "do I need a computer program?" is a sensible step in any project, even when the answer is clearly "yes".

No comments:

Post a Comment