Writing code for a big scientific collaboration

[caption id="" align="alignleft" width="300" caption="Photo by ralphbijker"][/caption]

One of the striking thing about scientific software is the range of different contexts in which it's needed.  Scientists need quick-and-dirty scripts to process their data and plot their results; they need prototypes so that they can experiment with new statistical techniques; and they sometimes need to build new software tools that they'll use again and again in their research.  While a lot of this will be for their own personal research, sometimes the Scientist-Programmer finds themselves developing software as part of a large scientific collaboration.  This has some particular requirements.

Sleeping on the problem...

Some problems are difficult to solve. You can spend all day trying to find a solution and only end up with a list of approaches that don't work, plus a strong need for the alcoholic beverage of your choice. One possible way forward is strangely counter-intuitive: stop working on the problem.

Not for ever (obviously). But long enough to take a break and give your brain a rest. If you've been beating your head against a metaphorical brick wall all day (or week. or month...) and still haven't solved your problem, it's unlikely that a bit of a break will slow you down much. And it might just help. It can be as short a break as stopping for a cup of tea/coffee. You could leave the task until tomorrow, so your brain gets the night off. Or you could even leave the project for weeks (or more), if you really want some separation.

This has a number of advantages. Firstly, it gives your poor overworked (and often frustrated) brain a rest. But there's also a more subtle effect. There are psychological studies (and apologies that I don't have the references to hand) that suggest that problems can become more easily solved if you take a break. The suggestion is that your brain continues to work on the problem subconsciously while you are doing something else, so that when you return to it you might have new insights that your (now-rested) brain can work with. Anecdotally, I can relate to this. I hate leaving a problem unsolved, but sometimes when I've forced myself to leave it until tomorrow, I find a fresh approach and a good night's sleep lead to me solving the problem very quickly the next day.

There are lots of examples where this can be useful. Trying to understand the meaning of your latest set of experimental results. Solving a particularly knotty piece of mathematics. Finding that invisible bug in your code. Or how best to write that troublesome paragraph in your latest paper.

Happily, this strategy is easy to try (provided you can exercise a little willpower to let go temporarily of the problem that's been bugging you). So the next time you're wrestling with a problem and find you're not winning, consider giving up for a while. Take a break. Eat lunch. Get a good night's sleep. Then come back and see if you can't solve the problem.

The 'programming for scientists' link montage

Image via Wikipedia

Continuing our theme of gathering together useful stuff for the scientists-programmer we present a list of interesting, useful, informative and possibly fun links. Enjoy!