What kind of computing does science really need?

Here's a talk by Greg Wilson of the University of Toronto that pretty well captures what I think about how computing in the sciences really needs to be done, rather than the focus on bigger supercomputers.

Other options (Low Res or podcast) available here.

Organic Code Development

I always find it amusing how some of my best coding happens so organically. That is to say, there is no true overarching design usually, just an end goal. Take for instance, my latest creation. I've been working on adding support for wind barbs to the python plotting library matplotlib . I initially tried to design it well, think about where the natural breaks in the code would be and then just build these smaller pieces. This worked for some of the simple things, but in the complexities of figuring out transformations and coordinate systems, it got to be a real bear to work top down. Perhaps things like stub functions and other mock objects might have been able to help here, but that's beyond my software development skill set at the moment.


So out went that approach, and I started with the simple goal of getting a single barb to appear on the plot, using the most direct and manual way possible. Once that was achieved, I added a couple more to the test and made sure that they scaled right, and were positioned properly. From here then some refactoring occurred and things were broken into more logical groups, and manual hard coded things were replaced with more flexible (and arguably) simple code. Here's the (more or less) final result.



I now have 320 lines of commented and documented code ("borrowing" a lot of the documentation and input processing from a similar matplotlib function, quiver). I'm just amused that this kind of procedure always seems to work better for me. I was stuck for days trying to get this working, trying to figure out where to begin. It was so much easier to start with a simple, small goal and build on it. Then again, this might work better for me because I'm a scientist who codes rather than a software engineer/developer. It could also be because every time I do something like this, I'm learning a new API or even an entirely new library. Maybe if I finally developed a deep expertise with something, I might be able to think at a slightly higher level....nah. :)


I think I've really only ever succeeded in designing and implementing one project, my radar emulator . While it turned out well, it's not a design that at this point I'm in love with (though I'm stuck with it at this point). Then again, I started that in 2003, when I was far less experienced. Another project I've been kicking around in my head forever, a simple data visualization program, is persistently stuck in the design phase. Granted, it's predecessor grew organically and became unwieldy, but that was C++, where refactoring is a nightmare. But it did have the benefit of actually existing in code and being (somewhat) useful, which is more than I can say for an idea floating around in my head.

Anyhow, less rambling, more code! On to Skew-T's!

Links for the day (aka. Other people's stuff)

A couple of posts of other people that resonate well with me:
http://www.johndcook.com/blog/2008/07/15/getting-to-the-bottom-of-things/

I think this one hits the nail on the head about multi-tasking and information overload. Also explains why the days that I manage to ignore Google reader and Thunderbird for extended periods of time are my most productive, even though the actual total time spent doing either isn't any less. I probably would see great productivity gains if I managed to restrict how often I look at either of those. I've noticed this summer just how hard it is to get back in a groove on something once you've gotten out, both taking 5 minutes away from intensive coding or taking a multi-day (week?) break from a certain topic. Our brains are like computers in a way, there's only so much you can keep readily accessible.

http://bitecode.co.uk/2008/07/my-views-on-python/

There's not much more I can add here, python just rocks. :) I can't believe there was once upon a time that I did analysis and even generated images using only C (and maybe *shudder* Excel).

Programmatically saving Matlab figures

This problem has been bugging me a long time, and I've finally nailed it. So doing a whole bunch of Matlab processing, and instead of saving each figure by hand using the GUI, which can be tedious after more than a few figures, you'd like to have your code include these steps. (Using python and matplotlib of course, this is simple). Matlab does have the saveas to allow doing this, but it has two drawbacks for me:

1. When I save figures, I usually maximize them so that the spacing looks better and I get higher resolution figures
2. Regardless of the maximization, saveas resizes everything and you get (usually) a bad mix of spacing and font sizes. So many of my figures have labels and things that look like crap, with tick labels overlapping each other and other issues like this. At least, that's my experience using Matlab on Linux. At any rate, saveas does not normally give me figures I want to hand in with a homework assignment, let alone a presentation or a publication.

I have found a couple of nuggets that allow me to programmatically create graphics just like the ones I would have on screen. The first nugget, which solves the 2nd problem of resizing is as follows:

set(fig, 'PaperPositionMode', 'auto')

where fig is your figure handle. You can also just pass gcf. This wonderful little command, according to the Matlab documentation (where this was buried): "...ensures that the printed version is the same size as the onscreen version. With PaperPositionMode set to auto MATLAB does not resize the figure to fit the current value of the PaperPosition." Fantastic.

The second nugget is a way to set the figure size to fill the screen (I found this one in the Mathworks file share through a Google search):

set(fig,'units','normalized','outerposition',[0 0 1 1]);

You may or may not want to save and restore the original value of units around this call. At any rate, with these two commands, I can now create figures with my code that look how I want them without requiring any intervention on my part using the GUI. That's wonderful for making my workflow better.

Delorme Earthmate LT-40 and Linux

I just got the new Delorme Earthmate LT-40. It works pretty well, getting a location lock under windows much more quickly the old version I had (Earthmate USB predating the LT-20). I think it took ~ 5minutes from a completely cold start to get a differential GPS lock. I also noticed last night that it managed to get a 3D lock while I was sitting in my living room. Very impressive, as I only had it plugged in to test that I could actually see it, I didn't actually expect it to work. :)

However, it did not work for me under Linux using the 2.6.24 kernel. I plugged it in, it was recognized by the cypress_m8 usb-to-serial driver and the /dev/ttyUSB0 device was created. However, trying to cat this device resulted in no output. A bit of googling showed these threads relating to the Earthmate LT-20:

http://ubuntuforums.org/archive/index.php/t-220821.html
http://www.mail-archive.com/linux-usb-devel@lists.sourceforge.net/msg52237.html

A few posts to the linux-usb list and I was directed to this patch series:

Cypress_m8 Patch 1: Feature Buffer Fixes
Cypress_m8 Patch 2: Packet Format
Cypress_m8 Patch 3: Get Config Unsafe for LT-20
Cypress_m8 Patch 4: Null Check
Cypress_m8 Patch 5: Limit Baud Rate to <= 4800

Patch #3 is the important one here, but needs the first 2 to be applied easily. I've assembled them together into easily used diffs here. Those looking for a quick fix should simply comment out the call to

cypress_set_dead(port);

at line 408 (at least it is for 2.6.24) of the file drivers/usb/serial/cypress_m8.c.

Thanks go Mike Isely for writing the patches and directing me to them. This series of patches is slated to go into 2.6.26.

So for anyone looking to use their new Earthmate LT-40 (or even LT-20) under Linux, there is hope.