Sunday, June 29, 2008

Perls, Pythons, and Rs...

I am not a coder, plain and simple. I may work in Matlab some, but even that involves a lot of trial and error for me. What would take some of my friends 5 minutes can take me an hour or more as I muddle my way through function calls and syntax. I am definitely keen to learn though. In my geodesy class we had a semester-long project in which we became a virtual GPS receiver and had to solve for different position fixes (basically using multiple iterations of a least-squares analysis). Most people pick Excel to do it in, including the prof. All that copy and paste and only one line in which to type your equations just gets me, and I cannot do it. I decided on Matlab. I definitely struggled with it a lot, but it was relatively easy to debug when there were issues, and in the end it actually worked.

Now one of my other profs is trying to sway me towards R instead of Matlab. The draw of R is that it is freeware, highly supported by users, and completely object-oriented. Apparently it is rapidly becoming the main coding software of statisticians. The drawback, for me obviously, is switching over and starting from scratch all over again. Other profs tell me to stick with Matlab, it is more mainstream and therefore more widely-accepted. I wonder though, which is better? Perhaps R is more intuitive, which would certainly be nice...

Perl versus Python is another deciding point for me. I recently worked with some Python code involving the same tide data from the blog below. Python seems relatively straightforward on first glance and is pretty powerful. It certainly ran through an entire month of 12-second tide data and converted all the raw temperatures, water levels, and time stamps in mere seconds. Mightily impressive. Then someone told me about Perl. It is also a great language and can do the same things as Python. If you look it up online, people rant and rave about both. A lot of folks seem to think that while Python is somewhat easier to script in than Perl, Perl is still the better choice. Python appears to be more user-friendly, but Perl can work with other programming languages (such as C++) better than Python. Also, it seems that while Perl can do everything Python can, the vice-versa is not necessarily true. My question is for someone who really has no experience with either, but wants to learn one to help with data processing, which one is better? Should I go for the one that is easier to use, or the one that has more compatibility? Does it even really matter for what I will be doing? How much coding will analyzing full-waveform lidar data even require?

Blah, why are there so many options?


  1. We've found that matlab and python have quickly become the main two dynamic languages. Matlab is mainly used due to it's more friendly interpreter, python is used for anything we need to repeat, do in batch or be deployed.

    Lately, the biggest task has been tide (residual) forecasting. The experiments tend to be done in matlab then the models developed are deployed in python.

    We've found that python tends to encourage better coding practices - using functions and better reuse of common code. There are probably a few reasons for this:

    1) python code tends to be written having already implemented the matlab version.
    2) there is generally a greater expectation of the lifetime of python code.
    3) namespaces, modules and the pythonpath in python tend to make organising code simpler. In matlab we often struggle to organise code and move it from project specific to "common" code. Taking work over from someone else in matlab tends to be trickier as their toolboxes and search paths need to be replicated.

    These issues may be resolvable in matlab, but with the skills we have in house, we haven't found a good solution yet.


  2. Thanks, Joe.
    I am definitely going to stick with Matlab over R right now. I had not even thought of Python versus Matlab. I just assumed that Matlab would be the more powerful statistical software and Python the more powerful coding and automating language. I am not familiar with any of Python's mathematical processing skills outside of simple plus, minus, and multiplication functions.
    Can it handle things like fast fourier transformations?
    Do you have any familiarity with Perl?
    Also, would be great to know who "We" is. Do you work at a university?

  3. Using numpy and scipy (two installable modules), python has a great set of numerical libraries. The syntax for these feels very much like matlab and has good support for things like fft.

    The first google page for "numpy fft" looks interesting. Particularly with ipython (a nice interactive prompt), which I have yet to try.

    I've had a little (a long time ago) experience with perl, but found python much much simpler. All of them have a similar level of power, in general you can do anything with any of them.

  4. I tried to come up with some useful things from my experience: thoughts on choosing which programming languages to use

    Another point is that ESRI's ArcGIS comes with python and you can supposedly write python from within Arc, but I have yet to try out this capability.

  5. Another post-response: