A Call for Generic Sensor Format (GSF) files

I am putting out a call for GSF files to help test out the GSF-reader I am writing.

I have tested my GSF-reader with files I have found around CCOM; however, they all seem to be generated by CARIS, which only outputs certain records within the GSF file.

I am particularly interested in GSF files containing one or all of the following records:

1) BRB Intensity subrecord with time-series intensity records for each beam

2) Simrad, or non-Reson, sonar-specific subrecords

3) Sector Number, and Detection Info subrecord arrays

4) Reson-specific Quality Flags array

5) HV Navigation Error record

If you have such a GSF file that you can send me, please email me.

Cheers!

Installing a Zagg Invisishield on iPhone 3G

I am posting this because a lot of people have posted about how hard it is to install a Zagg InvisibleShield on the iPhone 3G. I just put mine on and found it to be relatively easy, so I thought I would share a few tips in case it benefits anyone else.

1) Some folks complain that for best results you have to leave the phone off so it can "set up" for 12 to 24 hours to let the product cure. I simply installed mine at night so it can do most of its setting up while I am asleep.

2) Use the shield spray liberally! I ended up using the entire bottle just putting on the back piece because I had to realign it once and I wanted to keep my fingers moist. You can refill the bottle up to three times, so do not worry if you use a lot.

3) The corner pieces, were indeed, somewhat hard to do. They sort of stuck where they fell when I first placed the piece on the phone, so I had to peel them up, respray, and then press down again. Letting them set up a bit definitely helps as does the "palming method".

4) THE BEST ADVICE I found online was to stretch a piece of saran wrap over the top and bottom of the phone to help keep the corner pieces down while it cures. I did this for about 10 - 15 minutes only as I did not want to trap any moisture from the spray, but it did make a difference. My corners are all now nice and smooth.

The squeegee really did get most of the bubbles out, but some very small ones will remain. Zagg says these will work themselves out as the products settles down over the next day or two. I have been checking periodically as I let the phone sit, and indeed some of the small guys are already gone.

So far so good. I really like the shield and the phone looks great. I will still use my case for drop protection, but now I can rest assured about scratches.

I originally had an iFrogz Luxe case that I LOVED, until I realized it trapped stuff between the case and the phone, so the chrome and the back of the phone got scratched up a bit. I was always careful with the phone too, keeping it in a sunglasses pouch when in my purse, or keeping it in a separate pocket when in my book bag, so the scratches really annoyed me. Also, one of the tabs broke off after only a couple weeks, so I was worried about the case staying together if I were to drop the phone.

I now have a SwitchEasy Rebel. The case is not as nice looking or feeling as the iFrogz (in my opinion), but the protection seems to be way better and I am not worried about pieces snapping off.

Here are some pics of the phone about an hour and a half after putting on the Zagg:







UPDATE: Just FYI, by the next day, all micro-bubbles were gone, all the corners were smooth, and it looked awesome!

Follow Healy in Google Earth!

Thanks to Google and Kurt Schwehr, we can now follow Healy in Google Earth. Kurt's GeoRSS feed of the Healy's Aloftcon images is what is feeding the Google Earth visualization.

Check out Kurt's blog to see how to view it: Kurt's blog.

Healy Photostream on Flickr

The USCGC Healy has a photostream up on Flickr! Now we can all follow along as they traverse the Arctic! Check out the recently updated stream to see pics of a polar bear spotted just a couple days ago!

http://www.flickr.com/photos/cutterhealy/

CARIS GSF: Update

So CARIS sure is speedy with their replies. I normally always hear back within a few hours, and sure enough, I had a response waiting for me this morning!

As with most programs that can write to GSF, CARIS simply implements the C-code available for download from SAIC's website. The extra-padding issue could be a bug in the original C-code then. Perhaps the 0 - 3 byte padding allowance listed in the specification document is not actually enforced.

However, I also wonder if this issue is related to the fact that CARIS only writes out sonar-specific sub-records for RESON systems. The GSFs I generate from RESON systems do not have the extra padding, but the SIMRAD ones do. I wonder if something about how the C-code writes out the records causes there to be extra bytes when the sonar sub-record it missing.

Maybe it is supposed to signify an unknown sonar type with an empty subrecord?

If someone happens to stumble along this post that has had a similar issue, feel free to drop me a line.

CARIS GSF and the Mysterious 2 Extra Bytes

I have been plowing along trying to get my GSF-reader I am coding up in Matlab to work with a variety of different GSF files (diff. versions as well as files from different GSF-generators). This morning I got my code to work with some CARIS-generated GSF files that were giving me a bit of a problem.

GSF records are often packed so that the total record size is an integral multiple of 4, so I wrote a little clean-up statement in all my sub-routines that will automatically read any extra characters needed to pad the particular record.

This one group of CARIS GSF files, however, keep crashing the reader. Within the Bathymetry Ping Record, I kept getting sub-record types of 0, which do not exist. I was at the end of the ping subrecords for the first ping in the file, and I had padded the record to be divisible by 4. The problem was that the total size of the ping record (including all sub-records) was 868 bytes, and even with the padding, I was only reading 864. Where were the extra two bytes coming from?

I ended up opening the GSF file itself and looking at it in Hex mode. This can be done using UltraEdit on Windows (automatically opens binary in Hex format) or by using AquaMacs on the Mac. I used ftell(fid) in Matlab to tell me where in the file I was when the error occurred and then went to that spot in the Hex mode file (in UltraEdit: Search -> GoTo line/Page and type in the number of bytes to move forward. in AquaMacs: C-u #bytes and then hit the right arrow key).

What I saw when I did this was that, sure enough, the record had extra padding. The total number of bytes in the ping sub-records for each ping is 862. To make this divisible by 4, you only have to add 2 bytes (thus my initial reading of 864 bytes). Looking at the binary in Hex mode, however, I was able to see that the record was actually padded with 6 bytes, bring the total up 868. Once I made this correction, the code ran happily and I was able to read the whole GSF file successfully.

I am still not sure what is up with the extra two bytes though, and I am curious to see if this may be the cause of GeoCoder and Fledermaus crashing when trying to read these files. I am going to submit this to CARIS and suggest they take a peak at it themselves. This bug only seems to occur in GSF files exported from CARIS 6.1 (I have tested it with different files). GSF files exported using CARIS 5.4 do not have the extra padding.

GSF-reader now working!!

Okay, so I am really excited because my GSF-reader that I coded up in Matlab is now working!! It is not fully-functional yet (I still have to add some sonar-specific readers, and some other optional record types) but it works for one of the sample CARIS-generated GSF files I have.

One of the things holding me up was the fact that I did not realize records were padded with extra bytes to ensure the record size in bytes was divisible by 4. I am not sure why it matters if it is divisible by 4, but apparently it does to GSF.

My next issue to tackle is a Reson sonar-specific quality flag indicator that is written in bits, not bytes, and uses all kinds of bit shifts and masks (joy!). This record is no longer used, but some older GSF versions will include them. Once that is tackled, some of the other GSF files should start working as well.

Also, if I want others to be able to benefit from this work, I should probably eventually convert it over to Python. Having a Python-based GSF reader would be pretty sweet.

There already exists C-code of course that does all this, but I want to have something that generates separate records for the data so that I can play with the ping depths for example, or the backscatter intensities, in a familiar environment. The C-code is really written so that it can be incorporated into other programs. By writing a reader myself, I not only gain a better understanding of the data and how GSF stores them, but I read them into a program where I can readily perform mathematical analysis on them.

Methane Seeps from Arctic Seabed

Check out this discovery of deep-water methane seeps off Norway by a joint British-German research team:

http://news.bbc.co.uk/2/hi/science/nature/8205864.stm

below is a sonar image of some of the seeps

Writing a GSF-reader in Matlab

I recently ran across a couple issues with some GSF (Generic Sensor Format) files I have, so I decided to attempt to write a GSF reader in order to see what, exactly, I was dealing with. I have never written any kind of binary format reader before (though I did tinker around and add features to one written in Python), so I sort of jumped in the deep-end with this. I decided to use Matlab, since I am most comfortable with its scripting language at this point in time.

Here is a valuable lesson I just learned: Never take the binary specification file at face-value. Typos happen, and they certainly happened here. I have the GSF v. 3.01 specification file available from the SAIC website, and have been following it to write my reader. Everything has been going fine for the most part until I reached the attitude record.

According to the specification, the attitude record should look like this:
-->

Field Name	Description	Field Type	Count
NUM_MEASUREMENTS	Number of attitude measurements in this record.	I	2
ATTITUDE_TIME	Array of attitude measurement times	I	N*2*4
PITCH	Array of pitch measurements	I	N*4
ROLL	Array of roll measurements	I	N*4
HEAVE	Array of heave measurements	T	N*4
HEADING	Array of heading measurements	T	N*4
Attitude Record Size:			Variable

Should be pretty straightforward, so in my Matlab code I wrote the following function:

function [Num_meas, ATT_Time, ATT_Pitch, ATT_Roll, ATT_Heave, ATT_Heading] = readATTrecord(fid)
%% This function reads the Attitude record in a GSF file.

%% Element          Bytes   Type         Description
% Num_Measurements    2      int      number of attitude measurements (N)
% Attitude_Time       N*2*4  int      array of attitude meas. times
% Pitch               N*4    int      array of pitch measurements (hundreths of deg)
% Roll                N*4    int      array of roll measurements (hundreths of deg)
% Heave               N*4    char      array of heave measurements (hundreths of deg)
% Heading             N*4    char      array of heading measurements (hundreths of deg)

Num_meas = fread(fid,1,'int16');

for i = 1:Num_meas
ATT_Time(i,:) = fread(fid,2,'int32');
end
for i = 1:Num_meas
ATT_Pitch(i,:) = fread(fid,1,'int32');
end
for i = 1:Num_meas
ATT_Roll(i,:) = fread(fid,1,'int32');
end
for i = 1:Num_meas
ATT_Heave(i,:) = fread(fid,4,'char');
end
for i = 1:Num_meas
ATT_Heading(i,:) = fread(fid,4,'char');
end

This returned all sorts of funky data that made no sense, most notably the negative times and pitch angles of 120 degrees (that would be worse than the perfect storm!). I decided to look at the GSF library also available on the SAIC website in order to check out the source code. In the library directory, there is a file called GSF_dec.c, a C-code source file for decoding the GSF binary format. I looked up how the Attitude record was decoded in C, and saw this:

gsfDecodeAttitude(gsfAttitude *attitude, GSF_FILE_TABLE *ft, unsigned char *sptr)
{
unsigned char  *p = sptr;
gsfuLong        ltemp;
gsfuShort       stemp;
gsfsShort       signed_short;
int             i;
struct timespec basetime;
double          time_offset;

/* First four byte integer contains the observation time seconds */
memcpy(&ltemp, p, 4);
p += 4;
basetime.tv_sec = ntohl(ltemp);

/* Next four byte integer contains the observation time nanoseconds */
memcpy(&ltemp, p, 4);
p += 4;
basetime.tv_nsec = ntohl(ltemp);

/* Next two byte integer contains the number of measurements in the record */
memcpy(&stemp, p, 2);
p += 2;
attitude->num_measurements = ntohs(stemp);

>and a little farther down I saw:

/* Now loop to decode the attitude measurements */
for (i = 0; i <>num_measurements; i++)
{
/* Next two byte integer contains the time offset */
memcpy(&stemp, p, 2);
time_offset = ((double) ntohs (stemp)) / 1000.0;
p += 2;

LocalAddTimes (&basetime, time_offset, &attitude->attitude_time[i]);

/* Next two byte integer contains the pitch */
memcpy(&stemp, p, 2);
signed_short = (signed) ntohs(stemp);
attitude->pitch[i] = ((double) signed_short) / 100.0;
p += 2;

/* Next two byte integer contains the roll */
memcpy(&stemp, p, 2);
signed_short = (signed) ntohs(stemp);
attitude->roll[i] = ((double) signed_short) / 100.0;
p += 2;

/* Next two byte integer contains the heave */
memcpy(&stemp, p, 2);
signed_short = (signed) ntohs(stemp);
attitude->heave[i] = ((double) signed_short) / 100.0;
p += 2;

/* Next two byte integer contains the heading */
memcpy(&stemp, p, 2);
attitude->heading[i] = ((double) ntohs(stemp)) / 100.0;
p += 2;

The C-code shows that the specification table was completely wrong! There is only one reference time for the attitude reading, and the rest of the times are simply how many seconds past that reference time the rest of the measurements take place. Furthermore, the specifications stated that the Heave and Heading fields were text (something I thought was weird anyway) and in the C-code we clearly see they are integers. The specification also stated everything was 4-bytes, but it is actually only 2 (16 bits). This is very frustrating that the specification and the C-code do not match, especially since they both come from the same source.

At any rate, I am now able to write a working function that can be called in my main code:

function [ATT_Time, Num_meas, ATT_Offset, ATT_Pitch, ATT_Roll, ATT_Heave, ATT_Heading] = readATTrecord(fid)
%% This function reads the Attitude record in a GSF file.

%% Element          Bytes   Type         Description
% ATT_Time            2*4    int      time of attitude meas.
% Num_Measurements    2      int      number of attitude measurements (N)
% Time_Offset         N*2    int      time offset in seconds
% Pitch               N*2    int      array of pitch measurements (hundreths of deg)
% Roll                N*2    int      array of roll measurements (hundreths of deg)
% Heave               N*2    int      array of heave measurements (hundreths of deg)
% Heading             N*2    unit     array of heading measurements (hundreths of deg)

ATT_Time = fread(fid,2,'int32');
Num_meas = fread(fid,1,'int16');

for i = 1:Num_meas
ATT_Offset(i,:) = fread(fid,1,'int16');
end
for i = 1:Num_meas
ATT_Pitch(i,:) = fread(fid,1,'int16');
end
for i = 1:Num_meas
ATT_Roll(i,:) = fread(fid,1,'int16');
end
for i = 1:Num_meas
ATT_Heave(i,:) = fread(fid,1,'int16');
end
for i = 1:Num_meas
ATT_Heading(i,:) = fread(fid,1,'uint16');
end

Mobile Blog Photo Test

I wanted to test Google's mobile blogging with a photo, so here is a
pic I snapped with my phone of a weird bug on my porch.

Mobile Blogging

So google blogger lets users blog on the go in a couple different ways. You can use SMS messaging (standard text messages), MMS messaging (multimedia messaging), or your email. Since I do not have a text plan, I opted for the latter. You set up an email address for your blog (under your existing blog's settings) and then just email your posts. You can select to have mobile posts immediately published or saved as drafts.

So here is my first post sent from my phone. I can also see this as a great way to blog from sea, when you often have email access but not full web capabilities.

Importing Music into GarageBand and Making Ringtones

Okay, so I do not know about you, but I was somewhat annoyed to find out that not only will iTunes only let you create a ringtone from music you purchase from the iTunes Store, but that they still charge you an additional 99 cents to convert the song sample into ringtone format.
Total rip-off! Especially considering you already own the song!

Here's a free workaround: Kurt suggested that I try GarageBand, a music creation software that comes free with the Mac. It took a little futzing around, but I have now successfully (and freely) created a ringtone from Michael Jackson's The Way You Make Me Feel.

Here are the steps:

1) Open GarageBand and then drag-n-drop your music file (.mp3 works for sure, not sure about AAC) in it. The file should open right up as long as it is not DRM, so no purchases music from iTunes (though supposedly if you burn them to disc and then re-import, it loses the DRM)

2) Click on the scissors icon on the bottom left to bring up the track editor, and from there, click on the loop arrow icon to enable the loop feature. GarageBand ringtones are limited to loops of 40 seconds or less.

3) You should now see a second time bar appear just below the first at the top of the window (click the screen grab below to see what I am talking about). Just click and drag within the bar to select a portion of the song. The time bar will turn yellow where you select.


4) Just hit play to preview your selection. GarageBand will continuously loop through the selection, so you can get a pretty good idea of how your ringtone will sound.

5) Once you are happy with your selection, click on Share in the Menu Bar and select Send Ringtone to iTunes. It will convert the song for you and automatically open iTunes, where your song is stored in a new Ringtones folder. Now all you have to do is transfer it to your device.