Friday, September 10, 2010

station goes offline, and is revived (yet again)

On August 16th, 2010, Jim Hendee (CHAMP principal investigator) asked about our automated status reports, that were showing an absence of data from Port Everglades.  I first replied briefly:
Port Everglades transmissions went offline on August 4th and have not resumed.  Probably a transmitter/GPS problem, but I'm working with the data downloads now and I'll send an update if/when I learn more. 
And then I followed up with a more detailed report:
I've poked into the data a bit more, so here's a fuller picture of what's going on with the Port Everglades station.

First of all, it's offline.  Its last transmission was at (local) noon on Wednesday, August 4th.  We know it's a transmitter-only problem because Jack (by chance) visited the station on Friday, August 6th and collected the latest memory card, so we can look at two more days' worth of data since the failure.

And it's worth saying that, as far as I know, FACE doesn't need satellite transmissions for its work.  I believe Jack uses the 6-minute granular CT data that I extract from the memory cards, not the near-real time transmissions.  Of course it's good to have the near-real time stuff so we know immediately if the equipment fails, but it's not (as far as I know) a must-have for FACE.

On the ICON side, we prefer to have the satellite transmissions working, because that's kinda what we do.  We track our uptime statistics and we feed our numbers to NDBC, so we're generally happier if the transmitter is working.

Anyhow, PVGF1 uses one of our old SAT-HDR-GOES transmitters, and it's been in steady decline since the end of April.  It seems like it's a problem with the GPS subsystem.  The old SAT-HDR-GOES can't transmit unless it gets a GPS fix before every transmission (by contrast the new TX312 will continue to transmit for one month even if all of the GPS satellites were to simultaneously fall into the ocean).  In April we started seeing GPS timeouts when it reached started to go 5 minutes without a fix (normally it takes less than a minute).

By the end of May, these timeouts were happening regularly, about once a day.  The odd thing is that a regular pattern developed where transmissions would fail for three or four hours every day beginning at about 1 PM local.  If I had to guess, I might say that the problem is exacerbated by higher temperatures in the box?

On August 4th the error codes switched from a GPS error to a failsafe error.  This means that the transmitter has probably kicked into failsafe mode again.  I think Jack knows how to reset the failsafe on the transmitter, and he should try doing this the next time he visits.

But the big picture is that the transmitter itself is probably failing.  One of the old SAT-HDR-GOES that we had at Molasses Reef failed in much the same way.  We have four of these guys in all -- one running at Molasses, one failing at Port Everglades, and the other two in my office but I haven't been able to make either of them work, I think they are both dead.

So our options are:
  • reset the failsafe on the transmitter and hope it works a while longer
  • allow the transmitter to fail and just get data by memory card
  • replace the transmitter with a TX312
  • something more exotic, maybe use the cellular modem once it's freed up
Jack visited the station on September 10th, 2010, and then reported:
I did the  transmitter reset today 9/10/10 at about 13:00 EDT.  Hope it works.
And indeed Jack's action was successful, as I confirmed by email later that same day:
It's transmitting, at least for now.  It's got two full transmissions so far.  I'll cross my fingers and hope it lasts a while!