some transmission problems

In an email written May 27th, 2010, I wrote:

I've been updating my data spreadsheets and I noticed that the Port Everglades station has been dropping more satellite transmissions lately.  The problem is intermittent and from the diagnostics it is related to GPS acquisition timeouts.  So either the GPS subsystem is failing in the old SAT-HDR-GOES (bad) or there's a problem with the GPS antenna itself (not as bad).

The problem seems to have begun at the end of April (on April 27th, more or less).  This doesn't coincide with any of Jack's visits and it doesn't follow very closely on the April 16th work, so it doesn't seem like it's a result of moving things around in the box.

We dropped 6 records in March (which is normal), 11 records in April, and 23 records so far in May.  So the problem might be getting worse but it's still not serious.  And you could argue that the near-real-time transmissions are of secondary importance for this site since Jack (I think) depends more heavily on the 6-minute datasets that we retrieve from the memory cards.

Anyhow, something to keep an eye on.  Keep in mind that we don't have any more working SAT-HDR-GOES units.  We do have some GPS antennae but I can't be sure if they're functional without a SAT-HDR-GOES to test on.

CT failure and swap, programming correction

A newly-calibrated CT was deployed on March 22, 2010, and within four days it had failed.  It was later discovered to have flooded and was discarded as irreparable.  It is not certain why it failed.

In any case, plans were quickly made to repeat the events of March 22nd, when the entire system was powered down so that the CTs could safely be swapped underwater.  This was scheduled for April 16th, 2010.  At the same time, a new program was updated to fix a configuration issue with the transmitter (it had been reporting on the wrong PID, a data-processing inconvenience) and add some more granularly reporting data tables (5-second and 30-second averages, maxima and minima).

This time the new CT worked fine, and the programming/transmission changes had their intended effect.

CT swapped, barometer added, programming updated

A view of the PVGF1 CT, sticking up at center with cable emerging from below.

Given that this station's first CT had been deployed on March 19th, 2009, plans were made to swap it out in March of 2010.  The CT's cable was arranged in a buried conduit running from the station's electronics box to the water's edge and then along the channel's bottom to the base of the channel marker it was mounted on.  The cable was arranged in such a way that there was not sufficient slack to bring it to the surface, dry it off, and switch sensors without powering the entire system down.  Therefore we coordinated schedules to make sure we had who we needed on site to safely power down the station and verify that it had started correctly afterwards.

We also took this opportunity to add an analog barometer to the system, to settle any questions about the reliability of the barometric pressure reports from the Vaisala WXT ("weather transmitter").  And the programming was enhanced to begin storing 1-minute averages, minima and maxima as well as adding more parameters to the 6-minute data table.

In an oversight, the program was deployed with a satellite transmission platform id (PID) which has been used to test the program at AOML before deployment.  This proved to be a minor inconvenience for data processing and was corrected at the next programming update a month later.

data downloads, 2009-2010

This station was installed on March 19th, 2009 and its programming was updated on May 21st, 2009.  After this update, FACE adopted a routine of visits on roughly a monthly basis for accessing the ADCP and downloading its data via the cables that run into the main datalogger box.  During each visit, the datalogger's CF Memory card would be removed and replaced with another, blank card.  The old card's data would then be used to "patch" any holes in the station's satellite transmissions, and certain more granularly-timed data (10-minute averages and maxima/minina through May 21st, and 6-minute data thereafter) would become available for CHAMP/FACE analysis.

The next significant station equipment/programming update would take place on March 22nd, 2010.  ADCP and memory-card downloads in the period between May 2009 and March 2010 occurred as follows:

• June 5th, 2009
• July 7th, 2009
• July 31st, 2009
• August 28th, 2009
• September 24th, 2009
• October 13th, 2009
• November 16th, 2009
• January 15th, 2010
• March 11th, 2010

station goes offline, and is revived (again)

On November 3rd, 2009 I sent out an email reporting that the station had stopped transmitting again:

The PVGF1 Port Everglades station stopped transmitting this past weekend.  Its last transmission was sent at approximately 6:24pm local time on Saturday, October 31st.

I know you [Jack Stamates] were planning to visit the station after the cruise is over.  You might want to involve myself or Mike Shoemaker in the trip, or possibly, one of us Mikes might want to visit the station earlier than that, if we have the ability to open the box and permission to visit the Naval Base.

I'll look to see if there are any suspicious signs in the final transmissions but it looks like it's a clean break (i.e., no gradual power loss, no partial transmissions).  This may be a repeat of what happened last July.  Shoe fixed things that time, I think he may have just opened up the box and power-cycled the system.

Jack responded on November 5th, 2009 with the following update:

I pulled the power plug on the Campbell and the Sat transmiter, left the power off for a 30 sec then re inserted the plug...  Apparently this did not fix the problem.

Jack was able to visit the station again on November 16th, and on November 17th, 2009 I sent out the following update:

Jack visited the Port Everglades station again yesterday, and power-cycled the logger and transmitter.  He also swapped out the memory cards and brought me the retrieved card.  This didn't have any effect on transmissions (i.e., we are still offline, and have been since October 31st).  But I now have a memory card to answer some questions about the equipment.

First off, the datalogger seems to be perfectly ok.  It has been logging data nonstop since October 31st, right up until the card's retrieval yesterday.

Secondly, the transmitter is on, it is communicating with the logger, and it is continuing to acquire GPS fixes normally.  However, the error codes indicate that its failsafe mode has been tripped.  Basically it must have sensed conditions (voltage spike? temperature extreme?) that could have led to permanent damage, and it shut down its transmission systems.  Power-cycling will NOT reset the failsafe.  Instead, the "reset" button must be held down for at least 4 seconds, which will trigger a 60-second reset procedure.  It is entirely possible that this is all that is needed to get the station transmitting again.

Other considerations:  I currently have one other SAT-HDR-GOES transmitter, which I am testing on the roof.  It is not yet working.  I am also testing an updated pvgf1 logger program on the roof (among other things, this would add one of our old GE/Druck barometers, so I could sanity-check the air pressures returned by the WXT).  The new logger program (unlike the spare transmitter) IS working.

The bottom line is that our next logical step should probably be to do the transmitter-reset procedure on Jack's next visit.  I'd also like to visit the station to upgrade the programming and equipment, but I'm not ready yet.

Jack returned to Port Everglades and this time did a failsafe-reset on the satellite transmitter, which restarted the satellite transmissions.  I sent out the following update on November 19th, 2009:

Jack did a reset of the sat-hdr-goes transmitter at Port Everglades (pvgf1), and that seems to have done the trick.  We've received our first transmission from that station, with hopefully many more to come.  Good work, Jack!

low salinity readings, reliability of CT cleanings

On September 14th, 2009, Jack Stamates (FACE project) sent out an update about the Port Everglades CT.  He had noticed that the salinity readings had been unreasonably low, and in response he had contacted the dive company he was paying to do monthly instrument cleanings.

This resulted in an immediately resolution of the problem, and the CT salinity readings were once again reasonable.  It appears as though the contracted divers had not been adhering to the monthly CT cleaning schedule they'd agreed to.  Jack raised to issue to the awareness of the AOML dive collective and resolved to monitor the situation more closely from here on out.

regarding the accuracy of the station's barometer

[The following are excerpts of an email conversation between myself and Rex Hervey of NDBC.  When CHAMP first started feeding data from this station to NDBC, some questions were raised about the accuracy of its barometric pressure readings.  It so happened we had some similar questions about the WXT barometric pressures at LCIY2 at the same time.  In due course of time we determined that the pressure readings were all correct within instrument tolerances, and later (March 2010) we would add a redundant analog barometer to PVGF1 just to be sure.  Still, the model analyses shared by Rex are interesting enough to include on this blog.]

On June 25th, 2009, Rex wrote:

We are releasing the data from Port Everglades with the exception of pressure.  It looks like it is about one mb low.  We will continue to watch it.  If you can apply an offset, I can give you one after we have more time to look at it.  Picture is not up yet, but here is the web page: http://www.ndbc.noaa.gov/station_page.php?station=PVGF1

My reply on July 5th, 2009 was:

Can you tell me more about how you know the station's air pressure is off?  I assume you're comparing it with reports from surrounding stations but I'd like to know which ones.  The WXT520 weather transmitter is newly-calibrated and I'm not aware that we've had any problems with them in the past.

I could also put a standalone barometer out there for comparison purposes, although that will have to wait until we can coordinate another visit.

Rex followed up on July 6th, 2009 with:

It is compared to surrounding stations and numerical model analyses. Now that we have had a week or so to look at it, the bias is only about .3 to .4 mb low compared to model analyses.  That is within limits, so we will release it.

On August 12th I asked for another look at the situation, and Rex replied the same day with the following text and images:

I sent some plots comparing the baro reading with the NWS numerical model analyses.  Green Xs are from the model. One is over the last 2 months; the other is over the last month.  It appears as though it may be drifting closer to the model results over time.  Overall it is -.26 mb off, and more recently only -.15 mb off.  My initial evaluation after start up was only over a few observations.  I clearly should have given it more time.

One month.

Two months.